TY - JOUR AU1 - Almada, Marco AU2 - Maranhão, Juliano AB - Key Points Ongoing research projects, among them the Brazilian SPIRA and SoundCov initiatives, seek to diagnose Covid-19 and severe respiratory insufficiency through the analysis of voice recordings. The voice recordings may also be used to infer information about various personal traits, including some considered as sensitive by data protection law. The deployment of such apps may promote significant benefits in the context of a pandemic, since telemedicine avoids the risks of infection, both to potential patients and to the health-care professionals, as compared to presential consultation; those benefits, however, must be evaluated in light of the ethical and data protection concerns mapped in this article. The operation of voice-based medical apps involves various kinds of personal data, which means that those apps must follow the requirements imposed by Brazil’s General Data Protection Law (LGPD), such as the need for a legal basis for data processing and the purpose limitation of processing. The LGPD also provides a series of rights to users and other data subjects, such as the right to erasure and the right to information about the processing, which must be implemented by any diagnosis app. Introduction Our voices tell much about us, both through speech and non-verbal sounds, such as coughs. As an example, researchers have used voice recordings to identify whether a person has a respiratory disease, such as asthma or respiratory tract infections caused by common cold and flu.1 Since modern smartphones can collect audio samples that are good enough for this sort of diagnosis, some recent apps are deploying artificial intelligence (AI) techniques to predict whether an individual should seek hospital care or to assisting medical professionals in their evaluation of suspected Covid-19 cases. This article explores the ethical and legal issues that might arise from the use of voice data in the context of remote diagnosis and treatment of Covid-19, with particular attention to two applications currently under development in Brazil: SPIRA and SoundCov. Both apps follow the same general approach: a user records an audio of their voice, which is then analysed by machine learning algorithms,2 leading to a diagnosis that is provided to the user. However, each application uses a different medical and technological techniques. The SPIRA project, currently under development at the University of São Paulo, seeks to detect severe respiratory insufficiency associated with the SARS-CoV-2 virus,3 to indicate whether the user of the app must seek hospitalization.4 To obtain this diagnosis, the SPIRA app records the patient’s reading of a few pre-defined sentences.5 These recordings are analysed by a machine learning model trained to distinguish the voices of healthy persons from those of people afflicted with respiratory insufficiencies.6 SoundCov, an app developed by Fiocruz, Intel, and Instituto Butantan, trains a machine learning system to distinguish between the coughing sounds of a Covid-19-positive person and those of healthy people and people afflicted by other respiratory illnesses, such as pneumonia or tuberculosis.7 The application then combines the analysis of the coughing sounds with additional information about epidemiological variables and a patient’s health history, thus producing a final diagnosis.8 As of December 2020, the project is no longer operational,9 but similar approaches were proposed by other projects around the world.10 Developing the full potential of sound-based diagnosis requires paying attention to ethical and legal issues, such as those related to the quality of diagnoses, to the public health demands arising in the context of a pandemics, or to the possibility of misuse of a user’s personal data.11 Such risks stem from both the technical construction of diagnosis apps and their deployment in a sensitive field such as healthcare, requiring attention from app developers and users. The next section considers general issues related to the use of artificial intelligence in healthcare. After that, the article explores the legal treatment of voice data and the specific ethical and data protection issues that stem from remote voice diagnosis apps. In doing so, it provides a roadmap for future research on how to leverage the benefits from AI diagnosis and therapeutic decision-making while mitigating the ethical and data protection concerns related to such applications. The use of AI in health care In recent years, there were significant developments in the use of machine learning approaches to assist health care professionals (HCP) in medical diagnosis and clinical decision-making. Brazil is no exception, as local researchers have contributed to the development of applications using AI to classify clinical reports and deliver automated diagnosis.12 While the substantial health inequalities in the country mean that such technologies are still far from the lives of many Brazilians,13 artificial intelligence is not confined to academic environments, with applications both in the private14 and public15 healthcare. Although much of the excitement over AI reflects current hype,16 some applications have already reached a considerable degree of technological maturity, and the American FDA has approved at least 29 algorithms based on machine learning for clinical use.17 In some cases, machine learning systems are presented as achieving performance comparable to certified clinicians in practical tasks.18 However, automation does not necessarily require super-human performance: if a system shows ‘good enough’ performance on a particular task, then deploying it might be faster and more efficient than assigning a trained HCP for that task, freeing a professional for tasks that are not as easily automatable.19 Nevertheless, most of the tools are mainly intended not to replace, but to assist the HCP, as a form of augmented intelligence in the health care system.20 One form of augmentation comes from the use of remote diagnosis apps based on voice data. A key advantage of voice-based systems is that patient data can be collected through a smartphone app, rather than requiring specialized equipment and visits to a hospital or clinic. This advantage is particularly relevant in the context of a highly infectious disease such as Covid-19, which has extensively changed medical protocols for reducing contamination risk.21 Within the effort to avoid contamination, telemedicine,22 smartphones and apps have become essential devices to perform diagnosis and to assist patients.23 Remote tools that allow for diagnosis over the internet can thus be a preferable alternative to in-person visits. However, access to remote diagnosis can be uneven in Brazil: while almost 80 per cent of the Brazilian households are connected to the internet, many of these rely on precarious mobile connections, and even that might not be available to people based in remote locations.24 Voice-based diagnosis apps, such as SPIRA and SoundCov, can be used as tools for telemedicine during the pandemic. A user interested in obtaining a diagnosis would need to download the diagnosis app on their smartphone and then submit their voice samples for evaluation. Once the app obtains the required data, it produces an output: for SPIRA, whether a patient should seek health care; for SoundCov, a diagnostic of Covid-19. Hypothetically, those outputs may play various roles in the division of labour with the HCP: initial screening (triage), providing inputs to a decision by the HCP, providing a second opinion to the HCP, or even an automated decision that might or not be ratified by HCPs. Each use case for a remote diagnosis app leads to different ethical implications, related to the diagnosis procedure and the use of the app outputs. Before advancing a preliminary mapping of the ethical implications of such AI systems based on voice applied to health care, and considering that the protection of the patient’s personal data and privacy are among the main concerns regarding the use of AI in health care25 or regarding telemedicine in general,26 it is relevant to contextualize the audio-samples as voice data and as personal data. Voice data as personal data Voices play a broad range of functions, which are studied by fields such as linguistics27 and computer security.28 As a consequence, there are many definitions of ‘voice’ in common use,29 but, for the current discussion, this article will adopt the following definition: The voice is the output in the sound domain of the coordinated movements of an individual human talker’s speech organs, which is used principally but not exclusively by the talker for the acoustic encoding of linguistically meaningful utterances, and which after a period of exposure listeners may come to associate with that talker.30 Under this definition, references to ‘voice’ encompass not only speech, or the words said by a person, but also non-verbal sounds such as coughing and breathing sounds, as well as extraneous sounds produced by interferences on the output of such coordinated movements. Voice recordings may thus contain various sources of information:31 not just the ‘linguistic content’ of the uttered words and sentences, but also ‘voice characteristics’, such as the pitch or the nasality of an individual’s voice; ‘speech characteristics’, such as rhythm and accent; ‘non-speech human sounds’, such as coughs and breaths; and even ‘background sounds’ from a person’s environment. Information contained in voice recordings may be used for various purposes. Human beings can, in general, distinguish between two speakers based on their voice,32 and various software systems have been constructed for the same purpose.33 These technical developments allow for the use of voice as a biometric modality,34 that is, as a source of information for identifying a natural person through technological means, which demands specific caution and increased safeguards under Brazilian data protection law. The Brazilian General Data Protection Law (LGPD)35 defines sensitive personal data as [P]ersonal data on racial or ethnical origin, religious conviction, political opinion, association to a trade union or an organisation of religious, philosophical or political nature, data about health or sex life, genetic or biometric data, when linked to a natural person[.]36 Since some categories of data are frequently used as a cover for systematic discrimination and other forms of harmful behaviour towards groups and individuals,37 they are subject to specific legal requirements.38 According to the definition above, biometric data is sensitive personal data whenever it is linked to a natural person.39 While the LGPD itself does not define biometric data, technological practices consider that biometric data refers to attributes or features of a person that allow for their unique identification.40 Since voice data can be used for this kind of distinction,41 any voice data that is explicitly linked to a natural person should be treated as biometric data for the purposes of the LGPD. This is particularly relevant for the SoundCov app, which combines voice data with information about a patient’s clinical history—which, as a form of information about the health of a natural person, is covered by the LGPD’s definition of sensitive personal data42—,therefore linking its voice recordings with other data that allow for patient identification. Even if voice recordings are anonymized and therefore not associated with a specific natural person, they can still be associated with a speaker. It is technically possible, for example, to compare anonymized voice recordings from a remote diagnosis app with labelled recordings from other sources.43 Given this possibility of associating voice recordings with a natural person, voice data falls under the LGPD’s definition of personal data,44 even in the cases it is not directly used as biometric data. The immediate consequence of the classification of voice data as personal data is that any application that processes voice recordings, such as the remote diagnosis apps considered in this article, is subject to the requirements of the LGPD.45 However, data protection law is not limited to the voice data itself: since voice data can be associated with speakers, any inferences drawn from that data are also indirectly associated with them. Hence, inferences drawn from voice data must also be treated as personal data,46 except if suitably anonymized.47 Drawing inferences from voice data is, in fact, the main goal of the remote diagnosis apps under analysis in this article, as they use voice recordings to identify Covid-related conditions. However, the same recordings used for diagnosis can be used as grounds for various kinds of inferences. It has been claimed, for example, that computer systems can detect whether an individual is inebriated,48 or guess an individual’s socioeconomic status from elements such as vocabulary or modes of speech.49 Those inferences can be used to make decisions about that person,50 even in cases where the accuracy of such inferences is doubtful at best,51 and thus might end up having legal or otherwise meaningful consequences to the subject of the inference. The risks of inference-based decisions are even higher in the case for some forms of inference, such as those concerning racial or ethnic origin,52 sexual orientation,53 or mental health status.54 Since the main output of remote diagnosis apps—that is, the medical diagnosis itself—refers to the diagnosed individual’s health, it is, by itself, a form of sensitive personal data. If the voice recordings are used for inferences other than diagnosis, such as identifying patients or inferring their ethnical origin, the app will also give origin to further types of personal data, which might be covered by the definition of sensitive personal data and thus require additional care in their processing. Voice-based apps must thus attend to the requirements of data protection law, and the medical purposes of SPIRA and SoundCov require attention to the rules for sensitive personal data even if the voice recordings are not used for purposes beyond their primary goal. However, the privacy and data protection implications of voice data have not received much attention in the literature, with the exceptions focusing mostly on biometrics.55 This relative dearth of studies on voice privacy does not mean that the use or misuse of voice data poses little risk to individual rights.56 While SoundCov and SPIRA aim to use voice recordings as an instrument for medical diagnosis, respectively of Covid-19 and severe respiratory insufficiency, those recordings may also provide grounds for inferences that can intrude into the private lives of data subjects or otherwise harm them, for example by misgendering a person based on their voice. To provide an overview of the possible issues resulting from the use of voice data, the remainder of this article discusses ethical (the Section ‘Ethical implications of AI systems for voice-based diagnosis’) and data protection (the Sections ‘The use of personal data in voice-based diagnosis’ and ‘Data subject rights and voice-based diagnosis’) issues that can be identified in the proposed uses of SPIRA and SoundCov. Ethical implications of AI systems for voice-based diagnosis The use of voice-based AI systems for remote diagnosis raises questions that are related to various strands of ethical issues, which relate to current debates on the ethics of technology57 and medical ethics.58 Those issues can be perceived not just at the level of the treatment of individual patients, but also regarding public health decisions, such as the increased reliance on telemedicine.59 In this context, voice-based diagnosis apps will need to attend to those various concerns, which might require different—and even conflicting60—outcomes and methods. The primary approach to AI ethics has been the identification, assessment, and explanation of ethical principles affected by the deployment of such technology. A recent survey of more than 80 ethical reports or policy documents published by governmental and private entities around the globe has identified some convergence around the ethical principles of ‘transparency’ (regarding the use of AI and the explanation of its workings), ‘individual freedom and autonomy, fairness’ (non-discrimination), ‘non-maleficence, responsibility and privacy/data protection’.61 This set of principles bears some resemblance to the ‘principlism’ framework on bioethics,62 which adopts high-level principles of ‘autonomy, non-maleficence, beneficence and justice’.63 Nevertheless, this relative convergence between artificial intelligence principles and bioethics principles does not eliminate all of the uncertainty about how those principles should be interpreted when it comes to applications such as SPIRA and SoundCov. A recent attempt to bridge this gap64 has sought to identify ethical issues which are unique to AI algorithms in health care. If we look at AI as a source that provides evidence for decision-making, that evidence may be ‘inconclusive’:65 for example, a diagnosis app might not be able to provide a proper diagnosis to a user that is almost asymptomatic, as their voice might sound almost normal. The evidence provided by an AI may also be ‘inscrutable’, as patients lack oversight of the training data and decision criteria,66 or even ‘misguided’, if they are based on inadequate data, such as recordings of reduced quality.67 These factors may compromise the value of information derived from an AI system such as a remote diagnosis app, reducing its value as an epistemic tool. Even if a system is robust from an epistemic perspective,68 its use might still raise other ethical issues. From a normative perspective, the main concerns are that outcomes might be ‘unfair’,69 for example by disfavouring minorities in the outputs,70 or have unexpected ‘transformative’ effects in relationships such as the patient-HCP relation.71 Finally, ‘traceability’ is an overarching ethical outcome: since multiple agents are involved in the development, deployment, and use of AI systems, there is a ‘many hands’ problem72 that makes it difficult to pinpoint the sources of faults or biases within a system.73 If the goals of apps such as SPIRA and SoundCov are achieved, performing reliable diagnosis, the result would alleviate the costs and HCP labour required for diagnosing Covid-19 and respiratory insufficiency, while reducing exposition to the virus for both patients and HCPs. However, the potential advantages from the use of remote diagnosis apps must be weighed against both the general issues of healthcare AI described above and the specific issues related to telemedicine and the pandemic. The first set of issues raised by the use of remote diagnosis apps concerns their relevance to the public health responses to the pandemic. Given Covid-19’s infection patterns and effects, diagnosing and treating this disease requires significant resources, and healthcare systems all around the world have adopted trade-offs to avoid being overwhelmed.74 Some of these trade-offs are the result of attempts of HCPs to balance the interests of individual patients and the demands of the general response to the pandemic. For example, community health workers in Brazil had to adjust, and sometimes suspend, their visits to health-system users, both because of the increased risk of contagion and because of the risk of hostile reactions from pandemic denialists.75 In other cases, the trade-offs happen at the level of public policies, such as the preference for remote diagnosis and treatment whenever possible76 or the intensified use of personal data for tracking the pandemic.77 The usefulness of telemedicine in Brazil must be evaluated in light of the deep socioeconomic inequalities in the country. On the one hand, the availability of remote diagnosis and other tools of telemedicine allows patients to seek healthcare without exposing themselves, and potential hospital companions, to Covid-19. In the case of apps such as SPIRA and SoundCov, this could contribute to an earlier diagnosis, as users might be less inclined to postpone their testing until stronger symptoms appear. On the other hand, wealth inequality and the digital divide may reduce the access to telemedicine: while almost 80 per cent of the Brazilian households have access to the internet,78 about 74 per cent of the poorest households only use the internet through mobile devices, a number that falls to about 11 per cent among the wealthiest segments of the population.79 It follows that the telemedicine experience of a significant share of the Brazilian population may be negatively affected by slow connections and the download limits of mobile plans. Nevertheless, telemedicine was a growing practice in Brazil even before the pandemic.80 Another relevant dimension of telemedicine concerns its impact on the responsibility of HCPs. According to the Brazilian federal law that governs telemedicine during the pandemic, medical services provided through remote means are subject to the same normative and ethical standards that are applicable to in-person healthcare.81 In particular, patients must be informed of the limitations inherent to the medium of telemedicine,82 which include the possibility of reduced diagnosis accuracy.83 This demand, related to the patient’s autonomy, must also consider the patient’s inclination to accept a downgrade in accuracy to avoid exposure in the context of a pandemic. In the case of apps such as SPIRA and SoundCov, the remote diagnosis is not directly provided to a human HCP, neither it results in automated decisions, being instead an input for the user’s decision on whether to seek hospital care or medical orientation. Even so, depending on the degree of accuracy of the app or the degree of confidence, by the patient or by the HCP, in its predictions, some questions may be raised regarding the patient’s autonomy and also ethical and legal standards of medical responsibility,84 since the apps provide inputs that affect and influence human decisions to some extent, thus being a form of technological mediation of medical decisions.85 The final set of ethical issues raised in this article concerns the privacy of patients and the protection of their medical data during and after remote diagnosis.86 The patient’s trust in HCP confidentiality also becomes dependent on the security of the technical apparatus and the transparency about the processing of personal health data,87 both by physicians and those non-physicians involved in the data processing operations.88 Immediately connected with privacy—and patient autonomy—is the need to obtain informed consent not only to perform the medical intervention but also to the use of the technical apparatus. The remaining sections of this text approach these privacy and data protection issues through the lens of Brazilian data protection law, to show how the LGPD provides legal answers to several of the questions raised about the ethical use of remote diagnosis apps. The use of personal data in voice-based diagnosis Projects such as SPIRA and SoundCov strive to build applications that allow users to decide whether they should seek medical care for Covid-19. Those apps rely on voice data for generating a diagnosis, which means that their operation necessarily involves personal data, and the diagnosis itself is sensitive data that receives stronger protection under the LGPD.89 Furthermore, machine learning systems such as those at the core of SPIRA and SoundCov require a training process, in which the algorithms learn to distinguish between the voices of those who have been afflicted by Covid-19 and those who have not.90 Therefore, the operation of a remote diagnosis app will require a substantial volume of personal data, both from its immediate users and from the people used as a source for training data. In Brazil, the LGPD authorizes the processing of personal data, such as voice data, only if the processing operation has a legal basis.91 As we emphasize in the following paragraphs, some of these grounds, however, are not valid grounds for the processing of sensitive data, such as biometric data or information about an individual’s health.92 Since remote diagnosis apps depend on voice data—and, for SoundCov, additional information about the user—,it follows that the lawful operation of such apps is only possible if there are lawful bases for processing data both during the training of the machine learning system and during the diagnosis of a user.93 Only a few of the legal bases presented in the LGPD could ground the development and use of a remote diagnosis app.94 Among the possible options, both SPIRA and SoundCov rely on consent when collecting data:95 the training data is collected from volunteers96 who donate their voice recordings—and medical history, in the case of SoundCov—, and a diagnosis app can ask users whether they consent with the use of their data for diagnostic purposes, as well as other purposes that might be required. Since individuals are able, at least in theory, to consent to the processing of their health data,97 grounding remote diagnosis apps on data subject consent is prima facie possible. Consent is a suitable basis for data processing only if it manifests the data subject’s will98 regarding the processing of their data for specific purposes.99 In practice, various factors can impact the quality of data subject consent, potentially rendering it invalid as a basis for data processing. A first obstacle is that consent is invalid if it does not meet formal standards,100 which are more specific in the case of sensitive data.101 Consent is also invalid if it is defective.102 If users decide to rely on a diagnosis app because the only alternative to a remote diagnosis is no diagnosis at all — for example, because the local hospitals and testing centres are overloaded —, then their consent may be considered defective103 and is therefore inadequate as a basis for data processing. In healthcare research and practice, the gold standard for ensuing patient autonomy is ‘informed consent’, which is understood as the situation where patients are (i) provided with a forthright explanation that allows them to understand the consequences of the treatment or experiment; (ii) make an uncoerced decision to participate; and (iii) able to refuse or interrupt the treatment or experiment.104 In the case of voice-based diagnosis, a user can cease to use an app, while persons that volunteered their data for training the system can make use of their individual rights as data subjects to interrupt the use of their personal data.105 Coercion includes not only physical constraint but also covers situations in which moral factors strongly induce a decision, which are both covered by the LGPD.106 Nevertheless, the need to provide sufficient and contextual information about the treatment or experiment can be a challenge to remote apps such as SPIRA and SoundCov, which offer no opportunities for personal dialogue between users and HCPs. The LGPD establishes that data subjects have the right to obtain various types of information about the processing of their personal data, such as the specific purposes of processing and the form and duration of the processing operation.107 If data subjects are not provided with this information, or the information is misleading or abusive, then their consent is deemed void.108 However, machine learning applications are notoriously opaque to external observers,109 which poses additional difficulties to the task of providing users with the information they need to provide informed consent.110 Recognizing this difficulty, some recent studies have adopted approaches such as dynamic consent and cascade consent,111 in which users do not provide consent solely at the beginning of a data processing operation, but stay in touch with the changes in the data processing models. By following a similar approach, app developers could build a trust relationship with their users, thus avoiding or mitigating the risks of their exercising the right to withdraw their consent to future data processing operations. Data controllers112 of remote diagnosis apps must thus ensure that data subjects provide valid consent to the processing of their personal data,113 or rely on another legal basis for data processing. When it comes to the core functionality of such an app—that is, the diagnosis of a user—,there is no real alternative to consent: even if a data controller finds another legal basis, the operation of a diagnosis app requires the cooperation of the user. Otherwise, the potential user might refuse to install the app or provide voice samples to it, unless otherwise compelled by law.114 This is not to say that consent is the only legal basis for processing that is relevant for voice diagnosis apps. For instance, the development of a voice-based diagnosis app could be sped up by the use of recordings of healthy people made before the pandemic, such as the various recordings of Brazilian Portuguese speakers available at the Museum of the Portuguese Language in São Paulo, Brazil.115 These recordings, made before the pandemic, could hypothetically be used as a source of healthy voices for training a machine learning model for remote diagnosis. However, the people who provided the recordings did not offer consent to this hypothetical use, only to the educational purposes for which the museum originally recorded their voices. Furthermore, it may be unfeasible in practice to go after each data subject and request the consent for this new use of their voices, leading to the question of whether there are any legal bases that would authorize the reuse of voice data for a beneficial healthcare solution in the absence of consent from the speakers. As long as the original data was lawfully collected, its reuse would be possible under one of the lawful bases provided by the LGPD: the processing of personal data by a research institution for the purpose of research, provided that the data is anonymized to the extent possible.116 There are, however, significative limitations to this alternative. First, it is only available to public or non-profit institutions that are dedicated to research.117 Secondly, the possibility of inferring biometrics from voice samples reduces the possibilities for anonymization of personal data. Thirdly, the LGPD does not allow for legitimate interest as grounds for the processing of health data. Since these limitations may hinder the development of beneficial research or applications employing AI, it has been suggested that such issues point to a tension between individual data protection and common goods such as public health. This conflict could be addressed through a broader view of the values underlying data protection, which should be balanced with the responsible use of data within a trustworthy AI setting and insights from the ethics of medical data donation.118 Legal bases other than consent are also relevant because voice data can be deployed to purposes other than diagnosis.119 The LGPD subjects any reuse of personal data to the principle of ‘purpose specification’: the purposes of data processing must be legitimate, specific, explicit, and informed to the data subject; also, any new purposes cannot be incompatible with the purposes of the original data processing.120 In the case of remote diagnosis apps based on voice, this means that any additional use of voice data, beyond the original diagnosis, must not be incompatible with the purpose of Covid-19 diagnosis—for example, by exposing users to additional risk of contagion—and must, itself, have a legal basis and meet the requirements imposed by the LGPD to data processing in general. Since the consent given to an app such as SPIRA or SoundCov is narrowly defined, in light of the conditions for informed consent,121 it is not automatically extended to other purposes. Beyond the need to control the purpose and the legal grounds of data processing, data controllers have additional duties regarding personal data. According to the LGPD, data processing shall be terminated whenever the purposes of processing are achieved, whenever data becomes irrelevant to the purposes of the processing, after the end of the processing period,122 or in response to a communication by the data subject or a determination by the data processing authority.123 Once that happens, data controllers must delete all personal data, but they are allowed to retain data in a few specific cases.124 In the case of apps such as SPIRA and SoundCov, these duties125 apply not just to the diagnosis provided to individual users and the personal data collected from them, but also to the personal data used to train the machine learning algorithm used for diagnosis.126 Data controllers and processors are liable for damages ensuing from processing that does not comply with data protection law.127 This liability applies not just to cases in which a data controller or processor fails to comply with the LGPD, but also whenever they fail to ensure information security for the processed data.128 In the case of the deployment of apps like SPIRA and SoundCov, failure to secure personal data might end up providing third parties with access to the diagnoses of data subjects, or even to the voice recordings and medical information used to produce this diagnosis. Seeing that security risks and the possibility of inappropriate use of personal data are inherent to data processing, the LGPD has established a general duty to data controllers and processors: these actors are required to adopt security, technical, and administrative measures to prevent any form of unauthorized access or inadequate or unlawful processing of personal data.129 Such measures must be observed from the early stages of the conception of a product or service, such as a remote diagnosis app, and also during its execution,130 and the security of personal data must be ensured even after the termination of processing.131 This approach, known internationally as ‘data protection by design’,132 addresses the risks that stem from personal data processing by adopting measures that are adequate for eliminating or mitigating those risks in the specific context of a data processing application. Data subject rights and voice-based diagnosis By mandating data protection by design, the LGPD adopts an approach of systemic protection,133 in which the technical system itself must incorporate measures that ensure the protection of personal data. Data protection requires in turn the implementation of a series of data subject rights present in the LGPD, such as the right to obtain information about the processing,134 to request the erasure of data processed on the grounds of their previous consent,135 and not to have their sensitive personal data used for risk selection when contracting a health care plan.136 As a result, voice-based apps for remote diagnosis—as well as other systems for processing personal data— must incorporate technical, security, and organizational measures that allow data subjects to exercise their data protection rights. In the context of a remote diagnosis app based on voice, some data protection rights become particularly relevant. According to the LGPD, the right to access information about data processing is only fulfilled if this information is provided in a ‘clear, adequate, and ostensible format’,137 a standard that must be evaluated in light of the resources available to the data subject and the characteristics of a data processing operation. For remote diagnosis, the relevant standard is that of informed consent, which means that software designers might need to adopt specific technical approaches to ensure that users and volunteers can understand how and why the system uses their data.138 Valid consent to the use of a remote diagnosis app also requires the enforcement of other rights, such as the rights to understand the consequences of refusing consent, to obtain information about data processing before offering consent139 and to be informed before any changes to the purposes of the processing.140 Finally, informed consent also requires the possibility of withdrawing consent after the start of the treatment or experiment.141 If a data subject invokes this right, her or his personal data can be no longer processed, but previous data processing does not lose its validity,142 which means that any personal data provided to SPIRA or SoundCov, or generated during the operation of these apps, is still lawful until the data controller receives a deletion request.143 This combination between the rights to the deletion of personal data and withdrawal of consent ensures that data subjects can interrupt their relation with the app without being penalized for their decision. Once a remote diagnosis app is trained on the data provided by volunteers, it can be used to diagnose users without recourse to a physician or other healthcare professional.144 This diagnosis is produced through the application of the trained machine learning model to the information provided by its user, which means that the diagnosis is the result of automated processing of personal data. Therefore, a diagnosis produced by an app such as SPIRA and SoundCov is an instance of automated decision-making, which labels users as Covid-positive or -negative based on the evaluation of their voice data.145 The LGPD provides data subjects with a right that can be opposed to automated decision-making: the right to request the review of automated decisions: A data subject has the right to request the review of decisions made solely on the basis of automated data processing which affect her or his interests, including decisions aimed at defining her or his personal, professional, consumption, and credit profile or the aspects of her or his personality.146 By definition, a review is an ex post response to decision-making, that is, an evaluation of a decision that has already happened, rather than an anticipatory measure.147 Therefore, the LGPD does not specify situations in which automated decision-making is forbidden, but other forms of data processing would be lawful.148 It also does not specify ‘who’ is supposed to perform the review: the original version of LGPD Article 20 required that any reviews had to be performed by a human, but this requirement was removed by subsequent law.149 Nevertheless, since there are few situations, under the current state of the art, in which a useful review can be performed through automated means, requesting a review is, in practice, a request for ‘human intervention’ in the decision-making process.150 The LGPD establishes a single limitation to the right to review: one can only request a review of an automated decision that impacts her or his interests. For a remote diagnosis app, this condition is not usually satisfied: if only the user has access to the app diagnosis, then the only impact the diagnosis produces is on that user’s decision to seek medical assistance or not. In this case, the review is straightforward, as the data subject can simply decide to seek an evaluation at a clinic or a hospital.151 However, the right to review might be relevant if the app diagnosis is used for any purposes beyond informing individual decisions. As of December 2020, no such applications have been proposed, but an example would be a situation in which the remote diagnosis is used as a tool for public policy. If users are forced to quarantine after receiving a positive diagnosis from the app, then they would have a right to request a review, since the automated decision that labelled them as positive for Covid led to restrictions of their freedom of movement. If the quarantine order combines the information from the app with inputs from a human, such as an evaluation by HCPs, the LGPD right to review would not be applicable, since the human involvement means that the decision is not ‘solely’ based on automated processing. But, if a human reviewer lacks the means to change the decision made by the automated system—for example, in situations that require quick decisions152—,any human intervention will amount to a confirmation of the automated decision.153 In such cases, nominal human involvement does not exclude the possibility of review of an automated decision.154 For a decision based on data provided by a Covid-19 diagnosis app, meaningful review means that a human—ideally a healthcare professional—should be able to analyse the factors that led an application to a particular decision and, if needed, override them or refer the analysis to a human specialist. Thus it will be possible to retain two conditions for meaningful human control over automated systems:155 the human reviewer will be able to adjust the decision in order to ‘track’ the relevant moral and circumstantial factors, while external observers will be able to ‘trace’ the outcome of the diagnosis, and any decisions based on it, to at least one human in the decision chain—either those responsible for the initial system development or the human reviewer that decided to substitute their reasoning for the original decision logic. The LGPD establishes two additional requirements for automated decision-making systems, which are meant to support oversight of decision-making systems. First, it establishes a ‘right to an explanation’:156 data controllers are required to provide clear and adequate information about the criteria used for a particular decision. With this information in hand, data subjects would be able to make informed choices about whether to request a review, while reviewers would be better equipped to identify why a decision produced negative results.157 But, since trade secrets and industrial secrets may limit the extent of data controller disclosure, the LGPD provides an alternative mechanism. Whenever those forms of secrecy are invoked as a reason not to provide an explanation, the national data protection authority can audit the processing to evaluate whether it displays discriminatory behaviour.158 This authority may also demand that data controllers produce Data Protection Impact Assessments that describe the risks from data processing and the measures and safeguards adopted in response to those risks.159 By attributing these powers to the data protection authority,160 the LGPD seeks to ensure the possibility of review and oversight over automated decisions even in situations that render individual-level explanations unfeasible or impractical. The full set of measures and safeguards for the review of automated decision-making is not required for the intended use of remote diagnosis apps: helping users to decide whether they should seek hospital care. Still, the rights laid down in the LGPD can protect users from the misuse of their diagnosis and personal data, both by the app itself and by third parties who gain access to the data. Therefore, the design of apps such as SPIRA and SoundCov must ensure that users have full access to their data protection rights. Concluding remarks Remote diagnosis based on voice data may be a useful tool in the fight against Covid-19 and future pandemics, as it allows patients to obtain an evaluation of their condition before they even leave their homes, thus minimizing the exposure risks for a patient and avoiding overload of hospitals and clinics. To fulfil this potential, any diagnosis app should be carefully evaluated to define its role in diagnosis and therapeutic decision-making, given the possible effects of wrongful diagnosis and the need to account for data protection concerns. This article has outlined some of the ethical and data protection implications of voice-based AI systems for diagnostic and therapeutic decision-making in the context of a pandemic. Based on the limited existing literature on voice biometrics, and the more abundant literature on medical ethics, the article has presented an initial mapping of ethical and legal issues for the deployment of diagnosis apps against the current Covid-19 pandemic. Given the urgency of the pandemic challenge, further development of the apps should seek close collaboration with ethical and legal specialists, both to prevent harms that are already identifiable and to map potential medium- and long-term risks from voice-based diagnosis apps. Therefore, despite the many open questions regarding the medical use of voice data, it seems prudent to move forward with research on this field. Juliano Maranhão is a professor of Law at the University of São Paulo, where Marco Almada obtained his law degree. That university hosted the SPIRA project, which is discussed in the article. However, neither of the authors is directly involved in SPIRA. Marco Almada’s work on this article was partially funded by a Fundación Carolina doctoral grant. Juliano Maranhão’s work on this article was partially funded by FAPESP/IBM’s funding of C4AI-USP (Proc. 2019/07665-4). The authors thank the anonymous reviewers for their comments on previous versions of this article. 1 Jacob Leon Kröger, Otto Hans-Martin Lutz and Philip Raschke, ‘Privacy Implications of Voice and Speech Analysis – Information Disclosure by Inference’ in Michael Friedewald and others (eds), Privacy and Identity Management. Data for Better Living: AI and Privacy: 14th IFIP WG 9.2, 9.6/11.7, 11.6/SIG 9.2.2 International Summer School, Windisch, Switzerland, August 19–23, 2019, Revised Selected Papers (Cham: Springer International Publishing 2020) 248. 2 Machine learning is an approach to the construction of artificial intelligence systems that use computation to discover regularities in data, and then use those regularities as the basis for predictions, categorization, or decision-making. See: Joanna J Bryson, ‘The Artificial Intelligence of the Ethics of Artificial Intelligence: An Introductory Overview for Law and Regulation’ in Markus D Dubber, Frank Pasquale and Sunit Das (eds), The Oxford Handbook of Ethics of Artificial Intelligence (Oxford, New York: OUP 2020). 3 Alexander E Gorbalenya and others, ‘The Species Severe Acute Respiratory Syndrome-Related Coronavirus: Classifying 2019-NCoV and Naming It SARS-CoV-2’ (2020) 5 Nature Microbiology 536. 4 Estudo SPIRA, ‘Doe sua voz e ajude no diagnóstico da Covid-19’ (2020) accessed 15 October 2020. 5 Denise Cassati, ‘Ferramenta Computacional Detecta Doença Por Meio Da Variação Da Voz’ Jornal da USP (São Paulo, 19 June 2020) accessed 4 December 2020. 6 Ibid. 7 ‘Sistema de inteligência artificial reconhece a tosse do paciente com Covid’ G1 (2 July 2020) accessed 4 December 2020. 8 ‘Fiocruz cria aplicativo para identificar tosse de paciente com covid-19’ EBC Rádios (10 August 2020) accessed 4 December 2020. 9 The project website, formerly at , is no longer functional as of December 4, 2020. The latest version of the site that is available at Wayback Machine () dates from 11 November 2020, and the link for collecting training data () no longer accepts submissions. 10 See, eg the Covid-19 Sounds App accessed 4 December 2020, project at the University of Cambridge. 11 Kröger, Lutz and Raschke (n 1). 12 See, eg Michel Oleynik, Marcelo Finger and Diogo FC Patrão, ‘Automated Classification of Pathology Reports’ (2015) 216 Studies in Health Technology and Informatics 1040; Ramon Pires and others, ‘A Data-Driven Approach to Referable Diabetic Retinopathy Detection’ (2019) 96 Artificial Intelligence in Medicine 93; Eduardo Valle and others, ‘Data, Depth, and Design: Learning Reliable Models for Skin Lesion Analysis’ (2020) 383 Neurocomputing 303. 13 Celia Landmann-Szwarcwald and James Macinko, ‘A Panorama of Health Inequalities in Brazil’ (2016) 15 International Journal for Equity in Health 174. 14 Bruno Romani, ‘Einstein e USP criam inteligência artificial para detectar coronavírus’ Estadão - Link (9 April 2020) accessed 4 December 2020. 15 ‘Primeiro tomógrafo do Brasil com inteligência artificial e atendimento 100% SUS é instalado na Bahia’ (Sesab - Secretaria da Saúde do Estado da Bahia) accessed 5 December 2020; Pedro Ivo Oliveira, ‘SUS pretende usar inteligência artificial para agilizar atendimentos’ Agência Brasil (Brasília, 12 November 2019) accessed 5 December 2020. 16 Thomas Ploug and Søren Holm, ‘The Four Dimensions of Contestable AI Diagnostic - A Patient-Centric Approach to Explainable AI’ (2020) 107 Artificial Intelligence in Medicine. 17 Stan Benjamens, Pranavsingh Dhunnoo and Bertalan Meskó, ‘The State of Artificial Intelligence-Based FDA-Approved Medical Devices and Algorithms: An Online Database’ (2020) 3 npj Digital Medicine 1. 18 Andre Esteva and others, ‘Dermatologist-Level Classification of Skin Cancer with Deep Neural Networks’ (2017) 542 Nature 115. 19 Michael J Rigby, ‘Ethical Dimensions of Using Artificial Intelligence in Health Care’ (2019) 21 AMA Journal of Ethics 121. 20 Elliott Crigger and Christopher Khoury, ‘Making Policy on Augmented Intelligence in Health Care’ (2019) 21 AMA Journal of Ethics 188. 21 Brasil, ‘Orientações Para Manejo de Pacientes Com Covid-19’ (Ministério da Saúde 2020) accessed 4 December 2020. 22 Judd E Hollander and Brendan G Carr, ‘Virtually Perfect? Telemedicine for Covid-19’ (2020) 382 New England Journal of Medicine 1679. 23 Gabriele Cervino and Giacomo Oteri, ‘COVID-19 Pandemic and Telephone Triage before Attending Medical Office: Problem or Opportunity?’ (2020) 56 Medicina 250. 24 Martha Maria Prata- Linhares and others, ‘Social Distancing Effects on the Teaching Systems and Teacher Education Programmes in Brazil: Reinventing without Distorting Teaching’ (2020) 46 Journal of Education for Teaching 554. 25 Ploug and Holm (n 16). 26 Benedict Stanberry, ‘Legal Ethical and Risk Issues in Telemedicine’ (2001) Computer Methods and Programs in Biomedicine. 27 Dominic Watt, Peter S Harrison and Lily Cabot-King, ‘Who Owns Your Voice? Linguistic and Legal Perspectives on the Relationship between Vocal Distinctiveness and the Rights of the Individual Speaker’ (2020) 26 International Journal of Speech Language and the Law 137. 28 Michael Fairhurst, Biometrics: A Very Short Introduction (Oxford, New York: OUP 2018). 29 Watt, Harrison and Cabot-King (n 27) 139. The diversity of meanings for ‘voice’ becomes even greater when one considers the metaphorical senses that might be ascribed, such as the unique literary ‘voice’ of a writer. 30 Ibid 144. 31 Kröger, Lutz and Raschke (n 1) 244. 32 Watt, Harrison and Cabot-King (n 27) 144. 33 Kröger, Lutz and Raschke (n 1) 243. 34 John Petersen, ‘The Complexity of Consent and Privacy in Biometrics – Worldwide’ (2019) 2019 Biometric Technology Today 5. 35 Brazil, Federal Law 13.709/2018 (General Data Protection Law (LGPD)) [2018]. 36 LGPD, art 5, II. 37 Frederick Schauer, Profiles, Probabilities, and Stereotypes (Cambridge, Mass.: Belknap Press 2006) 150. 38 LGPD, chap II, arts 11–13. See the Sections ‘The use of personal data in voice-based diagnosis’ and ‘Data subject rights and voice-based diagnosis’ of this article for a discussion of the consequences of these articles for the processing of sensitive data in remote diagnosis apps. 39 LGPD, art 5, II. 40 Even if not always successfully: see Fairhurst (n 28) 7. 41 Watt, Harrison and Cabot-King (n 27) 147. 42 LGPD, art 5, II (seen 36). 43 LGPD art 12 establishes that anonymized data are not considered personal data, unless the anonymization process can be reversed through reasonable efforts, such as the comparative approach from the example in the main text. While LGPD art 12, s 1, specifies that reasonableness must be evaluated through objective criteria, the determination of standards for what counts as safe anonymization is left to the National Data Protection Authority (ANPD) (Brazil, Federal Decree 10.474 [2020], Annex I, art 4, II, b), which has not yet offered binding rules or technical guidelines for anonymization. On the current (lack of) activity of this authority, see n 160. 44 LGPD, art 5, I, defines personal data as ‘information associated to an identified or identifiable natural person’. 45 Which are examined in the Sections ‘The use of personal data in voice-based diagnosis’ and ‘Concluding Remarks’ of this article. 46 For anonymization standards, see LGPD, art 12. 47 LGPD, art 12. 48 Kröger, Lutz and Raschke (n 1) 247. 49 Ibid 250. 50 For example, an evaluation of a person’s voice for traces of inebriation may be used as an instrument to combat drunk driving, while inferences of socioeconomic status from voice may be used as inputs for price discrimination. 51 Os Keyes, ‘The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition’ (2018) 2 Proceedings of the ACM on Human-Computer Interaction 1. 52 Allison Koenecke and others, ‘Racial Disparities in Automated Speech Recognition’ (2020) 117 Proceedings of the National Academy of Sciences 7684. 53 S Sulpizio and others, ‘Auditory Gaydar: Perception of Sexual Orientation Based on Female Voice’ (2020) 63(1) Language & Speech 184. 54 Kröger, Lutz and Raschke (n 1) 249. 55 See, eg Andreas Nautsch and others, ‘Preserving Privacy in Speaker and Speech Characterisation’ (2019) 58 Computer Speech & Language 441; Piergiorgio Vittori, ‘Ultimate Password: Is Voice the Best Biometric to Beat Hackers?’ (2019) 2019 Biometric Technology Today 8; Yaowei Han and others, ‘Voice-Indistinguishability: Protecting Voiceprint in Privacy-Preserving Speech Data Release’, Proceedings of the IEEE International Conference on Multimedia & Expo 2020 (2020) accessed 9 June 2020. 56 Kröger, Lutz and Raschke (n 1). 57 See, eg Brent Daniel Mittelstadt and others, ‘The Ethics of Algorithms: Mapping the Debate’ (2016) 3 Big Data and Society; AI HLEG, ‘Ethics Guidelines for Trustworthy AI’ (European Commission 2019); Christoph Bartneck and others, ‘Application Areas of AI’ in Christoph Bartneck and others (eds), An Introduction to Ethics in Robotics and AI (Cham: Springer International Publishing 2021); Rafael A Calvo and others, ‘Supporting Human Autonomy in AI Systems: A Framework for Ethical Enquiry’ in Christopher Burr and Luciano Floridi (eds), Ethics of Digital Well-Being: A Multidisciplinary Approach (Cham: Springer International Publishing 2020). 58 See, eg Ploug and Holm (n 16); Irene Y Chen, Peter Szolovits and Marzyeh Ghassemi, ‘Can AI Help Reduce Disparities in General Medical and Mental Health Care?’ (2019) 21 AMA Journal of Ethics 167; AI Tauber, Patient Autonomy and the Ethics of Responsibility (MIT Press 2005)[AQ] ‘Tauber 2005’ in fn 58; Julian Savulescu, ‘Bioethics: Why Philosophy Is Essential for Progress’ (2015) 41 Journal of Medical Ethics 28; Ruth Faden, Justin Bernstein and Sirine Shebaya, ‘Public Health Ethics’ in Edward N Zalta (ed), The Stanford Encyclopedia of Philosophy (Stanford University 2020)[AQ] ‘Faden et al 2020’ in fn 58. 59 Hollander and Carr (n 22). 60 As an example, public health imperatives may prevent the deployment of the best possible levels of individual health care under certain circumstances. 61 Anna Jobin, Marcello Ienca and Effy Vayena, ‘The Global Landscape of AI Ethics Guidelines’ (2019) 1 Nature Machine Intelligence 389. 62 Brent Mittelstadt, ‘Principles Alone Cannot Guarantee Ethical AI’ (2019) 1 Nature Machine Intelligence 501, 501. 63 Tom L Beauchamp and James F Childress, Principles of Biomedical Ethics (8th edn, Oxford, New York: OUP 2019). 64 Jessica Morley and others, ‘The Ethics of AI in Health Care: A Mapping Review’ (2020) 260 Social Science & Medicine. 65 Ibid 3. 66 Ibid 4. 67 Ibid. 68 On robustness in machine learning and scientific practice, see, among others, Leif Hancox-Li, ‘Robustness in Machine Learning Explanations: Does It Matter?’, Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Association for Computing Machinery 2020) accessed 28 January 2020; Mieke Boon, ‘Understanding Scientific Practices: The Role of Robustness Notions’ in Léna Soler and others (eds), Characterizing the Robustness of Science: After the Practice Turn in Philosophy of Science (Netherlands, Springer 2012) accessed 16 October 2019; Simón C Smith and Subramanian Ramamoorthy, ‘Counterfactual Explanation and Causal Inference in Service of Robustness in Robot Control’, Proceedings of the 10th IEEE International Conference on Development and Learning (IEEE 2020) accessed 25 September 2020; Ryan Muldoon, ‘Robust Simulations’ (2007) 74 Philosophy of Science 873. 69 Morley and others (n 64) 4. 70 Ziad Obermeyer and others, ‘Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations’ (2019) 366 Science 447. 71 Morley and others (n 64) 5. 72 Merel Noorman, ‘Computing and Moral Responsibility’ Stanford Enciclopedia of Philosophy (2018). 73 Morley and others (n 64) 6. 74 Alexandra M Dunham, Travis N Rieder and Casey J Humbyrd, ‘A Bioethical Perspective for Navigating Moral Dilemmas Amidst the COVID-19 Pandemic’ (2020) 28 The Journal of the American Academy of Orthopaedic Surgeons 471. 75 Gabriela Lotta and others, ‘Community Health Workers Reveal COVID-19 Disaster in Brazil’ (2020) 396 The Lancet 365. 76 Hollander and Carr (n 22). 77 Jéssica Andrade Modesto and Marcos Ehrhardt Junior, ‘Danos colaterais em tempos de pandemia: preocupações quanto ao uso dos dados pessoais no combate a COVID-19’ (2020) 8 Revista Eletrônica Direito e Sociedade - REDES 143; Bethania de Araujo Almeida and others, ‘Preservação da privacidade no enfrentamento da COVID-19: dados pessoais e a pandemia global’ (2020) 25 Ciência & Saúde Coletiva 2487. 78 Linhares and others (n 24). 79 Angelica Mari, ‘Smartphones Become a Lifeline for Poor Brazilians’ (ZDNet, 11 November 2020) accessed 5 December 2020. 80 Angélica Baptista Silva and others, ‘Three Decades of Telemedicine in Brazil: Mapping the Regulatory Framework from 1990 to 2018’ (2020) 15 PLOS ONE e0242869. 81 Brazil, Federal Law 13.989/2020 [2020], art 5. 82 Ibid, art 4. 83 Stanberry (n 26). 84 On medical liability in Brazilian law, see Miguel Kfouri Neto, Responsabilidade Civil Do Médico (Nova Edição, Revista dos Tribunais 2019); Filipe Antônio Marchi Levada, ‘A responsabilidade civil do médico durante a pandemia’ (Consultor Jurídico, 30 April 2020) accessed 5 December 2020. 85 Federal Law 13.989/2020 (n 81), art 3. 86 Stanberry (n 26). 87 Timothy M Hale and Joseph C Kvedar, ‘Privacy and Security Concerns in Telehealth’ (2014) 16 AMA Journal of Ethics 981. 88 BM Dickens and RJ Cook, ‘Legal and Ethical Issues in Telemedicine and Robotics’ (2006) 94 International Journal of Gynecology & Obstetrics 73. 89 See text to n 36, and LGPD, art 5, II. 90 On the relevance of data for the construction of machine learning systems, see David Lehr and Paul Ohm, ‘Playing with the Data: What Legal Scholars Should Learn About Machine Learning’ (2017) 51 University of California, Davis Law Review 653. 91 LGPD, art 7. 92 LGPD, art 11. 93 Data processing for strictly academic purposes is excluded from the scope of the LGPD (art 4, II, b). But, even in this case, the law explicitly states that arts 7 and 11, which require a legal basis for processing, are still applicable. 94 As an example, the connection between credit protection (LGPD, art 7, X) and the diagnosis of Covid-19 is tenuous at best. For a comprehensive analysis of the legal bases for data processing in the LGPD, see Chiara Spadaccini de Teffé and Mario Viola, ‘Tratamento de dados pessoais na LGPD: estudo sobre as bases legais’ (2020) 9 civilistica.com 1. 95 LGPD, arts 7, I, and 11, I. 96 See the SPIRA project website (n 4). 97 For European discussions which are directly relevant to the Brazilian context, see European Data Protection Board, ‘Guidelines 03/2020 on the Processing of Data Concerning Health for the Purpose of Scientific Research in the Context of the COVID-19 Outbreak’ (21 April 2020); Patrik Hummel, Matthias Braun and Peter Dabrock, ‘Data Donations as Exercises of Sovereignty’ in Jenny Krutzinna and Luciano Floridi (eds), The Ethics of Medical Data Donation (Cham: Springer International Publishing 2019). 98 LGPD, art 8, caput: ‘The consent from Article 7, I, must be provided in writing or by any other means that demonstrate the data subject’s manifestation of will’. 99 LGPD, art 8, s 4: ‘Consent must refer to specific purposes, and generic authorisations of personal data processing shall be considered void’. 100 Such as LGPD, art 8, s 1: ‘If consent is provided in writing, it must be presented in a clause that is separated from the other contractual clauses’. The following paragraphs of this paper continue the discussion on formal requirements in the specific contexts of voice apps. 101 LGPD, art 11, I, extends the requirement from art 8, s 1 (n 100) by requiring specific and separate consent to the specific purposes of the processing of sensitive personal data. 102 LGPD, art 8, s 3. 103 Regarding moral coercion as a defect of consent in Brazilian law, see Caio Mário da Silva Pereira and Maria Celina Bodin de Moraes, Introdução ao Direito Civil (30th edn, Rio de Janeiro: Forense 2017) ch XVIII. 104 Joaquím Clotet, ‘O Consentimento Informado nos Comitês de Ética em Pesquisa e na Prática Médica: Conceituação, Origens e Atualidade’ (2009) 3 Revista Bioética accessed 6 December 2020. 105 See the Section ‘Data subject rights and voice-based diagnosis’. 106 LGPD, art 8, s 3: ‘Data processing based on defective consent is prohibited’. 107 LGPD, art 9. 108 LGPD, art 9, s 1. 109 Jenna Burrell, ‘How the Machine “Thinks”: Understanding Opacity in Machine Learning Algorithms’ (2016) 3 Big Data & Society 1. 110 Steffen Augsberg and Ulrich von Ulmstein, ‘Requisitos de Consentimento Modificados: O Direito de Proteção de Dados Pode Aprender Com o Direito Da Saúde?’ in Ricardo Campos, Georges Abboud and Nelson Nery Jr (eds), Proteção de dados e regulação (São Paulo: Thomson Reuters Brasil 2020). On the challenges of explaining artificial intelligence systems, see also Adrien Bibal and others, ‘Legal Requirements on Explainability in Machine Learning’ [2020] Artificial Intelligence and Law. 111 Augsberg and von Ulmstein (n 110). 112 LGPD, art 5, VI, defines a data controller as a ‘natural or legal person, governed by public or private law, who is in charge of the decisions regarding personal data processing’. 113 LGPD, art 8, s 2. 114 Brazilian Constitution, art 5, II. 115 Museu da Língua Portuguesa, Exposição Principal < https://www.museudalinguaportuguesa.org.br/memoria/exposicao-principal/> accessed 4 December 2020. 116 LGPD, art 7, IV (general rule); art 11, II, c (sensitive data). 117 LGPD, art 5, XVIII. 118 Hummel, Braun and Dabrock (n 93). 119 See the Section ‘Voice data as personal data’ of this article. 120 LGPD, art 6, I. 121 See, eg the SPIRA website and data collection form (n 4). 122 Bart Custers, ‘Click Here to Consent Forever: Expiry Dates for Informed Consent’ (2016) 3 Big Data & Society. It follows that reuse without new consent is still bound by the expiry date of the initial consent, which might be a problem for ephemeral data sources, such as customer service recordings, but less so for more perennial recordings, such as recordings generated for a museum. 123 LGPD, art 15. 124 LGPD, art 16. 125 For a general treatment of the legal issues stemming from the termination of data processing, see Gisela Sampaio da Cruz Guedes and Rose Melo Venceslau Meireles, ‘Término do Tratamento de Dados’ in Gustavo Tepedino, Ana Frazão and Milena Donato Oliva (eds), Lei Geral de Protecao de Dados Pessoais e Suas Repercussoes no Direito Brasileiro (São Paulo: Thomson Reuters Brasil 2019). 126 Since SoundCov’s website is no longer available and there is no information about the project on Fiocruz’s website (see n 9), there is no publicly accessible information on how the SoundCov data controller has performed their duties regarding the termination of treatment. 127 LGPD, art 42. 128 LGPD, art 44. 129 LGPD, art 46. 130 LGPD, art 46, s 2. 131 LGPD, art 47. 132 European Data Protection Board, ‘Guidelines 4/2019 on Article 25 on Data Protection by Design and by Default’ (20 October 2020). 133 Wolfgang Hoffmann-Riem, ‘Artificial Intelligence as a Challenge for Law and Regulation’ in Thomas Wischmeyer and Timo Rademacher (eds), Regulating Artificial Intelligence (Cham: Springer International Publishing 2020) 11. 134 LGPD, art 9. 135 LGPD, art 18, VI. 136 LGPD, art 11, s 5. 137 LGPD, art 9. 138 Arianna Rossi and Gabriele Lenzini, ‘Transparency by Design in Data-Informed Research: A Collection of Information Design Patterns’ (2020) 37 Computer Law & Security Review; Ploug and Holm (n 16). 139 LGPD, art 9, s 1. 140 LGPD, art 9, s 2. 141 LGPD, art 18, VIII and IX. For further analysis of these rights, see Ana Frazão, ‘Direitos Básicos Dos Titulares de Dados Pessoais’ (2019) XXXIX Revista do Advogado 33, s 4.7. 142 LGPD, art 8, s 5. 143 LGPD, art 18, VI. The impact of a request for data deletion upon machine learning systems that have already been trained from personal data is still an open question: Eduard Fosch Villaronga, Peter Kieseberg and Tiffany Li, ‘Humans Forget, Machines Remember: Artificial Intelligence and the Right to Be Forgotten’ (2018) 34 Computer Law & Security Review 304. 144 That is true at each individual patient evaluation, but both apps involve healthcare specialists in their development processes. 145 On the meaning of ‘decision’ within ‘automated decision-making’, see Lee A Bygrave, ‘Article 22: Automated Individual Decision-Making, Including Profiling’ in Christopher Kuner, Lee A Bygrave and Docksey, Christopher (eds), The EU General Data Protection Regulation (GDPR): A Commentary (Oxford, New York: OUP 2020) 532. 146 LGPD, art 20. 147 Philip AE Brey, ‘Anticipatory Ethics for Emerging Technologies’ (2012) 6 NanoEthics 1. 148 Unlike the regulatory approach adopted by the European Union: art 29 Working Party, ‘Guidelines on Automated Individual Decision-Making and Profiling for the Purposes of Regulation 2016/679’ (WP 251, 6 February 2018). 149 Nuria López, ‘Um direito, um dever: guia para o art. 20 da LGPD’ in Renato Opice Blum (ed), Proteção de Dados - Desafios e Soluções na Adequação à Lei (1a Edição, Editora Forense 2020) 322. 150 On the implementation of human intervention as a data subject right, see Marco Almada, ‘Human Intervention in Automated Decision-Making: Toward the Construction of Contestable Systems’, Proceedings of the 17th International Conference on Artificial Intelligence and Law (ICAIL 2019) (ACM 2019); Claudio Sarra, ‘Put Dialectics into the Machine: Protection against Automatic-Decision-Making through a Deeper Understanding of Contestability by Design’ (2020) 20 Global Jurist. 151 Refusing an examination on the grounds that one has already been diagnosed by an app would deprive a data subject of their right to review, but that would not be the most egregious violation of law in this case. 152 Ben Wagner, ‘Liable, but Not in Control? Ensuring Meaningful Human Agency in Automated Decision-Making Systems’ (2019) 11 Policy & Internet 104. 153 Art 29 Working Party (n 148). 154 López (n 149) 315. For arguments in the same direction regarding automated decision-making in the European Union, see Maja Brkan, ‘Do Algorithms Rule the World? Algorithmic Decision-Making and Data Protection in the Framework of the GDPR and Beyond’ (2019) 27 International Journal of Law and Information Technology 91, 98; Article 29 Working Party (n 148) 21. 155 Filippo Santoni de Sio and Jeroen van den Hoven, ‘Meaningful Human Control over Autonomous Systems: A Philosophical Account’ (2018) 5 Frontiers in Robotics and AI. 156 LGPD, art 20, s 1. On the debate over the meaning of the right to an explanation—and related concepts such as the right to meaningful information about decision-making—,see, among others, Margot E Kaminski, ‘The Right to Explanation, Explained’ (2019) 34 Berkeley Technology Law Journal 189; Andrew D Selbst and Julia Powles, ‘Meaningful Information and the Right to Explanation’ (2017) 7 International Data Privacy Law 233; Bibal and others (n 110); and Margot E Kaminski and Gianclaudio Malgieri, ‘Algorithmic Impact Assessments under the GDPR: Producing Multi-Layered Explanations’ International Data Privacy Law forthcoming. 157 Kaminski (n 156). 158 LGPD, art 22, s 2. 159 LGPD, art 38. On the relevance of such assessments for the governance of automated decision-making, see Maria Cecília Oliveira Gomes, ‘Relatório de Impacto à Proteção de Dados. Uma Breve Análise Da Sua Definição e Papel Na LGPD.’ [2020] Revista do Advogado 174; Eleni Kosta, ‘Article 35: Data Protection Impact Assessment’ in Christopher Kuner, Lee A Bygrave and Docksey, Christopher (eds), The EU General Data Protection Regulation (GDPR): A Commentary (Oxford, New York: OUP 2020); Margot E Kaminski and Gianclaudio Malgieri, ‘Algorithmic Impact Assessments under the GDPR: Producing Multi-Layered Explanations’ International Data Privacy Law accessed 7 December 2020. 160 The National Data Protection Authority (ANPD) was formally established by a presidential decree in 26 August 2020 (Brazil, Federal Decree 10.474 [2020]), which established its organisational structure and staffing allocation. However, as of December 2020, this authority did not emit any guidelines or rulings related to voice data or the pandemic. © The Author(s) 2021. Published by Oxford University Press. All rights reserved. For permissions, please email: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model) TI - Voice-based diagnosis of covid-19: ethical and legal challenges JF - International Data Privacy Law DO - 10.1093/idpl/ipab004 DA - 2021-01-28 UR - https://www.deepdyve.com/lp/oxford-university-press/voice-based-diagnosis-of-covid-19-ethical-and-legal-challenges-ahKCLBf3HO SP - 1 EP - 1 VL - Advance Article IS - DP - DeepDyve ER -