Comparative Table of Human Rights Impact Assessment Frameworks for AI

Tool Author Year Target Audience Target System AI devep. cycle cover Person in Charge of its implementation Phases Components Stakeholder integration Human Rights Base Instruments on HHRR used Follow-up Mechanism Limitations Human Rights, Ethical, and Social Impact Assessment (HRESIA) Alessandro Mantelero 2022 Developers of AI/data systems (public and private sector) Policy-makers aiming to evaluate compliance and risk Civil society organisations advocating for human rights Researchers working on responsible AI or tech governance. --- AI system developers, deployers, and users to identify key values guiding the design and implementation of AI products and services. Data-intensive AIbased systems affecting individuals and society. It adopts a by-design approach from the earliest stages and is characterised by a circular approach that follows the product/service throughout its lifecycle, which is also in line with the circular product development models that focus on flexibility and interaction with users to address their needs. Organizations developing or deploying AI models, it can also be used by supervisory authorities and auditing bodies to monitor risk management in relation to the impact of data use on individual rights and freedoms. It is a tool that can be easily used by entities involved in AI development from the outset in the design of new AI solutions and can follow the product/service throughout its lifecycle, providing speci c, measurable and comparable evidence on potential impacts, their probability, extension, and severity, and facilitating comparison between possible alternative options. An analysis of relevant human rights, the definition of relevant ethical and social values and the targeted application of these frameworks to given AI cases the first relies on questionnaires and risk assessment tools, while the second is built on the use of experts to address societal challenges associated with the development and implementation of AI solutions. Second layer: Social and ethical values which play an important role in addressing non-legal issues associated with the adoption of certain AI solutions and their acceptability, and the balance between the different human rights and freedoms, in different contexts and periods. Third layer: Assessment of the concrete case based on specific sets of rights, values and principles-----HRESIA model combines different components, from selfassessment questionnaires to participatory tools. It does integrate rightsholder and stakeholder The key principles for AI builds on existing binding instruments and the contextualisation of their guiding principles. This is based on the assumption that the general principles provided by international human rights instruments should underpin all human activities including AI-based innovation. It is based on a common architecture grounded on intentional instruments with normative strength (charters of fundamental rights) (Blueprint) UN Guiding Principles on Business and Human Rights (UNGPs) It is an iterative approach to AI design, based on risk assessment and mitigation. Since in most cases the assessment is not based on measurable variables, the impact on rights and freedoms is necessarily the result of expert evaluation. -Its voluntary nature (Blueprint) Human Rights Impact Assessments (HRIA) Danish Institute 2020 • Developers of digital projects, products and/or services (e.g. software or app developers, who write, debug and execute the source code of the software or application), who want to: a) Better understand how human rights are relevant to the design, development, sale and end-use of digital products and services. b) Be better at anticipating potential human rights impacts and thereby changing the design of the digital product or service. c) Better understand their involvement with negative human rights impacts related to the application and end-use of digital products and services. • Companies buying digital projects, products and/or services (whether offthe-shelf or specifically tailored to the needs of the company in question), who want to: a) Better understand how human rights are relevant to the procurement, deployment and use of digital products and services. b) Better anticipate and change the design or potential use-cases of the product or service to address actual or potential human rights impacts. c) Better evaluate, monitor and communicate how human rights impacts of the digital service or product are being managed. Digital projects, products and services The HRIA should be conducted early in the project cycle or development phase of the product or service •Human rights practitioners and consultants conducting impact assessments of digital projects, products or services. • Companies, in particular staff who are responsible for commissioning and overseeing impact assessments, whether those businesses are developing the digital projects, products or services themselves or are buying them. • Public entities that are procuring and using digital products and services—in relation to e.g. e-governance initiatives, digital health services, automated decision-making in court systems, and other forms of public service delivery. Its has five phases: 1) planning and scoping; 2) data collection and context analysis; 3) analysing impacts; 4) impact prevention, mitigation, remediation; and 5) reporting and evaluation. Explanatory guidance, practical examples and case studies relevant to digital activities are provided throughout the five phases. The cross-cutting section on stakeholder engagement includes an introduction to consulting with rightsholders and other relevant parties, as well as information on relevant stakeholders to engage with. It also also includes a section on stakeholder engagement as the key cross-cutting component throughout the phases of HRIA. Engagement with customers, customer advocates, employees, contractors, investors, analysts, industry bodies, regulators and government, suppliers, and the broader community may need to be considered, as well as aligning this review with Impact Assessments required by ISO Standards, Privacy Impact Assessments, Security Assessments and others. UN Guiding Principles on Business and Human Rights (UNGPs) - International Human Rights as Bechmarks UN Guiding Principles on Business and Human Rights (UNGPs) The reassessment of the observations and conclusions in a HRIA and the actions taken as a result should be part of ongoing HRDD processes It does not provide in-depth guidance on broader human rights due diligence. It does not focus on impacts linked to labour rights in the hardware supply chain, impacts related to physical infrastructure, or cumulative environmental or labour rights related impacts. Fundamental Rights Impact Assessment (AFRIA) ALIGNER n.d. Law Enforcement Agencies (LEAs) To LEAs who aim to deploy AI systems for the purposes of prevention, investigation, detection or prosecution of criminal offences or execution of criminal penalties (i.e., law enforcement purposes) within the EU. Prior to the deployment of the AI system LEAs should establish a diverse and multidisciplinary team, responsible for performing the AFRIA. The team should include members of the organisation with legal, operational, and technical expertise. It is also advisable to involve the organisation’s data protection officer in the AFRIA process. If possible, LEAs should engage in discussions with the provider of the AI system assessed to clarify eventual uncertainties on the functioning of the AI system itself First, it helps LEAs identify and mitigate the risks posed by the deployment of a certain AI system in relation to ethical principles and (selected) fundamental rights of individuals. Second, it is a suitable instrument for LEAs to explain and record their decision-making processes. The AFRIA consists of two different, but connected, templates: the Fundamental Rights Impact Assessment and the AI System Governance. --------- 1. Presumption of innocence and right to an effective remedy and to a fair trial; 2. Right to equality and nondiscrimination; 3. Freedom of expression and information; and 4. Right to respect for private and family life and right to protection of personal data. Ethical principles and fundamental rights Performing an AFRIA is an iterative process. The AFRIA needs to be recorded, reviewed, and updated throughout the whole lifecycle of the AI system to reflect eventual changes in the functioning of the technology and/or its circumstances of deployment. AFRIA is not designed to be used in the following circumstances: a. During the development stage of the AI systems, even if carried out by LEAs; and b. When deploying AI systems for purposes other than law enforcement ones Human Rights, Democracy, and the Rule of Law Assurance Framework for AI Systems The Alan Turing Institute 2021 Designers,cdevelopers, and users of AI systems AI systems Across the entire AI project lifecycle AI project team (developers, managers, governance bodies). Duty-bearers (organizations) must ensure compliance. 1. The Preliminary ContextBased Risk Analysis (PCRA) 2. The Stakeholder Engagement Process (SEP), 3. The Human Rights, Democracy, and the Rule of Law Impact Assessment (HUDERIA), The Human Rights, Democracy, and Rule of Law Assurance Case (HUDERAC) Questions HUDERAF has been developed with this in mind. Stakeholder Engagement ► Human dignity and individual autonomy ► Equality and non-discrimination ► Respect for privacy and personal data protection ► Transparency and oversight ► Accountability and responsibility ► Reliability ► Safe innovation - International human rights standards - Council of Europe Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law HUDERAC processes are iterative and organised to respond appropriately to changes in production and implementation factors as well as in the operating environments of AI systems. The HUDERAF model will, however, need to stay responsive to the development of novel AI innovations and use-cases. Tool Author Year Target Audience Target System AI devep. cycle cover Person in Charge of its implementation Phases Components Stakeholder integration Human Rights Base Instruments on HHRR used Follow-up Mechanism Limitations Fundamental Rights and Algorithm Impact Assessment (FRAIA) Goverment of Netherlands 2022 Formulated in any instance where a government organisation considers developing, delegating the development of, buying, adjusting and/or using an algorithm A specific algorithm Before implementation or after [Even when an algorithm is already being used, the FRAIA may serve as a tool for reflection]. The FRAIA can be used when a government agency is considering the development, procurement, or use of an algorithm. The FRAIA can also serve as an evaluation tool for algorithms already deployed. Interdisciplinary teams Four parts: three parts cover the three decision-making stages regarding the use of an algorithm, while the last part centres on the broader questions about fundamental rights. Questionnaires It mentions: "Citizen", Data Ethics Consultant,Policy Advisor, among others Fundamental rights (constitutional rights in the Netherlands) Constitutional rights in the Netherlands . Treat the FRAIA as a dynamic document that can be modified and supplemented as the project progresses. The FRAIA does not automatically catch such (ethical) issues, given that it is ultimately a fundamental rights test. If a case does not infringe on a fundamental right, there may be a risk of other non-fundamental rights-related considerations being disregarded. Assessment List for Trustworthy Artificial Intelligence (ALTAI) for selfassessment European Comission 2020 Organisations that aim to identify how proposed AI systems might generate risks, and to identify whether and what kind of active measures may need to be taken to avoid and minimise those risks AI systems Developed, deployed, procured or used A multidisciplinary team of people. These could be from within and/or outside the organization organisation multidisciplinary team of people. These could be from within and/or outside your organisation 1. Human Agency and Oversight; 2. Technical Robustness and Safety; 3. Privacy and Data Governance; 4. Transparency; 5. Diversity, Nondiscrimination and Fairness; 6. Societal and Environmental Well-being; 7. Accountability. Questions and additional explanatory notes for many of the questions • AI designers and AI developers of the AI system; • data scientists; • procurement officers or specialists; • front-end staff that will use or work with the AI system; • legal/compliance officers; • management. Fundamental Rights European Union to refer to human rights enshrined in the EU Treaties, the Charter of Fundamental Rights (the Charter) , and international human rights Law. Not mentioned The individual or collective members of the High Level Expert Group on AI do not offer any guarantee as to the compliance of an AI system assessed by using ALTAI with the 7 requirements for Trustworthy AI. Human Rights AI Impact Assessment Law Commissio n of Ontario and Ontario Human Rights Commissio n . 2024 This tool is expected to help designers, developers, operators and owners of AI systems to identify and reduce bias and discrimination. To any algorithm, automated decisionmaking system or artificial intelligence system This assessment should be completed: 1) when the idea for the AI system is explored and developed; 2) before the AI system is made available to external parties (eg: before a vendor makes a model or application available to purchasers, or service providers deploy an AI technology for customer service); 3) within ninety days of a material change in the system; 4) yearly as part of regular maintenance and reviews. The team responsible for conducting an HRIA should include people with a variety of socio-technical expertise – technological, legal, human rights, business strategy and community advocacy Two parts. Part A is an assessment of the AI system for human rights implications. Part B is about mitigation. Once the AI system has been categorized, Part B provides a series of questions to assist organizations to minimize the identified human rights issues in the given AI system. Questions for multidisciplinary teams and consultations with impacted communities. We encourage parties to consider the perspectives of as many different stakeholders as possible when answering, particularly from those who may be impacted by the AI system. Human Rights Human rights are protected by the Ontario Human Rights Code (the Code), the Canadian Human Rights Act (the Act), and the Canadian Charter of Rights and Freedoms (the Charter) AI assessments are iterative and should be considered circular, not linear. Parties should be assessing an AI system for human rights issues, taking steps to mitigate issues, and then returning to assess the system again. While the use of the HRIA is to help identify, address and remedy potential adverse human rights impacts, an organization or individual will not be protected from liability for adverse human rights impacts, including unlawful discrimination, if they claim they complied with or relied on the HRIA. Human Rights Impact Assessment Tool for AI (HRIA for AI’) National Human Rights Commissio n of Korea. n.d. All AI developed or procured directly by public institutions and high-risk AI utilized in the private sector that poses a high risk of human rights violations For public institutions, the HRIA for AI should be conducted regardless of the risk level. In the private sector, the HRIA for AI should be performed before developing or adopting high-risk AI as the foundational technology for projects or policies. In the private sector, the HRIA for AI should be performed before developing or adopting high-risk AI as the foundational technology for projects or policies. This assessment is not limited to preliminary evaluations but can also include periodic and post-implementation evaluations for continuous management and monitoring. An independent organization separate from the AI development team or related business departments. This could be a department responsible for AI ethics, human rights management, ESG management, or a third-party organization with expertise in human rights and AI technology. Stage 1: Planning and Preparation Stage 2: Analysis and Assessment Stage 3: Improvement and Remedy Stage 4: Disclosure and Review Common: Stakeholder Participation Questions and stakeholder participation at evey stage It is crucial to identify both internal and external stakeholders related to the AI, and to consult with them or gather their opinions on the human right impacts of the AI. Human Rights Constitution of the Republic of Korea and international human rights norms [ Universal Declaration of Human Rights (UDHR), the International Covenant on Economic, Social and Cultural Rights (ICESCR), and the International Covenant on the Civil and Political Rights (ICCPR) are included in international human rights norms]. Reviewing obstacles or issues encountered during the HRIA process and reflecting them in the organization’s internal regulations can aid in future HRIA efforts. During Stage 4, it is essential to verify whether the procedures for post-HRIA steps are in place. The checklist items are not intended to regulate companies or institutions or impose detailed obligations. While they are considerations for detecting and mitigating the potential for human rights violations by AI, the purposes, technical means, and application areas of AI are highly diverse, and the impacts and methods affected to human rights can also vary. Therefore, the HRIA is not about meeting every checklist item or providing the correct answers to the questions. Instead, it should serve as a medium for communication and discussion among stakeholders and a process for strengthening each other’s capabilities. Ethical impact assessment: a tool of the Recommendation on the Ethics of Artificial Intelligence UNESCO 2023 Government officials (individuals and teams) involved in the procurement of AI systems - Procurement officers AI systems All stages Project team - Broadly to both individuals involved in procurement roles and, where relevant, other individuals involved in the design and development of the AI system (e. g., project manager, administrators, technical team, etc.) EIA is intended to be a living document that will be filled out progressively and iteratively at different stages including: ❙ During project research, design, development and pre-procurement (for example, to reflect on the scope of the project, its legitimate aims and whether AI is an appropriate solution); ❙ During the procurement process itself, when the EIA can help both in selecting a supplier and in formulating contractual obligations; ❙ Following project deployment, the EIA should be revisited at regular intervals, especially since the answers may change over time as the project evolves. Scoping questions, Stakeholder engagement plans, and Impact assessment tables To inclusively assess the impacts of this AI project, it is necessary to consult a diverse range of stakeholders. UNESCO principles: Safety and Security Fairness, Non-Discrimination, Diversity Sustainability Privacy and Data Protection Human Oversight and Determination Transparency and Explainability; Accountability and Responsibility Awareness and Literacy Universal Declaration of Human Rights and European Convention on Human Rights Following project deployment, the EIA should be revisited at regular intervals, especially since the answers may change over time as the project evolves. It is important to note that it is not a universal tool and therefore it will need to be adapted to the specific circumstances it is used in and to the regulatory regime in each country.