Don’t Let Governments Buy AI Systems That Ignore Human Rights

Merve Hickok,Evanna Hu

DOI: https://doi.org/10.58875/acog7116

2024-04-01

Issues in science and technology

Abstract:Even in the absence of broader AI regulation, federal procurement provisions could set expectations for data quality, model performance, risk assessments, and documentation.

engineering, industrial, multidisciplinary,multidisciplinary sciences,social issues

What problem does this paper attempt to address?

Public procurement of artificial intelligence systems: new risks and future proofing

Merve Hickok

DOI: https://doi.org/10.1007/s00146-022-01572-2

2022-10-03

Abstract:Public entities around the world are increasingly deploying artificial intelligence (AI) and algorithmic decision-making systems to provide public services or to use their enforcement powers. The rationale for the public sector to use these systems is similar to private sector: increase efficiency and speed of transactions and lower the costs. However, public entities are first and foremost established to meet the needs of the members of society and protect the safety, fundamental rights, and wellbeing of those they serve. Currently AI systems are deployed by the public sector at various administrative levels without robust due diligence, monitoring, or transparency. This paper critically maps out the challenges in procurement of AI systems by public entities and the long-term implications necessitating AI-specific procurement guidelines and processes. This dual-prong exploration includes the new complexities and risks introduced by AI systems, and the institutional capabilities impacting the decision-making process. AI-specific public procurement guidelines are urgently needed to protect fundamental rights and due process.
AI Procurement Checklists: Revisiting Implementation in the Age of AI Governance

Tom Zick,Mason Kortz,David Eaves,Finale Doshi-Velez

2024-04-23

Abstract:Public sector use of AI has been quietly on the rise for the past decade, but only recently have efforts to regulate it entered the cultural zeitgeist. While simple to articulate, promoting ethical and effective roll outs of AI systems in government is a notoriously elusive task. On the one hand there are hard-to-address pitfalls associated with AI-based tools, including concerns about bias towards marginalized communities, safety, and gameability. On the other, there is pressure not to make it too difficult to adopt AI, especially in the public sector which typically has fewer resources than the private sector$\unicode{x2014}$conserving scarce government resources is often the draw of using AI-based tools in the first place. These tensions create a real risk that procedures built to ensure marginalized groups are not hurt by government use of AI will, in practice, be performative and ineffective. To inform the latest wave of regulatory efforts in the United States, we look to jurisdictions with mature regulations around government AI use. We report on lessons learned by officials in Brazil, Singapore and Canada, who have collectively implemented risk categories, disclosure requirements and assessments into the way they procure AI tools. In particular, we investigate two implemented checklists: the Canadian Directive on Automated Decision-Making (CDADM) and the World Economic Forum's AI Procurement in a Box (WEF). We detail three key pitfalls around expertise, risk frameworks and transparency, that can decrease the efficacy of regulations aimed at government AI use and suggest avenues for improvement.

Computers and Society,Artificial Intelligence
Rules for robots, and why medical AI breaks them

Barbara J Evans

DOI: https://doi.org/10.1093/jlb/lsad001

IF: 3.4

2023-01-01

Journal of Law and the Biosciences

Abstract:Abstract This article critiques the quest to state general rules to protect human rights against AI/ML computational tools. The White House Blueprint for an AI Bill of Rights was a recent attempt that fails in ways this article explores. There are limits to how far ethicolegal analysis can go in abstracting AI/ML tools, as a category, from the specific contexts where AI tools are deployed. Health technology offers a good example of this principle. The salient dilemma with AI/ML medical software is that privacy policy has the potential to undermine distributional justice, forcing a choice between two competing visions of privacy protection. The first, stressing individual consent, won favor among bioethicists, information privacy theorists, and policymakers after 1970 but displays an ominous potential to bias AI training data in ways that promote health care inequities. The alternative, an older duty-based approach from medical privacy law aligns with a broader critique of how late-20th-century American law and ethics endorsed atomistic autonomy as the highest moral good, neglecting principles of caring, social interdependency, justice, and equity. Disregarding the context of such choices can produce suboptimal policies when - as in medicine and many other contexts - the use of personal data has high social value.

ethics,law,medicine, legal,medical ethics
Public Procurement for Responsible AI? Understanding U.S. Cities' Practices, Challenges, and Needs

Nari Johnson,Elise Silva,Harrison Leon,Motahhare Eslami,Beth Schwanke,Ravit Dotan,Hoda Heidari

2024-11-08

Abstract:Most AI tools adopted by governments are not developed internally, but instead are acquired from third-party vendors in a process called public procurement. While scholars and regulatory proposals have recently turned towards procurement as a site of intervention to encourage responsible AI governance practices, little is known about the practices and needs of city employees in charge of AI procurement. In this paper, we present findings from semi-structured interviews with 18 city employees across 7 US cities. We find that AI acquired by cities often does not go through a conventional public procurement process, posing challenges to oversight and governance. We identify five key types of challenges to leveraging procurement for responsible AI that city employees face when interacting with colleagues, AI vendors, and members of the public. We conclude by discussing recommendations and implications for governments, researchers, and policymakers.

Computers and Society,Artificial Intelligence,Human-Computer Interaction
Regulating Generative AI: A Pathway to Ethical and Responsible Implementation

Jonathan Luckett

DOI: https://doi.org/10.5121/ijci.2023.120508

2023-08-12

International Journal on Cybernetics & Informatics

Abstract:Artificial intelligence (AI) is becoming more and more prevalent in our daily lives, and its potential applications are practically limitless. However, as with any technology, there are concerns about how AI could be misused or abused. One of the most serious concerns is the potential for discrimination, particularly against women or minorities, when AI systems are used for tasks like job hiring. Additionally, there are concerns about privacy and security, as AI could be used to monitor people's movements or launch cyberattacks. To address these concerns, regulations must be developed to ensure that AI is developed and used ethically and responsibly. These regulations should address issues like safety, privacy, security, and discrimination. Finally, it is important to educate the public about AI and how to use it safely and responsibly. In this paper, I will examine the AI regulations and challenges that exist today, particularly in the United States. Two regulations I will focus on are the AI in Government Act of 2020 and the National Artificial Intelligence Initiative Act of 2020. Additionally, I will examine two Executive Orders that have addressed the issue of AI in the federal government. Finally, I will conclude with some policy considerations and recommendations for federal agencies.
The Challenges of AI-Decision-Making in Government and Administrative Law: A Proposal for Regulatory Design

Benedict Sheehy,Yee-Fui Ng

DOI: https://doi.org/10.18060/28360

2024-06-10

Indiana Law Review

Abstract:Artificial Intelligence (“AI”) decision-making is rapidly becoming a key technology in government. AI decision-making has a number of significant benefits. These include reduced costs, consistency in decision-making, and the potential for reduced bias. It is not, however, an unalloyed good. There are a number of risks associated with the technology. For example, it may have inherent biases or be prone to developing certain biases. These issues are most evident when AI is used to deal with vulnerable populations, populations unaware of or less able to deal with advanced technologies and sophisticated systems in general. With these populations, AI has the potential to do significant harm. Governments have struggled to understand and address AI decision-making appropriately. And nowhere is this clearer than in dealing with vulnerable populations. This Article addresses these issues using theories of effective regulation and two case studies of AI decision-making in Australia. It proposes two regulatory innovations: a default advocate for negative decisions “Golden Rule”, and a monitoring body that, among other things, implements Professor Aziz Huq’s proposed “right to a well-calibrated decision.”
Governments Setting Limits on AI

Esther Shein

DOI: https://doi.org/10.1145/3640506

IF: 22.7

2024-03-26

Communications of the ACM

Abstract:Many countries and regions are considering, or trying to implement, regulations on the training and use of artificial intelligence.

computer science, theory & methods, software engineering, hardware & architecture
A Human Rights-Based Approach to Responsible AI

Vinodkumar Prabhakaran,Margaret Mitchell,Timnit Gebru,Iason Gabriel

DOI: https://doi.org/10.48550/arXiv.2210.02667

2022-10-06

Abstract:Research on fairness, accountability, transparency and ethics of AI-based interventions in society has gained much-needed momentum in recent years. However it lacks an explicit alignment with a set of normative values and principles that guide this research and interventions. Rather, an implicit consensus is often assumed to hold for the values we impart into our models - something that is at odds with the pluralistic world we live in. In this paper, we put forth the doctrine of universal human rights as a set of globally salient and cross-culturally recognized set of values that can serve as a grounding framework for explicit value alignment in responsible AI - and discuss its efficacy as a framework for civil society partnership and participation. We argue that a human rights framework orients the research in this space away from the machines and the risks of their biases, and towards humans and the risks to their rights, essentially helping to center the conversation around who is harmed, what harms they face, and how those harms may be mitigated.

Artificial Intelligence,Computers and Society
Towards Regulatable AI Systems: Technical Gaps and Policy Opportunities

Xudong Shen,Hannah Brown,Jiashu Tao,Martin Strobel,Yao Tong,Akshay Narayan,Harold Soh,Finale Doshi-Velez

2024-03-27

Abstract:There is increasing attention being given to how to regulate AI systems. As governing bodies grapple with what values to encapsulate into regulation, we consider the technical half of the question: To what extent can AI experts vet an AI system for adherence to regulatory requirements? We investigate this question through the lens of two public sector procurement checklists, identifying what we can do now, what should be possible with technical innovation, and what requirements need a more interdisciplinary approach.

Artificial Intelligence,Computers and Society
Regulatory Markets: The Future of AI Governance

Gillian K. Hadfield,Jack Clark

2023-04-26

Abstract:Appropriately regulating artificial intelligence is an increasingly urgent policy challenge. Legislatures and regulators lack the specialized knowledge required to best translate public demands into legal requirements. Overreliance on industry self-regulation fails to hold producers and users of AI systems accountable to democratic demands. Regulatory markets, in which governments require the targets of regulation to purchase regulatory services from a private regulator, are proposed. This approach to AI regulation could overcome the limitations of both command-and-control regulation and self-regulation. Regulatory market could enable governments to establish policy priorities for the regulation of AI, whilst relying on market forces and industry R&D efforts to pioneer the methods of regulation that best achieve policymakers' stated objectives.

Artificial Intelligence,General Economics
Data-Centric AI Governance: Addressing the Limitations of Model-Focused Policies

Ritwik Gupta,Leah Walker,Rodolfo Corona,Stephanie Fu,Suzanne Petryk,Janet Napolitano,Trevor Darrell,Andrew W. Reddie

2024-09-26

Abstract:Current regulations on powerful AI capabilities are narrowly focused on "foundation" or "frontier" models. However, these terms are vague and inconsistently defined, leading to an unstable foundation for governance efforts. Critically, policy debates often fail to consider the data used with these models, despite the clear link between data and model performance. Even (relatively) "small" models that fall outside the typical definitions of foundation and frontier models can achieve equivalent outcomes when exposed to sufficiently specific datasets. In this work, we illustrate the importance of considering dataset size and content as essential factors in assessing the risks posed by models both today and in the future. More broadly, we emphasize the risk posed by over-regulating reactively and provide a path towards careful, quantitative evaluation of capabilities that can lead to a simplified regulatory environment.

Computers and Society,Artificial Intelligence
Promises and limits of law for a human-centric artificial intelligence

David Restrepo Amariles,Pablo Marcello Baquero

DOI: https://doi.org/10.1016/j.clsr.2023.105795

IF: 2.707

2023-04-01

Computer Law & Security Review

Abstract:While the concept of human-centric artificial intelligence (AI) has emerged as a key principle to govern AI systems, two obstacles for its implementation remain largely understated. First, the excessive focus on accountability at the design stage of AI systems, overshadowing the fact that human values can be affected at different stages across the AI life cycle. Second, the market-driven approach of current regulatory initiatives, limited in their ability to actively promote human values. In this article, we argue for a twofold approach to tackle these limitations. On one hand, we propose a co-evolutionary and life cycle approach to tackle the lack of accountability of AI systems, showing that this approach can help ensure accountability beyond the design stage by enabling meaningful human control and human-AI interaction across the entire lifecycle of the system. On the other hand, we propose that regulatory initiatives should balance the market-driven approach by giving a more predominant role to human rights and by introducing explicitly the notion of proportionality test. This rebalancing would serve to handle conflicts between the objectives pursued by AI systems circulating in the markets and the need for an effective protection of human rights.

law
Negative Human Rights as a Basis for Long-term AI Safety and Regulation

Ondrej Bajgar,Jan Horenovsky

DOI: https://doi.org/10.1613/jair.1.14020

2023-04-20

Abstract:If autonomous AI systems are to be reliably safe in novel situations, they will need to incorporate general principles guiding them to recognize and avoid harmful behaviours. Such principles may need to be supported by a binding system of regulation, which would need the underlying principles to be widely accepted. They should also be specific enough for technical implementation. Drawing inspiration from law, this article explains how negative human rights could fulfil the role of such principles and serve as a foundation both for an international regulatory system and for building technical safety constraints for future AI systems.

Computers and Society,Artificial Intelligence
Designing for Human Rights in AI

Evgeni Aizenberg,Jeroen van den Hoven

DOI: https://doi.org/10.1177/2053951720949566

2020-07-07

Abstract:In the age of big data, companies and governments are increasingly using algorithms to inform hiring decisions, employee management, policing, credit scoring, insurance pricing, and many more aspects of our lives. AI systems can help us make evidence-driven, efficient decisions, but can also confront us with unjustified, discriminatory decisions wrongly assumed to be accurate because they are made automatically and quantitatively. It is becoming evident that these technological developments are consequential to people's fundamental human rights. Despite increasing attention to these urgent challenges in recent years, technical solutions to these complex socio-ethical problems are often developed without empirical study of societal context and the critical input of societal stakeholders who are impacted by the technology. On the other hand, calls for more ethically- and socially-aware AI often fail to provide answers for how to proceed beyond stressing the importance of transparency, explainability, and fairness. Bridging these socio-technical gaps and the deep divide between abstract value language and design requirements is essential to facilitate nuanced, context-dependent design choices that will support moral and social values. In this paper, we bridge this divide through the framework of Design for Values, drawing on methodologies of Value Sensitive Design and Participatory Design to present a roadmap for proactively engaging societal stakeholders to translate fundamental human rights into context-dependent design requirements through a structured, inclusive, and transparent process.

Computers and Society,Artificial Intelligence,Machine Learning
Position Paper: Model Access should be a Key Concern in AI Governance

Edward Kembery,Ben Bucknall,Morgan Simpson

2024-12-01

Abstract:The downstream use cases, benefits, and risks of AI systems depend significantly on the access afforded to the system, and to whom. However, the downstream implications of different access styles are not well understood, making it difficult for decision-makers to govern model access responsibly. Consequently, we spotlight Model Access Governance, an emerging field focused on helping organisations and governments make responsible, evidence-based access decisions. We outline the motivation for developing this field by highlighting the risks of misgoverning model access, the limitations of existing research on the topic, and the opportunity for impact. We then make four sets of recommendations, aimed at helping AI evaluation organisations, frontier AI companies, governments and international bodies build consensus around empirically-driven access governance.

Computers and Society
Can everyday AI be ethical. Fairness of Machine Learning Algorithms

Philippe Besse,Celine Castets-Renard,Aurelien Garivier,Jean-Michel Loubes

DOI: https://doi.org/10.48550/arXiv.1810.01729

2018-10-03

Abstract:Combining big data and machine learning algorithms, the power of automatic decision tools induces as much hope as fear. Many recently enacted European legislation (GDPR) and French laws attempt to regulate the use of these tools. Leaving aside the well-identified problems of data confidentiality and impediments to competition, we focus on the risks of discrimination, the problems of transparency and the quality of algorithmic decisions. The detailed perspective of the legal texts, faced with the complexity and opacity of the learning algorithms, reveals the need for important technological disruptions for the detection or reduction of the discrimination risk, and for addressing the right to obtain an explanation of the auto- matic decision. Since trust of the developers and above all of the users (citizens, litigants, customers) is essential, algorithms exploiting personal data must be deployed in a strict ethical framework. In conclusion, to answer this need, we list some ways of controls to be developed: institutional control, ethical charter, external audit attached to the issue of a label.

Other Statistics,Artificial Intelligence
Auction-Based Regulation for Artificial Intelligence

Marco Bornstein,Zora Che,Suhas Julapalli,Abdirisak Mohamed,Amrit Singh Bedi,Furong Huang

2024-10-03

Abstract:In an era of "moving fast and breaking things", regulators have moved slowly to pick up the safety, bias, and legal pieces left in the wake of broken Artificial Intelligence (AI) deployment. Since AI models, such as large language models, are able to push misinformation and stoke division within our society, it is imperative for regulators to employ a framework that mitigates these dangers and ensures user safety. While there is much-warranted discussion about how to address the safety, bias, and legal woes of state-of-the-art AI models, the number of rigorous and realistic mathematical frameworks to regulate AI safety is lacking. We take on this challenge, proposing an auction-based regulatory mechanism that provably incentivizes model-building agents (i) to deploy safer models and (ii) to participate in the regulation process. We provably guarantee, via derived Nash Equilibria, that each participating agent's best strategy is to submit a model safer than a prescribed minimum-safety threshold. Empirical results show that our regulatory auction boosts safety and participation rates by 20% and 15% respectively, outperforming simple regulatory frameworks that merely enforce minimum safety standards.

Computer Science and Game Theory,Artificial Intelligence,Computers and Society,General Economics
The risks of risk-based AI regulation: taking liability seriously

Martin Kretschmer,Tobias Kretschmer,Alexander Peukert,Christian Peukert

DOI: https://doi.org/10.48550/arXiv.2311.14684

2023-11-03

Computers and Society

Abstract:The development and regulation of multi-purpose, large "foundation models" of AI seems to have reached a critical stage, with major investments and new applications announced every other day. Some experts are calling for a moratorium on the training of AI systems more powerful than GPT-4. Legislators globally compete to set the blueprint for a new regulatory regime. This paper analyses the most advanced legal proposal, the European Union's AI Act currently in the stage of final "trilogue" negotiations between the EU institutions. This legislation will likely have extra-territorial implications, sometimes called "the Brussels effect". It also constitutes a radical departure from conventional information and communications technology policy by regulating AI ex-ante through a risk-based approach that seeks to prevent certain harmful outcomes based on product safety principles. We offer a review and critique, specifically discussing the AI Act's problematic obligations regarding data quality and human oversight. Our proposal is to take liability seriously as the key regulatory mechanism. This signals to industry that if a breach of law occurs, firms are required to know in particular what their inputs were and how to retrain the system to remedy the breach. Moreover, we suggest differentiating between endogenous and exogenous sources of potential harm, which can be mitigated by carefully allocating liability between developers and deployers of AI technology.
Ethics and Responsible AI Deployment

Petar Radanliev,Omar Santos

2023-11-12

Abstract:As Artificial Intelligence (AI) becomes more prevalent, protecting personal privacy is a critical ethical issue that must be addressed. This article explores the need for ethical AI systems that safeguard individual privacy while complying with ethical standards. By taking a multidisciplinary approach, the research examines innovative algorithmic techniques such as differential privacy, homomorphic encryption, federated learning, international regulatory frameworks, and ethical guidelines. The study concludes that these algorithms effectively enhance privacy protection while balancing the utility of AI with the need to protect personal data. The article emphasises the importance of a comprehensive approach that combines technological innovation with ethical and regulatory strategies to harness the power of AI in a way that respects and protects individual privacy.

Computers and Society,Human-Computer Interaction,Social and Information Networks,Systems and Control
Artificial Intelligence and Human Rights: Corporate Responsibility in AI Governance Initiatives

Lottie Lane

DOI: https://doi.org/10.1080/18918131.2022.2137288

2023-01-07

Nordic Journal of Human Rights

Abstract:Private businesses are central actors in the development of artificial intelligence (AI), meaning they have a key role in ensuring that AI respects human rights. Meanwhile, international human rights law (IHRL) has been scrambling to catch up with technological developments that have occurred since the establishment of its state-centric framework that were not envisaged by its drafters. Despite progress in the development of international legal standards on business and human rights, uncertainties regarding the role and responsibilities of AI businesses remain. This article addresses these uncertainties from a governance perspective and against the backdrop of the public/private divide; it views laws as instruments of governance, which comprises activities by many public and private actors. Section 2 briefly assesses the current framework of IHRL regarding AI and businesses, focusing on the lack of legal certainty. Section 3 critically analyses AI initiatives beyond IHRL that have been adopted at international, regional, and national levels to gain insight into specific standards of behaviour expected of AI businesses, as well as to challenge a dichotomous public/private divide in this context. Section 4 provides conclusions and recommendations.

Don’t Let Governments Buy AI Systems That Ignore Human Rights

Public procurement of artificial intelligence systems: new risks and future proofing

AI Procurement Checklists: Revisiting Implementation in the Age of AI Governance

Rules for robots, and why medical AI breaks them

Public Procurement for Responsible AI? Understanding U.S. Cities' Practices, Challenges, and Needs

Regulating Generative AI: A Pathway to Ethical and Responsible Implementation

The Challenges of AI-Decision-Making in Government and Administrative Law: A Proposal for Regulatory Design

Governments Setting Limits on AI

A Human Rights-Based Approach to Responsible AI

Towards Regulatable AI Systems: Technical Gaps and Policy Opportunities

Regulatory Markets: The Future of AI Governance

Data-Centric AI Governance: Addressing the Limitations of Model-Focused Policies

Promises and limits of law for a human-centric artificial intelligence

Negative Human Rights as a Basis for Long-term AI Safety and Regulation

Designing for Human Rights in AI

Position Paper: Model Access should be a Key Concern in AI Governance

Can everyday AI be ethical. Fairness of Machine Learning Algorithms

Auction-Based Regulation for Artificial Intelligence

The risks of risk-based AI regulation: taking liability seriously

Ethics and Responsible AI Deployment

Artificial Intelligence and Human Rights: Corporate Responsibility in AI Governance Initiatives