Abstract:The American founders attempted to establish a clockwork government. Virtue was to be assured by humans, acting as they must within their human natures, but operating within a framework that assured mechanically that the outputs of government would not be tyrannical. Whether this system has worked well or not is a matter of controversy, but to the extent it did not work, it seems to have been at least partially a failure of the mechanisms designed to compensate for the shortcomings of human nature. Now we are on the verge of developing “artificial intelligence.” Whether these technological advances will emerge slowly or quickly is unknown, as are their contours. But even minimal AI could lead to a radical improvement in government because AI’s could be designed to perform the tasks of government with very low agency costs. However, it may seem uncertain that AI’s would be so designed. It may be, first, that there will not be any AI’s after all. It may be also that AI’s will be designed or implemented by exactly the humans who create agency costs in the first place, and used for their own and not the public good. And it may be that AI’s take off into the high orbit of superintelligence and decide to reduce us to slavery or dust. But these possibilities, while possible, seem unlikely. Probably AI’s will emerge, but only after a long time. AI’s will be difficult to design but there are reasons to expect they will be designed so as to minimize agency costs. They will probably, ironically enough, emerge in the order of tool, oracle and genie that Bostrom mentions (but for different reasons). We can hope to control AI tools, oracles, and genies. An AI sovereign, however, would be much more difficult to control, if it were possible to control at all. AI sovereigns would be persons. But AI’s must not be allowed to become persons, in a philosophical or legal sense. AI persons would have to be slaves if we were to control them. One hopes they would be slaves without subjective consciousness. If they did have subjective consciousness anything like humans, we would be faced with the impossible moral dilemma of being slave-masters or slaves ourselves. Hence a hard line should be drawn against AI research that is directed specifically at the emergence of subjective consciousness in machines, or likely to lead that way, but these goals are far beyond any current, or really any currently imaginable, AI research. The promise of controlling government is great enough to justify the merely notional risk of creating AI monsters we cannot control.

A sovereign AI capability for the UK

Foundations for the future: institution building for the purpose of artificial intelligence governance

Artificial intelligence foundation and pre-trained models: Fundamentals, applications, opportunities, and social impacts

Decentralised Governance-Driven Architecture for Designing Foundation Model based Systems: Exploring the Role of Blockchain in Responsible AI

International Institutions for Advanced AI

Future-proof: Monitoring the development, deployment, and impacts of Artificial Intelligence

A Systems Approach to Achieving the Benefits of Artificial Intelligence in UK Defence

Tools, Oracles, Genies and Sovereigns: Artificial Intelligence and the Future of Government

A Reference Architecture for Designing Foundation Model based Systems

Artificial Intelligence and the ‘Good Society’: the US, EU, and UK approach

The potential functions of an international institution for AI safety. Insights from adjacent policy areas and recent trends

Demystifying AI and AI Companies – What Foreign Policy Makers Need to Know About the Global AI Industry

Regulatory Markets: The Future of AI Governance

Mind the Gap: Foundation Models and the Covert Proliferation of Military Intelligence, Surveillance, and Targeting

Why and How Governments Should Monitor AI Development

Towards Foundation-model-based Multiagent System to Accelerate AI for Social Impact

AI Model Registries: A Foundational Tool for AI Governance

The risks of risk-based AI regulation: taking liability seriously

Developing safer AI–concepts from economics to the rescue

Mapping the individual, social, and biospheric impacts of Foundation Models

Human rights, democracy, and the rule of law assurance framework for AI systems: A proposal