Abstract:Existential risks are those that threaten the entire future of humanity. Many theories of value imply that even relatively small reductions in net existential risk have enormous expected value. Despite their importance, issues surrounding human‐extinction risks and related hazards remain poorly understood. In this article, I clarify the concept of existential risk and develop an improved classification scheme. I discuss the relation between existential risks and basic issues in axiology, and show how existential risk reduction (via the maxipok rule) can serve as a strongly action‐guiding principle for utilitarian concerns. I also show how the notion of existential risk suggests a new way of thinking about the ideal of sustainability. • Existential risk is a concept that can focus long‐term global efforts and sustainability concerns. • The biggest existential risks are anthropogenic and related to potential future technologies. • A moral case can be made that existential risk reduction is strictly more important than any other global public good. • Sustainability should be reconceptualised in dynamic terms, as aiming for a sustainable trajectory rather than a sustainable state. • Some small existential risks can be mitigated today directly (e.g. asteroids) or indirectly (by building resilience and reserves to increase survivability in a range of extreme scenarios) but it is more important to build capacity to improve humanity’s ability to deal with the larger existential risks that will arise later in this century. This will require collective wisdom, technology foresight, and the ability when necessary to mobilise a strong global coordinated response to anticipated existential risks. • Perhaps the most cost‐effective way to reduce existential risks today is to fund analysis of a wide range of existential risks and potential mitigation strategies, with a long‐term perspective. Existential risk is a concept that can focus long‐term global efforts and sustainability concerns. The biggest existential risks are anthropogenic and related to potential future technologies. A moral case can be made that existential risk reduction is strictly more important than any other global public good. Sustainability should be reconceptualised in dynamic terms, as aiming for a sustainable trajectory rather than a sustainable state. Some small existential risks can be mitigated today directly (e.g. asteroids) or indirectly (by building resilience and reserves to increase survivability in a range of extreme scenarios) but it is more important to build capacity to improve humanity’s ability to deal with the larger existential risks that will arise later in this century. This will require collective wisdom, technology foresight, and the ability when necessary to mobilise a strong global coordinated response to anticipated existential risks. Perhaps the most cost‐effective way to reduce existential risks today is to fund analysis of a wide range of existential risks and potential mitigation strategies, with a long‐term perspective. In this article Nick Bostrom claims that issues surrounding human‐extinction risks and related hazards remain poorly understood. He clarifies the concept of existential risk and develops an improved classification scheme, showing how the notion of existential risk suggests a new way of thinking about the ideal of sustainability.

Should We Be Very Cautious or Extremely Cautious on Measures That May Involve Our Destruction?

Existential Risk Prevention as Global Priority

On the Unimportance of Superintelligence

Playing it safe: information constrains collective betting strategies

Biotechnology and the lifetime of technical civilizations

Evaluating sustainability actions under uncertainty: the role of improbable extreme scenarios

The irrationality of human confidence that an ageless existence would be better

Risk evaluation and behaviour: defining appropriate frames of reference

Intensified behavioural modernity of human: an insight into extinction crisis

Avoiding the "Great Filter": Extraterrestrial Life and Humanity's Future in the Universe

Faustian bargains: Short‐term and long‐term contingencies in phylogeny, ontogeny, and sociogeny

Initial Analysis of a Simple Numerical Model that Exhibits Antifragile Behavior

Big Bang, Low Bar -- Risk Assessment in the Public Arena

Are decisions under risk malleable?

Population ethical intuitions

An evolutionary model of bounded rationality and intelligence

Your money or your life? The carbon-development paradox

A behavioral gap in survival beliefs

Managing the risk of uncertain threshold responses: comparison of robust, optimum, and precautionary approaches

The shortness of human life constitutes its limit

Big troubles, imagined and real