On Artificial Superintelligence and the Problem of Charismatic Extinction Threats
DOI:
https://doi.org/10.55613/jeet.v35i2.166Keywords:
Artificial Superintelligence, Extinction Threat, Charismatic Extinction Threat, Finite Pool of WorryAbstract
This paper focuses on the challenge of finding a rational response to Artificial Superintelligence (ASI) as an extinction threat. We allow that artificially superintelligent beings are possible. This leaves open the question of how much we should worry about them. We treat ASI as a charismatic extinction threat. Our starting analysis of charisma comes from the sociologist Max Weber (1947). We extend the concept of charisma beyond individual personalities to events including extinction threats. Our principal example of a charismatic extinction threat is Skynet, the human-unfriendly AI of the movies of the Terminator franchise. Skynet’s charisma interferes with the processes by which we rationally evaluate future risks. Our exploration of the psychological and emotional dimensions of assessing extinction threats considers work by the Nobel laureate economist Robert Shiller (2019) in the emerging field of narrative economics. We connect the virality of extinction stories with the work of the psychologist Elke Weber. According to Elke Weber (2010) we have a finite pool of worry to allocate to all of our future concerns. The charisma of Skynet means that we risk worrying too much about it and, as a consequence, worrying insufficiently about the uncharismatic challenge of climate change. We conclude with a brief discussion of a proposal that could lead to a more rational allocation of worry about extinction and other threats to humanity. We counsel imagination insurance for an intrinsically uncertain future.
References
Agar, N. (2016). Don’t Worry about Superintelligence. Journal of Ethics and Emerging Technologies 26(1): 73-82.
Bostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. Oxford: Oxford University Press.
Courchanp F. et al. (2018). The Paradoxical Extinction of the Most Charismatic Animals. PLOS Biology 16(4): e2003997.
Kean, S. (2025). The Comet Panic of 1910, Revisited. Science History Institute. Available at: https://www.sciencehistory.org/stories/magazine/the-comet-panic-of-1910-revisited/
Keil, F. (2006). Doubt, deference, and deliberation: Understanding and using division of cognitive labor. In Oxford Studies in Epistemology. Tamar Szabo Gendler and John Hawthorne (eds.). Oxford: Oxford University Press, 2006, pp. 143-166.
Lynas, M. et al. (2021). Letter to Environmental Research “Greater than 99% consensus on human caused climate change in the peer-reviewed scientific literature” Environ. Res. Lett. 16 114005DOI 10.1088/1748-9326/ac2966
Mukherjee, S. (2010). The Emperor of All Maladies: A Biography of Cancer. New Yor: Scribner.
Parshall, A. (2023). Could the Zombie Fungus in TV’s The Last of Us Really Infect People?. Scientific American. Available at: https://www.scientificamerican.com/article/could-the-zombie-fungus-in-tvs-the-last-of-us-really-infect-people/
Shiller, R. (2019). Narrative Economics: How Stories go Viral and Drive Major Economic Events. Princeton University Press.
Stephens-Davidowitz, S. (2017). Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are. Dey Street Books.
Taylor, C. (2014). How Star Wars Conquered the Universe. New York: Basic Books.
Tegmark, M. (2017). Life 3.0: Being Human in the Age of Artificial Intelligence. New York: Knopf.
Vilaça, M. & Lavazza, A. (2022). Not Too Risky: How to Take a Reasonable Stance on Human Enhancement. Filosofia Unisinos 23(3): e23305.
Weber, E. (2010). What Shapes Perceptions of Climate Change? Climate Change (1): 332-342.
Weber, M. (1947). The Theory of Social and Economic Organization. Oxford: Oxford University Press.
Yampolskiy, R. (2015). Artificial Superintelligence: A Futuristic Approach. New York: Chapman and Hall/CRC Press.
Zuboff, S. (2019). The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. New York: Profile Books.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Nicholas Agar, Murilo Vilaça
This work is licensed under a Creative Commons Attribution 4.0 International License.
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution 4.0 International license (CC-BY 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are encouraged to post their work online (e.g., in institutional repositories or on their website) after publication, while providing bibliographic details that credit JEET (See The Effect of Open Access).
How to Cite
