On Artificial Superintelligence and the Problem of Charismatic Extinction Threats

Abstract

This paper focuses on the challenge of finding a rational response to Artificial Superintelligence (ASI) as an extinction threat. We allow that artificially superintelligent beings are possible. This leaves open the question of how much we should worry about them. We treat ASI as a charismatic extinction threat. Our starting analysis of charisma comes from the sociologist Max Weber (1947). We extend the concept of charisma beyond individual personalities to events including extinction threats. Our principal example of a charismatic extinction threat is Skynet, the human-unfriendly AI of the movies of the Terminator franchise. Skynet’s charisma interferes with the processes by which we rationally evaluate future risks. Our exploration of the psychological and emotional dimensions of assessing extinction threats considers work by the Nobel laureate economist Robert Shiller (2019) in the emerging field of narrative economics. We connect the virality of extinction stories with the work of the psychologist Elke Weber. According to Elke Weber (2010) we have a finite pool of worry to allocate to all of our future concerns. The charisma of Skynet means that we risk worrying too much about it and, as a consequence, worrying insufficiently about the uncharismatic challenge of climate change. We conclude with a brief discussion of a proposal that could lead to a more rational allocation of worry about extinction and other threats to humanity. We counsel imagination insurance for an intrinsically uncertain future.