2. Underestimating non-obvious risks

Global risks are divided into two categories: obvious and non-obvious. An obvious risk would be nuclear war, a non-obvious risk would be the destruction of an Earth by the creation of a stable strangelet during a particle accelerator experiment. Non-obvious risks may be more dangerous, because their severity and probability are unknown, and therefore suitable countermeasures are generally not taken. Some non-obvious risks are known only to a narrow circle of experts who express contradictory opinions about their severity, probability, and mechanism of emergence. A detached onlooker, such as a military general or a head of state, may be completely unable to distinguish between the expert advice given and might as well flip a coin to determine who to listen to regarding the non-obvious risks. This makes inadequate preparation for these risks highly likely, whereas well-understood risks such as nuclear war are better anticipated and ameliorated.

Making estimates based on past rates of discovery of new global risks, it seems as if the number of new risks expands exponentially over time. Therefore we can anticipate a great increase in the number of global risks in the 21st century, the nature of which may be impossible for us to guess now, and fall into the category of non-obvious risks.

Obvious risks are much more convenient to analyze. There are huge volumes of data we can use to assess these perils. The volume of this analysis can conceal the fact that there are other risks about which little is known. Their assessment may not be amenable to rigorous numerical analysis, but they are severe risks all the same (for example, risks from incorrectly programmed Artificial Intelligence).