40. Generalizing from fictional evidence

In the ancestral environment, where our bodies and brains evolved and were formed, there was no such thing as movies. If you saw something happening, it was real. So our brains are not well suited to telling the difference between movies and reality. We think about movies subconsciously, or even consciously, as if they actually happened, though they are completely made up. Unfortunately, the scientific understanding level of a Hollywood screenwriter is usually somewhere between that of a 7^th grader and an 8^th grader. The tendency to recall movies and books as if they were actual events is called generalizing from fictional evidence²⁷.

The most audacious example are movies about Artificial Intelligence, which postulate that AIs would have human-like, or anthropomorphic thinking, such as clannishness, anthropomorphic rebellion, or a desire for revenge. An AI, being a machine, would not have any of these animalistic tendencies unless they were explicitly programmed into it, which they likely would not be. Thus, the greatest danger to humanity is from AI that is indifferent or insufficiently benevolent to us, and emphatically not AI that has a specific grudge or malevolence against us²⁸. This crucial point makes all the difference in the world in terms of how we will design an AI to optimize for safety.

Another issue with fiction, previously discussed, is that futuristic stories tend to make the future similar to today, but with just a few added details. For instance, in Total Recall, the technology was very similar to that of the year when the movie was made (1990), except there was interplanetary travel and slightly more advanced computer. In Back to the Future (1985), the main differences of the future appeared to be hoverboards and flying cars. In the real future, many details will simultaneously be different, not just a few.

Yet another problem unique to fiction is that forces that clash tend to be equally balanced. If Star Wars were real, the Empire would just use the Death Star to blow the entire Rebel fleet out of the sky. If Terminator were real, the assassin robot would just snipe the protagonist from a mile away, without ever being seen. If The Matrix Reloaded were real, the AI would just destroy the subterranean human city of Zion with nuclear weapons. In reality, extreme power asymmetries and “unfair” match-ups happen all the time. In 1518-1520, about 90-100 Spanish cavalry and 900-1,300 infantry were able to conquer and subjugate the Aztec civilization, an empire of millions of people.