40. Generalizing from fictional
evidence
In the ancestral environment, where our
bodies and brains evolved and were formed, there was no such thing as movies.
If you saw something happening, it was real. So our brains are not well suited
to telling the difference between movies and reality. We think about movies
subconsciously, or even consciously, as if they actually happened, though they
are completely made up. Unfortunately, the scientific understanding level of a
Hollywood screenwriter is usually somewhere between that of a 7th
grader and an 8th grader. The tendency to recall movies and books as
if they were actual events is called generalizing from fictional evidence27.
The most audacious example are movies
about Artificial Intelligence, which postulate that AIs would have human-like,
or anthropomorphic thinking, such as clannishness, anthropomorphic rebellion,
or a desire for revenge. An AI, being a machine, would not have any of these
animalistic tendencies unless they were explicitly programmed into it, which
they likely would not be. Thus, the greatest danger to humanity is from AI that
is indifferent or insufficiently benevolent to us, and emphatically not AI that
has a specific grudge or malevolence against us28. This crucial
point makes all the difference in the world in terms of how we will design an
AI to optimize for safety.
Another issue with fiction, previously
discussed, is that futuristic stories tend to make the future similar to today,
but with just a few added details. For instance, in Total Recall, the
technology was very similar to that of the year when the movie was made (1990),
except there was interplanetary travel and slightly more advanced computer. In Back
to the Future (1985), the main differences of the future appeared to be
hoverboards and flying cars. In the real future, many details will
simultaneously be different, not just a few.
Yet another problem unique to fiction
is that forces that clash tend to be equally balanced. If Star Wars were
real, the Empire would just use the Death Star to blow the entire Rebel fleet
out of the sky. If Terminator were real, the assassin robot would just
snipe the protagonist from a mile away, without ever being seen. If The
Matrix Reloaded were real, the AI would just destroy the subterranean human
city of Zion with nuclear weapons. In reality, extreme power asymmetries and
ÒunfairÓ match-ups happen all the time. In 1518-1520, about 90-100 Spanish
cavalry and 900-1,300 infantry were able to conquer and subjugate the Aztec
civilization, an empire of millions of people.