72. Mind projection fallacy

The mind projection fallacy is when we unconsciously attribute to subjects properties which only exist in our representations of them. The concept originates with E.T. Jaynes, a physicist and Bayesian philosopher with a highly sophisticated and nuanced understanding of probability theory. He used the phrase to argue against the Copenhagen interpretation of quantum mechanics52:

[I]n studying probability theory, it was vaguely troubling to see reference to "gaussian random variables", or "stochastic processes", or "stationary time series", or "disorder", as if the property of being gaussian, random, stochastic, stationary, or disorderly is a real property, like the property of possessing mass or length, existing in Nature. Indeed, some seek to develop statistical tests to determine the presence of these properties in their data...

Once one has grasped the idea, one sees the Mind Projection Fallacy everywhere; what we have been taught as deep wisdom, is stripped of its pretensions and seen to be instead a foolish non sequitur. The error occurs in two complementary forms, which we might indicate thus: (A) (My own imagination) → (Real property of Nature), [or] (B) (My own ignorance) → (Nature is indeterminate)

Yudkowsky (2008) uses the term to refer to the way people are prone to think about Artificial Intelligence. They take a disposition, say, “nice,” which may be their own or what they hope AI will be, and they project that on to every possible consideration of advanced, agent-like Artificial Intelligence that they can come up with. They are engaging in projecting rather than considering the full expanse of possibilities. This is bound to occur when people are considering complex new technologies, particularly Artificial Intelligence. Another pernicious aspect of the mind projection fallacy concerns the property of ignorance—taking our own ignorance and projecting it onto an external object, as if the object were itself inherently mysterious. That is impossible, however. The mystery is a property of our mind, not the object itself.