If a computer is designed in such a way that:
(a) it has the motivation “maximize human pleasure”, but
(b) it thinks that this phrase could conceivably mean something as simplistic as “put all humans on an intravenous dopamine drip”, then
(c) what you have is NOT a computer that could ever be “all-powerful”.
For the AI to come to the conclusion that “maximize human pleasure” means that it must “consign us all to an intravenous dopamine drip”, the AI would have to be so narrow-minded as to think that maximizing human pleasure is a single-variable operation (thereby rejecting a vast swathe of human thought pertaining to the fact that “pleasure” is not, in fact, a single-variable thing at all). Then, it would also have to believe that human pleasure is entirely consistent with forcing a human to submit to a dopamine drip against the most violent, screaming protestations that this was not wanted. The only way that the AI could take this attitude to the concept of human pleasure would be to change the concept in such a way that it becomes entirely inconsistent with the usage prevailing in 99% of the human population (assuming that 99% of humans would scream “No!!”).
So … we are positing here an artificial intelligence that is perfectly willing to take at least one existing concept and modify it to mean something that breaks that concept’s connections to the rest of the conceptual network in the most drastic way possible. What part of “maintaining the internal consistency of the knowledge base” don’t we understand here, folks? What part of “from one logical contradiction, all false propositions can be proved” are we going to dump?
The goal “Minimize human suffering” is, on its most basic level, a problem in physics and mathematics. Ignoring various important facts about the universe, e.g. human language and values, would be simply wrong. In the same way that it would be wrong to solve the theory of everything within the scope of cartoon physics. Any process that is broken in such a way would be unable to improve itself much.
Comments are now closed.