From: Matt Mahoney (matmahoney@yahoo.com)
Date: Wed Feb 20 2008 - 20:59:11 MST
--- Daniel Burfoot <daniel.burfoot@gmail.com> wrote:
> The scenario I'm most afraid of is not a hard take-off leading to unfriendly
> AGI, but a pseudo-AI falling into the hands of evil men.
Bad science fiction. AI will evolve as a collective intelligence distributed
over the internet, not in some lab. It will be in everybody's hands.
The scenario I'm most afraid of is our own stupidity. CEV is a good idea, but
it won't be implemented because people want their immediate wishes granted
right now, not their extrapolated wishes.
> Furthermore, an intelligent system need not be fully general in order to
> give a small group of people the ability to dominate/enslave humanity. Think
> 1984, except Big Brother is equipped with a sophisticated computer vision
> and speech recognition system, so that the Ministry of Truth instantly knows
> if you've spoken anything ungrammatical (that is, your sentence was invalid
> under the grammar of NewSpeak). The Microsoft Office grammar checker becomes
> rather more sinister in this light.
Orwell did not predict that we would want surveillance. Nor is it necessary
to enslave humans by force. The pleasure of a thousand permanent orgasms is
just a matter of changing a few lines of code in your brain.
-- Matt Mahoney, matmahoney@yahoo.com
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT