From: Tennessee Leeuwenburg (hamptonite@gmail.com)
Date: Thu Jul 21 2005 - 20:20:42 MDT
Chris:
The point is that our individual *cannot learn* from physics alone,
what the experiencing of seeing a red rose is like. If she could learn
from physics alone, then she would not experience anything new,
period. She would be entirely un-surprised by what she experienced,
even though it was the first time, because she would know what to
expect.
It is not a challenge to the thought experiment to draw a distinction
between knowing the theory and knowing how to use it, like Michael
did.
I agree with him when he says, "This should be obvious; memorising
detailed instructions on how to ride a bicycle does not immediately
grant
you the ability to ride a bike competently, because you cannot
deliberatively modify your neural circuitry with an act of will."
But that is precisely what is interesting. A human cannot understand
logically everything that they can learn, nor can they describe with
phyics everything that is immanent (loosely, "real") to them.
This is, per se, interesting. This is where the debate lies. It was
not my intent to prove an argument with this, but it seemed that some
people were unfamiliar with the intuitive appeal of "qualia", and I
thought the thought experiment would help.
On 7/22/05, Eliezer S. Yudkowsky <sentience@pobox.com> wrote:
> > One day she lands on Earth at the end of her mission. Upon opening the
> > hatch, she casts her eyes first on an enormous bunch of red roses
> > which have been given to her.
> >
> > "Oh", she says, "so that's what it's like".
> >
> > Has she learnt anything new about colour? If you accept that she has,
> > then qualia must be real, because she already knew everything that
> > science could inform her about the world and about colour. There must,
> > therefore, be something real about colour which is not addressed by
> > science.
>
> Not that I'm still at the stage of my career when this sort of thing seemed
> mysterious to me, but:
>
> What new predictions can the scientist make once she actually sees a >bunch of red roses?
She can make new qualitative predictions. Even if I were to accept
(which I don't), that minds are reducible to brains, perfect physical
knowledge could stil only make predictions at the physical level.
Without a qualitative understanding of "how that feels" to a person,
the scientist would be un-able to provide a proper description of
experience / qualia.
Physics, for example, doesn't enable to me understand what language
means, nor does merely understanding the grammar and syntax and
symbolism of a language allow me to use it.
The suggestion is that for all its physical knowledge, humans have
access to knowledge that a nonfeeling machine simply cannot.
A machine, albeit capable of /acting in a way perfectly coherent with
a perfect understanding of physics and the appearance of intelligence/
might be no more than a big rock, as far as any sense of
self-understanding or greater meaning is concerned.
I agree that a machine seems capable of true intelligence,
consciousness and qualia. But equally, I think a machine is capable of
a lifeless mimic.
If consciousness is our inner life, and qualia is what that
consciousness is like, then a machine without qualia is a machine
without an inner life.
To me, that is the greater danger, compared with the possibility of an evil AGI.
Cheers,
-T
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT