Sometimes people have an argument against something, and they say "I don’t understand people’s position on x", but they really do understand, it’s just not a charitable interpretation. This is not that thing. I do not understand the position that GPT-N can suffer in the same way that an actual physical person or chicken can. It’s easier for me to sympathize with the position that the earth is flat or that tarot cards work, because those are things for which direct contradicting evidence doesn’t appear to people every day. Some of this genuine puzzlement will nevertheless be mixed with rhetorical questions designed to soothe my frustration, but the confusion is real. It’s possible that most of this post will come across as belittling, and I apologize—there are clearly people way smarter and more rational than me that believe this. I’m just walking through my entire thought process because to me it’s such a strange idea that I don’t know which part is actually contested. Anesthesiology is the branch of medicine dedicated to preventing people from suffering during medical operations. Most people in first world countries have taken anesthetics at some point in their lives. The acceptance or relevance of anesthetics by patients has no simple requirement on intelligence or rationality. It does not depend at all on whether or not someone thinks it’s "right" to feel pain in an abstract sense. A person with the intellectual capacity of a chimp and with very limited self awareness can be given anesthetics, and feel pain. Certainly dead people don’t feel pain, and the brain is a large part of the process by which pain asserts itself, but dentists do not hand out IQ tests to people to see if they need anesthetics before cutting someone’s mouth open. People can even feel pain while they are asleep. The capability of the party in question to "compute" and "simulate" are definitively not requirements for suffering to happen. Any proposal that sentience is the key defining factor in whether or not something can experience things needs to explain why people’s emotions and disposition are so easily affected by chemical injections that don’t appear to involve or demand any part of their self awareness. Yet these very smart, very rational people seem to suspect that if you have a computer that symbolically approximates the thought process of a human, you have a literal human with feelings and moral weight. I’m not even talking about simulating the entire human brain or body, even though I think the idea of ‘simulations’ suffering is patently absurd. I’m talking about using an AI trained to predict the next word, randomly selecting from the top 80% probability space of those words, and then repeating the loop. Eliezer apparently believes that if there were a pretrained transformer smart enough to accurately predict human dialog, and you ran top-k with a prompt about the holocaust, somehow (I think) the process of text generation would cause suffering. It wouldn’t have to be simulating a central nervous system or anything like that. As long as the text was describing human suffering, and an AI computed correct "enough" (enough being unspecified, of course) token probabilities, that’s all you need. What the hell is he talking about? I know this is more than just him because I hear this kind of talk all of the time. I have the same reaction to these ideas that I have to the people who develop really fancy theories about laughter, that have obvious contradictions in 90% of laughter that occurs in daily life. Have these people never heard of drugs? When I take cocaine, the part of my brain that solves math equations is nowhere to be found in the ensuing acute high. It’s not an instrumentally relevant component of my body, just like my ability to walk or see. What is the difference under this theory of "computation therefore qualia" between GPT-N and a human true crime author imagining what it be like to experience a fictional dismemberment? When does my murder novel, or perhaps murder novel writing, become a real example of torture and death under this theory of human experience? Is the difference between the author and GPT-N granularity? Accuracy? If I spend years doing the computations on pen and paper, does that work too? What if I get GPT-N to start writing murder novels and simulate the thought process of GPT-(N-1), a murder victim who features in the story? How would you administer heroin to a pretrained transformer?
I don’t know whether chickens or GPT-3 feel pain, but I have higher weightings on chickens feeling pain than GPT-3 doing so. However, my reason for having weights in this direction is nothing like yours. You seem to be reading the term "computation" as being explicit, symbolic computation. It isn’t. Even an unconscious human’s brain does an enormous amount of computation in the sense being meant, all the time. The idea of "computation" here is an abstraction that refers to the process of changing from one brain state to another, in the sense of the operation of all the neurons, synapses, hormones, neurotransmitters, and anything else that describes how the brain does what it does.
Comment
Comment
Of course it’s possible to lie. The argument goes the opposite direction: if in such a scenario the person really did feel different, it would be impossible for them to tell the truth, and that seems an extraordinary claim.
Comment
That makes more sense to me, actually.
After some more reading I think I understand better the (IMO completely bonkers) logic behind worrying if a GPT-N system is suffering. There are three pillars to this idea:
Comment
People do indeed worry about this, leading to things like ‘Solomonoff-weighted utilitarianism’ that assign higher moral relevance to minds with short description lengths.
The only core idea here is (1). (2) is completely superfluous. Discard it. (3) is a possible reason for concern, not any sort of "this is definitely happening" argument. Given that we don’t know what processes have qualia, that humans definitely do have qualia (well, at least I do), there’s at least some tiny amount more evidence that something simulating a human really well experiences qualia than things that don’t.
Comment
There’s a difference between saying that pleasure/pain accompany (mis)prediction , and saying that they are identical. The former doesn’t guarantee anything about an AI .
I think you’re confused by what the word "computation" means, in this context. If you take a fully-detailed mathematical description of physics, and combine it with a fully-detailed description of the configuration of atoms that makes up a brain, and you play the simulation forward, that’s a computation. It’s also a mind. You can abstract this a little bit; eg, you might only specify neurons and their connections, rather than individual atoms. That’s still mostly the same thing, it’s still a computation, and it’s still a mind. And if the neuron-connections you specify are the neuron-connections of a person who’s asleep, that, too, is a mind, and is a computation. All of these computations involve really, really astronomically large numbers of elements and steps. Whether the world’s largest supercomputers are big enough to run this sort of simulation, appropriately abstracted, is an open question. Whereas when you say:
Comment
I think you are confusing necessity and sufficiency. Of course it takes an enormous amount of compute to perform a fine grained simulation of physics...so it is necessary to perform a simulation of physics in order to achieve artificial qualia. But it’s still only a simulation: if no simulation, however detailed, is sufficient to have real qualia, it doesn’t .
When I said:
Comment
I’m moderately sure that GPT-3 is not sentient, but I accept the possibility that a hypothetical society with enough pen-and-paper might well be able to create something that would reasonably be called a person, and that the person so created could experience horrible things. I’m virtually certain that a mere hundred thousand people working for ten years won’t do it, though.
Comment
Comment
It’s not obfuscated, it’s just being precise. I believe that quantity matters, and you’re trying to handwave away the quantities involved to make it seem ridiculous. In this case, the quantities required are so large that I’m not even sure that a society could coordinate such an immense effort. But yes, setting complexity and scale difficulties aside, I do believe that. However, that handwaving is comparable to dismissing all life on Earth as just "chemicals mixed together". It has connotations like maybe as many as 10 different molecules instead of billions, a featureless soup instead of very detailed structures, thermodynamic equilibrium instead of active homeostasis far from equilibrium. In other words, a dismissive rhetorical trick instead of actually exploring the hypothesis.
Comment
The reason it seems silly to me has nothing to do with the quantity involved and everything to do with how abstract the suffering seems, and how ill-defined the map between ink and human experience is.
Comment
It’s only abstract because you’re choosing to visualize at the wrong level. Human suffering would seem equally abstract if you were incapable of seeing a person’s behaviour in real time, and could only examine them on the scale of a single cell at a time, and slowed down a billion to one.