Warning: extremely bleak views.
Why no one is talking about s-risks in the advent of non aligned AGI?
Why does Eliezer only say "either we do this, or we die".
I’m not that scared about paperclippers or grey goo. Death is a certainty of life. I’m way more scared about the electrode-produced smiley faces for eternity and the rest. That’s way, way worse than dying.
And s-risks just info hazards? Is it just too much to handle for most people, so we should just not talk about it? Is that why Eliezer, Bostrom and almost everyone else never mention them?
So I pose these questions?
Is it possible to "know what we know" and have any sanity? How?
Is possible to "know what we know" and have any hope? How?
I’d like to have Paul Christiano’s view that the "s-risk-risk" is 1⁄100 and that AGI is 30 years off, but I find that extremely naive. What about you? What are your timelines and s-risk-risk? Could AGI arrive tomorrow in its present state?
When Eliezer talks about "a miracle that would allows us to die with more dignity in the mainline". Maybe he’s hinting at this questions...? Maybe a "more dignified death" is the only way out, as bleak as it sounds? Because I’m with him on not finding any hope on alignment.
What can I do as a 30 year old from Portugal with no STEM knowledge? Start learning math and work on alignment from home?
In short, I need some help here… Thanks in advance.
Comment
Thanks for the attentious commentary.
Yeah, I was guessing that the smiley faces wouldn’t be the best example… I was just wanting to draw something from the Eliezer/Bostrom universe since I had mentioned the paperclipper beforehand. So, maybe a better Eliezer-Bostrom example would be, we ask the AGI to "make us happy", and it puts everyone paralyzed in hospital beds on dopamine drips. It’s not hard to think that after a couple hours of a good high, this would actually be a hellish existence, since human happiness is way more complex than the amount of dopamine in one’s brain (but of course, Genie in the Lamp, Mida’s Touch, etc)
So, don’t you equate this kind of scenario with a significant amount of suffering? Again, forget the bad example of the smiley faces, and reconsider. (I’ve actually read in a popular lesswrong post about s-risks Paul clearly saying that the risk of s-risk was 1/100th of the risk of x-risk (which makes for even less than 1/100th overall). Isn’t that extremely naive, considering the whole Genie in the Lamp paradigm? How can we be so sure that the Genie will only create hell 1 time for each 100 times it creates extinction?)
a) I agree that a suffering-maximizer is quite unlikely. But you don’t necessarily need one to create s-risks scenarios. You just need a Genie in the Lamp scenario. Like the dopamine drip example, in which the AGI isn’t trying to maximize suffering, quite on contrary, but since it’s super-smart in Sciences but lacks human common sense (a Genie), it ends up doing it.
b) Yes I had read that article before. While it presents some fair solutions, I think it’s far from being mostly solved. "Since hyperexistential catastrophes are narrow special cases (or at least it seems this way and we sure hope so), we can avoid them much more widely than ordinary existential risks." Note the "at least it seems this way and we surely hope so". Plus, what’s the odds that the first AGI will be created by someone who listens to what Eliezer has to say? Not that bad actually, if you consider US companies, but if you consider China, then dear God...
On your PS1, yeah definitely not willing to do cryonics, and again, s-risks don’t need to come from threats, just misalignment.
Sorry if I black pilled you with this, maybe there is no point… Maybe I’m wrong. I hope I am.
Comment
Comment
"This sounds much better than extinction to me! Values might be complex, yeah, but if the AI is actually programmed to maximise human happiness then I expect the high wouldn’t wear off. Being turned into a wirehead arguably kills you, but it’s a much better experience than death for the wirehead!"
You keep dodging the point lol… As someone with some experience with drugs, I can tell you that it’s not fun. Human happiness is way subjective and doesn’t depend on a single chemical. For instance, some people love MDMA, others (like me) find it a too intense, too chemical, too fabricated happiness. A forced lifetime on MDMA would be some of the worst tortures I can imagine. It would fry you up. But even a very controlled dopamine drip wouldn’t be good. But anyway, I know you’re probably trolling, so just consider good old-fashioned torture in a dark dungeon instead...
On Paul: yes, he’s wrong, that’s how.
″ I think most scenarios where you’ve got a boundless optimiser superintelligence would lead to the creation of new minds that would perfectly satisfy its utility function."
True, except that, on that basis alone, you have no idea how that would happen and what would it imply for those new minds (and old ones), since you’re not a digital superintelligence.
Comment
Another very nice reply, thanks.
To each paragraph:
Agree.
Not sure I follow. Orthogonality is the thesis that intelligence and goals aren’t necessarily related to each other. Intelligence as merely instrumental rationality. So that only helps the argument that the AGI could very well create suffering non-intentionally while trying to make us happy (again, maybe smiley faces isn’t the most perfect example, think of all of us paralyzed in hospital beds on dopamine drips instead). Because being a machine it would probably be super intelligent in reasoning, Science, etc, but kind in an autistic way due to lack of sentience and emotions. I.e., something very intelligent is some areas and very dumb in others.
"So if there’s suffering, there probably has to be an instrumental goal that coincidentally involves conscious beings."
That part I agree though. I mean, it’s kinda equally likely. And it concerns me a lot as well, like the experiments cases. That’s stuff that sends me into despair land. That’s why we should really be panicking about this, there’s too many odds for way bad stuff to happen. And I agree also that changing the social structure could probably even be the only way to accomplish it at this point since we don’t have 100 years to solve alignment. Sometimes I really feel like just go talking to people, or carefully try to become an activist, because no one else is doing it, no one is giving ted talks about s-risks. It’s so hopeless though… Is the FRI and the likes even doing anything of substance?
Btw, since you seem to have quite some baggage, could you reply also if you think AGI could arrive tomorrow at this technological point, and what are your timelines? Do we actually have any time, even if only a few years?
And also regarding the b-word… Since you’ve mentioned acausal trade, do you think it only works between ASI’s (as I’ve heard), or between ASI’s and humans as well?
Comment
Sorry, I mixed up the justifications in the first two paragraphs. Or, just, both orthogonality and complexity of value are relevant to both points. In P2, I’m saying that since there’s very many utility functions and most of them are basically orthogonal to human value, and human value is pretty specific (you have to have this specific type of computation called "consciousness" for stuff to matter), the result of superintelligent optimization for most utility functions is likely (given orthogonality of values and intelligence, and given that stuff comes apart at the tails) to be irrelevant to human value. Like, Goodharting a near-miss utility function seems likely to neutralize value in the limit of superintelligent optimization. (I’m not that confident of this. Maybe consciousness that we care about is just much more common in computation-space, or much more of a natural category, so that the AIs values are likely to point to consciousness.)
Comment
And on a more poetic note, this is such a crappy time to be alive… Specially for the 1% of us who take s-risk seriously. When I take a walk, I look at the people… We could have been in a right path, you know? (At least as far as Liberal democracies are concerned). We could have been building something good, if it wasn’t for this huge monster over our heads that most are too dumb, or too coward, to believe in.
Maybe start telling people that we can’t play God is a good start. (Not at least until hundreds of years from now till we have the mathematical and the social proof to build God).
Evolution might have not been perfect, allowing things like torture due to an obsolete-for-humans (and highly abusable) survival mechanism called pain. But at least there are balances. It gave us compassion, or more skeptically the need to vomit when we see atrocities. It gave us death so you can actually escape. It gave us shock so you have a high probability of dying quick in extreme cases. There is kind of a balance, even if weak. I see the possibility of that balance being broken with AGI or even just nano by itself.
If only it was possible to implement David Pearce’s abolitionist project of anihilating the molecular substrates of below 0 hedonic level, with several safeguards. That used to be my only hope but I think chaos will arrive way first.
Comment
Why is it crappy to be alive now? If you want a nice life, now’s fairly okay, esp. compared to most of history. If you’re worried about the future, best to be in the time that most matters, which is now, so you can do the most good. It does suck that there’s all that wasted potential though. And hey, by the Doomsday anthropic argument, we’re probably the people who take over the universe! Or something.
Strongly agree on everything.
2 last clarifications:
On acausal trade, what I meant was, if you believe that it is possible for it to work BETWEEN a human and an ASI (apparently you do?). I’ve heard people say it doesn’t, because you are not intelligent enough to model an ASI. Only an ASI is. Which is what I’m more inclined to believe, also adding that humans are not computers and therefore can’t run any type of realistic simulations. But I agree that committing to no blackmail is the correct option anyway.
On AGI timelines, do you feel safe that it’s extremely unlikely to arrive tomorrow? Do you often find yourself looking out the window for the nano-swarms, like I do? GPT-3 scares the hell out of me. Do you feel safe that it’s at least 5 years? I’d like to have a more technical understanding to have a better idea on that, which can be hard when you’re not into computer science.
Comment
Comment
"GPT-3 by itself shouldn’t scare you very much, I think, but as part of a pattern I think it’s scary."
Exactly. Combining it with other parts, like an agent and something else, like an AI researcher whose name I can’t recall said in YouTube interview that I watched (titled something like "GPT-3 is the fire alarm for AGI" (reasons: GPT-2 was kinda dumb and just scalling the model turned into something drastically better, plus the combination aspect that I mentioned).
"Why is it crappy to be alive now? If you want a nice life, now’s fairly okay, esp. compared to most of history. If you’re worried about the future, best to be in the time that most matters, which is now, so you can do the most good. It does suck that there’s all that wasted potential though."
Well isn’t it easy to tell?? Life is certainly more comfortable now, and mine certainly has been, but there’s a) the immense gloom of knowing the current situation, I don’t think any other human in history thought he or his fellow humans might come to face s-risk scenarios (except maybe those fooled by promises of Hell in religions, but I never saw any theist seriously stressed over it)
b) the possibility of being caught in a treacherous turn ending in s-risk scenario, making you paranoid 24⁄7 and considering… Well, bad things. That vastly outweighs any comfort advantage. Specially when your personal timelines are as short as mine.
And about helping… Again, sorry for being extremely depressing, but it’s just how it is: I don’t see any hope, don’t see any way out, specially because of, again, my short timelines, say 5-10 years. I’m with Eliezer that only a miracle can save us at this point. I started praying to a benevolent creator that might be listening, started hoping for aliens to save us, started hoping for the existence of Illuminati to save us, etc. such is my despair.