This is a repository for miscellaneous short things I want to post. Other people are welcome to make top-level comments here if they want. (E.g., questions for me you’d rather discuss publicly than via PM; links you think will be interesting to people in this comment section but not to LW as a whole; etc.)
Shared with permission, a google doc exchange confirming Eliezer still finds the arguments for alignment optimism, slower takeoffs, etc. unconvincing:
Comment
FWIW, I think Yudkowsky is basically right here and would be happy to explain why if anyone wants to discuss. I’d likewise be interested in hearing contrary perspectives.
Comment
Which of the "Reasons to expect fast takeoff" from Paul’s post do you find convincing, and what is your argument against what Paul says there? Or do you have some other reasons for expecting a hard takeoff? I’ve seen this post of yours, but as far as I know, you haven’t said much about hard vs soft takeoff in general.
Comment
It’s a combination of not finding Paul+Katja’s counterarguments convincing (AI Impacts has a slightly different version of the post, I think of this as the Paul+Katja post since I don’t know how much each of them did), having various other arguments that they didn’t consider, and thinking they may be making mistakes in how they frame things and what questions they ask. I originally planned to write a line-by-line rebuttal of the Paul+Katja posts, but instead I ended up writing a sequence of posts that collectively constitute my (indirect) response. If you want a more direct response, I can put it on my list of things to do, haha… sorry… I am a bit overwhelmed… OK here’s maybe some quick (mostly cached) thoughts:
Comment
Thanks! My understanding of the Bostrom+Yudkowsky takeoff argument goes like this: at some point, some AI team will discover the final piece of deep math needed to create an AGI; they will then combine this final piece with all of the other existing insights and build an AGI, which will quickly gain in capability and take over the world. (You can search "a brain in a box in a basement" on this page or see here for some more quotes.) In contrast, the scenario you imagine seems to be more like (I’m not very confident I am getting all of this right): there isn’t some piece of deep math needed in the final step. Instead, we already have the tools (mathematical, computational, data, etc.) needed to build an AGI, but nobody has decided to just go for it. When one project finally decides to go for an AGI, this EMH failure allows them to maintain enough of a lead to do crazy stuff (conquistadors, persuasion tools, etc.), and this leads to DSA. Or maybe the EMH failure isn’t even required, just enough of a clock time lead to be able to do the crazy stuff. If the above is right, then it does seem quite different from Paul+Katja, but also different from Bostrom+Yudkowsky, since the reason why the outcome is unipolar is different. Whereas Bostrom+Yudkowsky say the reason one project is ahead is because there is some hard step at the end, you instead say it’s because of some combination of EMH failure and natural lag between projects.
Comment
Ah, this is helpful, thanks—I think we just have different interpretations of Bostrom+Yudkowsky. You’ve probably been around before I was and read more of their stuff, but I first got interested in this around 2013, pre-ordered Superintelligence and read it with keen interest, etc. and the scenario you describe as mine is what I always thought Bostrom+Yudkowsky believed was most likely, and the scenario you describe as theirs—involving "deep math" and "one hard step at the end" is something I thought they held up as an example of how things *could *be super fast, but not as what they actually believed was most likely. From what I’ve read, Yudkowsky did seem to think there would be more insights and less "just make blob of compute bigger" about a decade or two ago, but he’s long since updated towards "dear lord, people really are just going to make big blobs of inscrutable matrices, the fools!" and I don’t think this counts as a point against his epistemics because predicting the future is hard and most everyone else around him did even worse, I’d bet.
Comment
Ok I see, thanks for explaining. I think what’s confusing to me is that Eliezer did stop talking about the deep math of intelligence sometime after 2011 and then started talking about big blobs of matrices as you say starting around 2016, but as far as I know he has never gone back to his older AI takeoff writings and been like "actually I don’t believe this stuff anymore; I think hard takeoff is actually more likely to be due to EMH failure and natural lag between projects". (He has done similar things for his older writings that he no longer thinks is true, so I would have expected him to do the same for takeoff stuff if his beliefs had indeed changed.) So I’ve been under the impression that Eliezer actually believes his old writings are still correct, and that somehow his recent remarks and old writings are all consistent. He also hasn’t (as far as I know) written up a more complete sketch of how he thinks takeoff is likely to go given what we now know about ML. So when I see him saying things like what’s quoted in Rob’s OP, I feel like he is referring to the pre-2012 "deep math" takeoff argument. (I also don’t remember if Bostrom gave any sketch of how he expects hard takeoff to go in Superintelligence; I couldn’t find one after spending a bit of time.) If you have any links/quotes related to the above, I would love to know! (By the way, I was was a lurker on LessWrong starting back in 2010-2011, but was only vaguely familiar with AI risk stuff back then. It was only around the publication of Superintelligence that I started following along more closely, and only much later in 2017 that I started putting in significant amounts of my time into AI safety and making it my overwhelming priority. I did write several timelines though, and recently did a pretty thorough reading of AI takeoff arguments for a modeling project, so that is mostly where my knowledge of the older arguments comes from.)
Comment
For all I know you are right about Yudkowsky’s pre-2011 view about deep math. However, (a) that wasn’t Bostrom’s view AFAICT, and (b) I think that’s just not what this OP quote is talking about. From the OP: I feel like a bunch of people have shifted a bunch in the type of AI x-risk that worries them (representative phrase is "from Yudkowsky/Bostrom to What Failure Looks Like > part 2 part 1") and I still don’t totally get why. It’s Yudkowsky/Bostrom, not Yudkowsky. And it’s WFLLp1, not p2. Part 2 is the one where the AIs do a treacherous turn; part 1 is where actually everything is fine except that "you get what you measure" and our dumb obedient AIs are optimizing for the things we told them to optimize for rather than for what we want. I am pretty confident that WFLLp1 is *not *the main thing we should be worrying about; WFLLp2 is closer, but even it involves this slow-takeoff view (in the strong sense, in which economy is growing fast before the point of no return) which I’ve argued against. I do *not *think that the reason people shifted from "yudkowsky/bostrom" (which in this context seems to mean "single AI project builds AI in the wrong way, AI takes over world" and to WFLLp1 is that people rationally considered all the arguments and decided that WFLLp1 was on balance more likely. I think instead that probably some sort of optimism bias was involved, and more importantly win by default (Yud + Bostrom stopped talking about their scenarios and arguing for them, whereas Paul wrote a bunch of detailed posts laying out his scenarios and arguments, and so in the absence of visible counterarguments Paul wins the debate by default). Part of my feeling about this is that it’s a failure on my part; when Paul+Katja wrote their big post on takeoff speeds I disagreed with it and considered writing a big point-by-point response, but never did, even after various people posted questions asking "has there been any serious response to Paul+Katja?"
Comment
Re (a): I looked at chapters 4 and 5 of Superintelligence again, and I can kind of see what you mean, but I’m also frustrated that Bostrom seems really non-committal in the book. He lists a whole bunch of possibilities but then doesn’t seem to actually come out and give his mainline visualization/"median future". For example he looks at historical examples of technology races and compares how much lag there was, which seems a lot like the kind of thinking you are doing, but then he also says things like "For example, if human-level AI is delayed because one key insight long eludes programmers, then when the final breakthrough occurs, the AI might leapfrog from below to radically above human level without even touching the intermediary rungs." which sounds like the deep math view. Another relevant quote:
Comment
I also wish I knew what Bostrom’s median future was like, though I perhaps understand why he didn’t put it in his book—the incentives all push against it. Predicting the future is hard and people will hold it against you if you fail, whereas if you never try at all and instead say lots of vague prophecies, people will laud you as a visionary prophet. Re (b) cool, I think we are on the same page then. Re takeoff being too fast—I think a lot of people these days think there will be plenty of big scary warning shots and fire alarms that motivate lots of people to care about AI risk and take it seriously. I think that suggests that a lot of people expect a fairly slow takeoff, slower than I think is warranted. Might happen, yes, but I don’t think Paul & Katja’s arguments are that convincing that takeoff will be this slow. It’s a big source of uncertainty for me though.
I’d personally like to find some cruxes between us some time, though I don’t yet know the best format to do that. I think I’ll wait to see your responses to Issa’s question first.
Comment
Likewise! I’m up for a video call if you like. Or we could have a big LW thread, or an email chain. I think my preference would be a video call. I like Walled Garden, we could do it there and invite other people maybe. IDK.
Comment
Did this ever happen?
Comment
I don’t think so? It’s possible that it did and I forgot.
A belief can be a negation in the sense of a contradiction , whilst not being a negation in the sense of a disproof. I dont think EY disproved RH’s position. I dont think he is confident he did himself, since his summary was called "what I believe if not why I believe it". And I dont think lack of time was the problem, since the debate was immense.
Comment
Interesting, yeah I wonder why he titled it that. Still though it seems like he is claiming here to have disproved RH’s position to some extent at least. I for one think RH’s position is pretty implausible, for reasons Yudkowsky probably mentioned (I don’t remember exactly what Yud said).
Comment
Why the "seems"? A master rationalist should be able to state things clearly , surely?
Facebook comment I wrote in February, in response to the question ‘Why might having beauty in the world matter?’: I assume you’re asking about why it might be better for beautiful objects in the world to exist (even if no one experiences them), and not asking about why it might be better for experiences of beauty to exist. [… S]ome reasons I think this:
Comment
Somewhat more meta level: Heuristically speaking, it seems wrong and dangerous for the answer to "which expressed human preferences are valid?" to be anything other than "all of them". There’s a common pattern in metaethics which looks like:
Old discussion of this on LW: https://www.lesswrong.com/s/fqh9TLuoquxpducDb/p/synsRtBKDeAFuo7e3
Rolf Degen, summarizing part of Barbara Finlay’s "The neuroscience of vision and pain":
Comment
[Epistemic status: Thinking out loud] If the evolutionary logic here is right, I’d naively also expect non-human animals to suffer more to the extent they’re (a) more social, and (b) better at communicating specific, achievable needs and desires. There are reasons the logic might not generalize, though. Humans have fine-grained language that lets us express very complicated propositions about our internal states. That puts a lot of pressure on individual humans to have a totally ironclad, consistent "story" they can express to others. I’d expect there to be a lot more evolutionary pressure to actually experience suffering, since a human will be better at spotting holes in the narratives of a human who fakes it (compared to, e.g., a bonobo trying to detect whether another bonobo is really in that much pain). It seems like there should be an arms race across many social species to give increasingly costly signals of distress, up until the costs outweigh the amount of help they can hope to get. But if you don’t have the language to actually express concrete propositions like "Bob took care of me the last time I got sick, six months ago, and he can attest that I had a hard time walking that time too", then those costly signals might be mostly or entirely things like "shriek louder in response to percept X", rather than things like "internally represent a hard-to-endure pain-state so I can more convincingly stick to a verbal narrative going forward about how hard-to-endure this was".
[Epistemic status: Piecemeal wild speculation; not the kind of reasoning you should gamble the future on.] Some things that make me think suffering (or ‘pain-style suffering’ specifically) might be surprisingly neurologically conditional and/or complex, and therefore more likely to be rare in non-human animals (and in subsystems of human brains, in AGI subsystems that aren’t highly optimized to function as high-fidelity models of humans, etc.):
Comment
Devoodooifying Psychology says "the best studies now suggest that the placebo effect is probably very weak and limited to controlling pain".
How is the signal being kept "costly/honest" though? Is the pain itself the cost? That seems somewhat weird …
From Twitter:
https://intelligence.org/2017/11/25/security-mindset-ordinary-paranoia/
https://intelligence.org/2017/11/26/security-mindset-and-the-logistic-success-curve/ Goal stability over time is part of the problem, but not the core problem. The core problem (for ML) is ‘ML models are extremely opaque, and there’s no way to robustly get any complex real-world goal into a sufficiently opaque system’. The goal isn’t instilled in the first place. "3. The AI probably embarks on a big project which ignores us and accidentally kills us" Rather: which deliberately kills us because (1) we’re made of atoms that can be used for the project, and (2) we’re a threat. (E.g., we could build a rival superintelligence.)
Chana Messinger, replying to Brandon Bradford:
Comment
It’s a conspiracy, man. You gotta pay for water that’s just sitting there in the ground. It’s the corporations tricking people into thinking they gotta drink water that came from their pipes because it’s been "sanitized". Wake up sheeple, they’re sanitizing your brains!!! And don’t get me started on showing up later for work! We all know I could be there in just 15 minutes if there were no traffic, but you see the corporations make more money the longer you drive your car, so they make sure everyone has to be at work at the same time so there’s a traffic jam. Then we all use more fuel, end up buying fast food and other junk because we lost so much time in traffic, and gotta put in extra effort at work because the boss is always mad about how we’re showing up late. And to top it all of, all those exhaust fumes are causing ozone holes and global warming which is just what they want because then they can use it to sell you more stuff like sunscreen or to control your life to fight "climate change". As for the microwave? Well, let’s just say it’s not a coincidence they blow up if you put tin foil in them. Me? I’m keeping my tin foil right where intended: on my head to block out the thought rays and subliminal messages sent out by their so-called "internet".
From an April 2019 Facebook discussion: **Rob Bensinger: **avacyn:
Comment
I really like the FB crossposts here, and also really like this specific comment. Might be worth polishing it into a top-level post, either here or on the EA Forum sometime.
Comment
Thanks! :) I’m currently not planning to polish it; part of the appeal of cross-posting from Facebook for me is that I can keep it timeboxed by treating it as an artifact of something I already said. I guess someone else could cannibalize it into a prettier stand-alone post.
From an April 2019 Facebook discussion: Rob Bensinger:
‘I’m in a longstanding relationship that’s turned sour. But I feel like I can’t just leave (or make other changes to my life) because I’m not having enough fun / my life isn’t satisfying as many values as I’d like; I feel like I need to find something objectively Bad my partner has done, so that I can feel Justified and Legitimate in leaving.’ People often feel like they’re not "allowed" to take radical action to improve their lives, because of others’ seeming claims on their life.
A lot of the distinct issues raised on https://www.facebook.com/robbensinger/posts/10160749026995447, like Jessica Taylor’s worry about using moral debt as a social lever to push people around. In my experience, this is not so dissimilar from the relationship case above; people think about their obligations in fuzzy ways that make it hard to see what they actually want and easy to get trapped by others’ claims on their output.
People feel like they’re being looked down on or shamed or insufficiently socially rewarded/incentivized/respected for things about how they’re trying to do EA. Examples might include ‘starting risky projects’, ‘applying to EA jobs’, ‘applying to non-EA jobs’, ‘earning to give’, ‘not earning to give’, ‘producing ideas that aren’t perfectly vetted or write-ups that aren’t perfectly polished’. (See e.g. the comments on https://www.facebook.com/robbensinger/posts/10161249846505447; or for the latter point, https://www.lesswrong.com/posts/7YG9zknYE8di9PuFg/epistemic-tenure and a bunch of other recent writings share a theme of ‘highly productive intellectuals are feeling pressure to not say things publicly until they’re super super confident of them’).
People feel like they can’t (even in private conversation) acknowledge social status and esteem (https://www.facebook.com/robbensinger/posts/10160475363630447), respectability, or ‘Ra’-type things (https://srconstantin.wordpress.com/2016/10/20/ra), or how those phenomena affects everyone’s preferences or judgments.
Modest epistemology in Inadequate Equilibria (https://equilibriabook.com/toc/)
A lot of CFAR-style personal debugging I’ve done has depended on my ability to catch myself in the mental motion of punishing myself (or disregarding myself, etc.) on the timescale of ‘less than a second’. And then stopping to change that response or analyze why that’s happening, so I can drill down better on underlying dispositions I want to improve. Cf. https://wiki.lesswrong.com/wiki/Ugh_field and https://www.lesswrong.com/posts/5dhWhjfxn4tPfFQdi/physical-and-mental-behavior.
Comment
Rob Wiblin, August 2019:
Copied from some conversations on Twitter: · · · · · · · · · · · · · · · **Eric Rogstad: **I think "illusionism" is a really misleading term. As far as I can tell, illusionists believe that consciousness is real, but has some diff properties than others believe. It’s like if you called Einstein an "illusionist" w.r.t. space or time. See my comments here: https://www.lesswrong.com/posts/biKchmLrkatdBbiH8/book-review-rethinking-consciousness **Rob Bensinger: *I mostly disagree. It’s possible to define a theory-neutral notion of ‘consciousness’, but I think it’s just true that ‘there’s no such thing as subjective awareness / qualia / etc.’, and I think this cuts real dang deep into the heart of what most people mean by consciousness. Before the name illusionism caught on, I had to use the term ‘eliminativism’, but I had to do a lot of work to clarify that I’m not like old-school eliminativists who think consciousness is obviously or analytically fake. Glad to have a clearer term now. I think people get caught up in knots about the hard problem of consciousness because they try to gesture at ‘the fact that they have subjective awareness’, without realizing they’re gesturing at something that contains massive introspective misrepresentations / illusions. Seeing that we have to throw out a key part of Chalmers’ explanandum *is an important insight for avoiding philosophical knots, even though it doesn’t much help us build a positive account. That account matters, but epistemic spring cleaning matters too. **Eric Rogstad: **
Comment
**Hrothgar: **What’s your answer to the hard problem of consciousness? **Rob Bensinger: **The hard problem makes sense, and seems to successfully do away with ‘consciousness is real and reducible’. But ‘consciousness is real and irreducible’ isn’t tenable: it either implies violations of physics as we know it (interactionism), or implies we can’t know we’re conscious (epiphenomenalism). So we seem to be forced to accept that consciousness (of the sort cited in the hard problem) is somehow illusory. This is… very weird and hard to wrap one’s head around. But some version of this view (illusionism) seems incredibly hard to avoid. (Note: This is a twitter-length statement of my view, so it leaves out a lot of details. E.g., I think panpsychist views must be interactionist or epiphenomenalist, in the sense that matters. But this isn’t trivial to establish.) **Hrothgar: **What does "illusory" mean here? I think I’m interpreting as gesturing toward denying consciousness is happening, which is, like, the one thing that can’t even be doubted (since the experience of doubt requires a conscious experiencer in the first place) **Rob Bensinger: **I think "the fact that I’m having an experience" seems undeniable. E.g., it seems to just be a fact that I’m experiencing this exact color of redness as I look at the chair next to me. There’s a long philosophical tradition of treating experience as ‘directly given’, the foundation on which all our other knowledge is built. I find this super compelling and intuitive at a glance, even if I can’t explain how you’d actually build a brain/computer that has infallible ‘directly given’ knowledge about some of its inner workings. But I think the arguments alluded to above ultimately force us to reject this picture, and endorse the crazy-sounding view ‘the character of my own experiences can be illusory, even though it seems obviously directly given’. An attempt to clarify what this means: https://nothingismere.com/2017/02/23/phenomenal-consciousness-is-a-quasiperceptual-illusion-objections-and-replies/ I don’t want to endorse the obviously false claim ‘light isn’t bouncing off the chair, hitting my eyes, and getting processed as environmental information by my brain.’ My brain is tracking facts about the environment. And it can accurately model many, many things about itself! But I think my brain’s native self-modeling gets two things wrong: (1) it models my subjective experience as a sort of concrete, ‘manifest’ inner world; (2) it represents this world as having properties that are too specific or arbitrary to logically follow from ‘mere physics’. I think there is a genuine perception-like (not ‘hunch-like’) introspective illusion that makes those things appear to be true (to people who are decent introspectors and have thought through the implications) -- even though they’re not true. Like a metacognitive optical illusion. And yes, this sounds totally incoherent from the traditional Descartes-inspired philosophical vantage point. Optical illusions are fine; calling consciousness itself an illusion invites the question ‘what is conscious of this illusion?’. I nonetheless think this weird view is right. I want to say: There’s of course something going on here; and the things that seems present in my visual field must correspond to real things insofar as they have the potential to affect my actions. But my visual field as-it-appears-to-me isn’t a real movie screen playing for an inner Me. And what’s more, the movie screen isn’t translatable into neural firings that encode all the ‘given’-seeming stuff. (!) The movie screen is a lie the brain tells itself—tells itself at the sensory, raw-feel level, not just at the belief/hunch level. (Illusion, rather than delusion.) And (somehow! this isn’t intuitive to me either!) since there’s no homunculus outside the brain to notice all this, there’s no ‘check’ on the brain forcing it to not trick itself in how it represents the most basic features of ‘experience’ to itself. The way the brain models itself is entirely a product of the functioning of that very brain, with no law of physics or CS to guarantee the truth of anything! No matter how counter-intuitive that seems to the brain itself. (And yes, it’s still counter-intuitive to me. I wouldn’t endorse this view if I didn’t think the alternatives were even worse!) Core argument:
Comment
Edit: What it implies is violations of physicalism. You can accept that physics is a map that predicts observations, without accepting that it is the map, to which all other maps must be reduced.
The epiphenomenalist worry is that, if qualia are not denied entirely, they have no causal role to play, since physical causation already accounts for everything that needs to be accounted for.
But physics is a set of theories and descriptions...a map. Usually, the ability of a map to explain and is not exclusive of another map’s ability to do so on. We can explain the death of Mr Smith as the result of bullet entering his heart, or as the result of a finger squeezing a trigger, or a a result of the insurance policy recently taken out on his life, and so on.
So why can’t we resolve the epiphenomenal worry by saying that that physical causation and mental causation are just different, non rivalrous, maps? I screamed because my pain fibres fired" alongside—not versus "I screamed becaue I felt a sharp pain". It is not the case that there is physical stuff that is doing all the causation, and mental stuff that is doing none of it: rather there is a physical view of what is going on, and a mentalistic view.
Physicalists are reluctant to go down this route, because physicalism is based on the idea that there is something special about the physical map, which means it is not just another map. This special quality means that a physical explanation excludes others, unlike a typical map. But what is it?
It’s rooted in reductionism, the idea that every other map (that is, every theory of the special sciences) can or should reduce to the physical map.
But the reducibility of consciousness is the center of the Hard Problem. If consciousness really is irreducible, and not just unreduced, then that is evidence against the reduction of everything to the physical, and, in turn, evidence against the special, exclusive nature of the physical map.
So, without the reducibility of consciousness, the epiphenomenal worry can be resolved by the two-view manoeuvre. (And without denying the very existence of qualia).
Comment
If the physics map doesn’t imply the mind map (because of the zombie argument, the Mary’s room argument, etc.), then how do you come to know about the mind map? The causal process by which you come to know the physics map is easy to understand:
Comment
Direct evidence. That’s the starting point of the whole thing. People think that they have qualia because it seems to them that they do.
Edit: In fact, it’s the other way round: we are always using the mind map, but we remove the subjectivity, "warm fuzzies" from it to arrive at the physics map. Ho wdo we know that physics is the whole story, when we start with our experience, and make a subset of it?
I’m not assuming that. I’m arguing against epiphenomenalism.
So I am saying that the mental is causal, but I am not saying that it is a kind of physical causality, as per reductive physicalism. Reductive physicalism is false because consciousness is irreducible, as you agree. Since mental causation isn’t a kind of physical causation, I don’t have to give a physical account if it.
And I am further not saying that the physical and mental are two separate ontologcal domains, two separate territories. I am talking about maps, not territories.
Without ontological dualism, there are no issues of overdetermination or interaction.
It’s apparently not true that 90% of startups fail. From Ben Kuhn:
Comment
I don’t have a cite handy as it’s memories from 2014 but when I looked into it I recall the 7 year failure rate excluding the obvious dumb stuff like restaurants was something like 70% but importantly the 70% number included acquisitions, so the actual failure rate was something like 60 ish.
Ben Weinstein-Raun wrote on social media:
Comment
Comment
I haven’t read Brian Tomasik’s thoughts on this, so let me know if you think I’m misunderstanding him / should read more. The hard problem of consciousness at least gives us a prima facie reason to consider panpsychism. (Though I think this ultimately falls apart when we consider ‘we couldn’t know about the hard problem of consciousness if non-interactionist panpsychism were true; and interactionist panpsychism would mean new, detectable physics’.) If we deny the hard problem, then I don’t see any reason to give panpsychism any consideration in the first place. We could distinguish two panpsychist views here: ‘trivial’ (doesn’t have any practical implications, just amounts to defining ‘consciousness’ so broadly as to include anything and everything); and ‘nontrivial’ (has practical implications, or at least the potential for such; e.g., perhaps the revelation that panpsychism is true should cause us to treat electrons as moral patients, with their own rights and/or their own welfare). But I see no reason whatsoever to think that electrons are moral patients, or that electrons have any other nontrivial mental property. The mere fact that we don’t fully understand how human brains work is not a reason to ask whether there’s some new undiscovered feature of particles \sim10^{31} times smaller than a human brain that explains the comically larger macro-process—any more than limitations in our understanding of stomachs would be a reason to ask whether individual electrons have some hidden digestive properties.
(Brian Tomasik’s view superficially sounds a lot like what Ben Weinstein-Raun is criticizing in his second paragraph, so I thought I’d add here the comment I wrote in response to Ben’s post:
Comment
Thanks for sharing. :) Yeah, it seems like most people have in mind type-F monism when they refer to panpsychism, since that’s the kind of panpsychism that’s growing in popularity in philosophy in recent years. I agree with Rob’s reasons for rejecting that view.
There’s another theory that isn’t even on Chalmers’s list: dual aspect neutral monism.
This holds that the physical sciences are one possible map of territory which is not itself, intrinsically, physical (or, for that matter, mental). Consciousness is another map, or aspect.
This approach has the advantage of dualism, in that there is no longer a need to explain the mental in terms of the physical, to reduce it to the physical, because the physical is no longer regarded as fundamental (nor is the mental, hence the "neutral"). Although an ontological identity between the physical and mental is accepted, the epistemic irreducibility of the mental to the physical is also accepted. Physicalism, in the sense that the physical sciences have a unique and priveleged explanatory role, is therefore rejected.
Nonetheless, the fact that the physical sciences "work" in many ways, that the physical map can be accurate, is retained. Moreover, since Dual Aspect theory is not fully fledged dualism, it is able to sidestep most or all of the standard objections to dualism.
To take one example, since the a conscious mental state and physical brain state are ultimately the same thing, the expected relationships hold between them. For instance, mental states cannot vary without some change in the physical state (supervenience follows directly from identity, without any special apparatus); furthermore, since mental states are ultimately identical to physical brain states, they share the causal powers of brain states (again without the need to posit special explanatory apparatus such as "psychophysical laws")This holds that the physical sciences are one possible map of territory which is not itself, intrinsically, physical (or, for that matter, mental). Consciousness is another map, or aspect.
This approach has the advantage of dualism, in that there is no longer a need to explain the mental in terms of the physical, to reduce it to the physical, because the physical is no longer regarded as fundamental (nor is the mental, hence the "neutral"). Although an ontological identity between the physical and mental is accepted, the epistemic irreducibility of the mental to the physical is also accepted. Physicalism, in the sense that the physical sciences have a unique and privileged explanatory role, is therefore rejected.
Nonetheless, the fact that the physical sciences "work" in many ways, that the physical map can be accurate, is retained. Moreover, since Dual Aspect theory is not fully fledged dualism, it is able to sidestep most or all of the standard objections to dualism.
To take one example, since the a conscious mental state and physical brain state are ultimately the same thing, the expected relationships hold between them. For instance, mental states cannot vary without some change in the physical state (supervenience follows directly from identity, without any special apparatus); furthermore, since mental states are ultimately identical to physical brain states, they share the causal powers of brain states, and in that way epiphenomenalism is avoided.
The more familiar kinds of dualism are substance and property dualism. Both take a physical ontology "as is" and add something extra, and both have problems with explaining how the additional substances or properties interact with physical substances and properties, and both of course have problems with ontological parsimony (Occam’s Razor).
In contrast to a substance or property, an aspect is a relational kind of thing. In Dual Aspect theory, a conscious state is interpreted as being based on the kind of relationship and entity has with itself, and the kind of interaction it has with itself. The physical is reinterpreted as a kind of interaction with and relation to the external. It is not clear whether this theory adds anything fundamentally new, ontologically, since most people will accept the existence of some kind of inner/outer distinction, although the distinction may be made to do more work in Dual Aspect theory. Reinterpreting the physical is a genuine third alternative to accepting (only) the physical, denying the physical, and suplementing the physical.
The more familiar kinds of dualism are substance and property dualism. Both take a physical ontology "as is" and add something extra, and both have problems with explaining how the additional substances or properties interact with physical substances and properties, and both of course have problems with ontological parsimony (Occam’s Razor).
In contrast to a substance or property, an aspect is a relational kind of thing. In Dual Aspect theory, a conscious state is interpreted as being based on the kind of relationship and entity has with itself, and the kind of interaction it has with itself. The physical is reinterpreted as a kind of interaction with and relation to the external. It is not clear whether this theory adds anything fundamentally new, ontologically, since most people will accept the existence of some kind of inner/outer distinction, although the distinction may be made to do more work in Dual Aspect theory. Reinterpreting the physical is a genuine third alternative to accepting (only) the physical, denying the physical, and suplemmenting the physical.
[Epistemic status: Thinking out loud, just for fun, without having done any scholarship on the topic at all.] It seems like a lot of horror games/movies are converging on things like ‘old people’, ‘diseased-looking people’, ‘psychologically ill people’, ‘women’, ‘children’, ‘dolls’, etc. as particularly scary. Why would that be, from an evolutionary perspective? If horror is about fear, and fear is about protecting the fearful from threats, why would weird / uncanny / out-of-evolutionary-distribution threats have a bigger impact than e.g. ‘lots of human warriors coming to attack you’ or ‘a big predator-looking thing stalking you’, which are closer to the biggest things you’d want to worry about in our environment of evolutionary adaptedness? Why are shambling, decrepit things more of a horror staple than big bulky things with claws? (I mean, both are popular, so maybe this isn’t a real phenomenon. I at least subjectively feel as though those uncanny things are scarier than super-lions or super-snakes.) Maybe we should distinguish between two clusters of fear-ish emotions:
Terror. This is closer to the fight-or-flight response of ‘act quick because you’re in imminent danger’. It’s a panicky ‘go go go go go!!’ type of feeling, like when a jumpscare happens or when you’re running from a monster in a game.
Dread. This is more like feeling freaked out or creeped out, and it can occur alongside terror, or it can occur separately. It seems to be triggered less by ‘imminent danger’ than by ambiguous warning signs of danger. So, a first question is why uncanny, mysterious, ‘unnatural’ phenomena often cause the most dread, even though they thereby become less similar to phenomena that actually posed the largest dangers to us ancestrally. (E.g., big hulking people with giant spears or snakes/dragons or werewolves correlate more with ‘things dangerous to our ancestors’ than decrepit zombies. Sure, creepiness maybe requires that the threat be ‘ambiguous’, but then why isn’t an ambiguous shadow of a maybe-snake or maybe-hulking-monster creepier than an obviously-frail/sickly monster?) Plausibly part of the answer is that more mysterious, inexplicable phenomena are harder to control, and dread is the brain’s way of saying something like ‘this situation looks hard to control in a way that makes me want to avoid situations like this’. Terror-inspiring things like jumpscares have relatively simple triggers corresponding to a relatively simple response—usually fleeing. Dread-inspiring things like ‘the local wildlife has gotten eerily quiet’ have subtler and more context-sensitive triggers corresponding to responses like ‘don’t necessarily rush into any hasty action, but do pay extra close attention to your environment, and if you can do something to get away from the stimuli that are giving you these unpleasant uneasy feelings, maybe prioritize doing that’.
Comment
A second question is why horror films and games seem to be increasingly converging on the creepy/uncanny/mysterious cluster of things, rather than on the overtly physically threatening cluster—assuming this is a real trend. Some hypotheses about the second question:
A: Horror games and movies are increasingly optimizing for dread instead of terror these days, maybe because it’s novel—pure terror feels overdone and out-of-fashion. Or because dread just lends itself to a more fun multi-hour viewing/playing experience, because it’s more of a ‘slow burn’. Or something else.
B: Horror games aren’t optimizing for dread to the exclusion of terror; rather, they’ve discovered that dread is a better way to maximize terror. Why would B be true? One just-so story you could tell is that humans have multiple responses to possible dangers, ranging from ‘do some Machiavellian scheming to undermine a political rival’ to ‘avoid eating that weird-smelling food’ to ‘be cautious near that precipice’ to ‘attack’ to ‘flee’. Different emotions correspond to different priors on ‘what reaction is likeliest to be warranted here?‘, and different movie genres optimize for different sets of emotions. And optimizing for a particular emotion usually involves steering clear of things that prime a person to experience a different emotion—people want a ‘purer’ experience.So one possibility is: big muscular agents, lion-like agents, etc. are likelier to be dangerous (in reality) than a decrepit corpse or a creepy child or a mysterious frail woman; but the correct response to hulking masculine agents is much more mixed between ‘fight / confront’ and ‘run away / avoid’, whereas the correct response to situations that evoke disgust, anxiety, uncertainty, and dread is a lot more skewed toward ‘run away / avoid’. And an excess of jumpscare-ish, heart-pounding terror does tend to incline people more toward running away than toward fighting back, so it might be that both terror and dread are better optimized in tandem, while ‘fight back’ partly competes with terror.On this view, ‘ratchet up the intensity of danger’ matters less for fear intensity than ‘eliminate likely responses to the danger other than being extra-alert or fleeing’.… Maybe because movie/game-makers these days just find it really easy to max out our danger-intensity detectors regardless? Pretty much everything in horror movies is pretty deadly relative to the kind of thing you’d regularly encounter in the ancestral environment, and group sizes in horror contexts tend to be smaller than ancestral group sizes. People who want to enjoy the emotions corresponding purely to the ‘fight’ response might be likelier to watch things like action movies. And indeed, action movies don’t make much use of jumpscares or terror (though they do like tension and adrenaline-pumping intensity). Or perhaps there’s something more general going on, like:
Hypothesis C: Dread increases ‘general arousal / sensitivity to environmental stimuli’, and then terror can piggy-back off of that and get bigger scares. Perhaps emotions like ‘disgust’ and ‘uncertainty’ also have this property, hence why horror movies often combine dread, disgust, and uncertainty with conventional terror. In contrast, hypothesis B seems to suggest that we should expect disgust and terror to mostly show up in disjoint sets of movies/games, because the correct response to ‘disease-ish things’ and the correct response to ‘physical attackers’ is very different.
The wiki glossary for the sequences / Rationality: A-Z ( https://wiki.lesswrong.com/wiki/RAZ_Glossary ) is updated now with the glossary entries from the print edition of vol. 1-2. New entries from Map and Territory:
Comment
This reminds me of something I’ve been meaning to ask:
Last I checked, the contents of the Less Wrong Wiki were licensed under the GNU Free Documentation License, which is… rather inconvenient. Is it at all possible to re-license it (ideally as CC BY-NC-SA, to match R:AZ itself)?
(My interest in this comes from the fact that the Glossary is mirrored on ReadTheSequences.com, and I’d prefer not to have to deal with two different licenses, as I currently have to.)
Comment
I can reach out to Trike Apps about this, but can we actually do this? Seems plausible that we would have to ask for permission from all editors involved in a page before we can change the license.
Comment
I have no idea; I cannot claim to really understand the GFDL well enough to know… but if doable, this seems worthwhile, as there’s a lot of material on the wiki which I and others could do various useful/interesting things with, if it were released under a convenient license.
Are there any other OK-quality rationalist glossaries out there? https://wiki.lesswrong.com/wiki/Jargon is the only one I know of. I vaguely recall there being one on http://www.bayrationality.com/ at some point, but I might be misremembering.
Comment
https://namespace.obormot.net/Jargon/Jargon
Comment
Fantastic!
It’s optimized on a very different axis, but there’s the Rationality Cardinality card database.
Comment
That counts! :) Part of why I’m asking is in case we want to build a proper LW glossary, and Rationality Cardinality could at least provide ideas for terms we might be missing.
Jeffrey Ladish asked on Twitter:
Comment
I suspect there’s a school of thought for which "singularity" was massively overoptimistic—is this what you mean by Kurzweilian magical thinking? That it’s a transition in a very short period of time from scarcity-based capitalism to post-scarcity utopia. Rather than a simple destruction of most of humanity, and of the freedom and value of those remaining.
Comment
Rejection of Eliezer’s five theses (which were written in response to Kurzweil): intelligence explosion, orthogonality, convergent instrumental goals, complexity of value, fragility of value.
Mystical, quasi-Hegelian thinking about surface trends like ‘economic growth’. See the ‘Actual Ray Kurzweil’ quote in https://www.lesswrong.com/posts/ax695frGJEzGxFBK4/biology-inspired-agi-timelines-the-trick-that-never-works.
Otherwise weird and un-Bayesian-sounding attitudes toward forecasting. Seems to think he has a crystal ball that lets him exactly time tech developments, even where he has no model of a causal path by which he could be entangled with evidence about that future development...?
From Facebook: Mark Norris Lance: [...] There is a long history of differential evaluation of actions taken by grassroots groups and similar actions taken by elites or those in power. This is evident when we discuss violence. If a low-power group places someone under their control it is kidnapping. If they assess their crimes or punish them for it, it is mob justice or vigilanteism. [...]John Maxwell: Does the low power group in question have a democratic process for appointing judges who then issue arrest warrants? That’s a key issue for me… "Mob rule" is bad because the process mobs use to make their judgements are bad. Doubly so if the mob attacks anyone who points that out. A common crime that modern mobs accuse people of is defending bad people. But if people can be convicted of defending bad people, that corrupts the entire justice process, because the only way we can figure out if someone really is bad is by hearing what can be said in their defense.
From https://twitter.com/JonHaidt/status/1166318786959609856:
Yeah, I’m an EA: an Estimated-as-Effective-in-Expectation (in Excess of Endeavors with Equivalent Ends I’ve Evaluated) Agent with an Audaciously Altruistic Agenda.
Comment
This is being cute, but I do think parsing ‘effective altruist’ this way makes a bit more sense than tacking on the word ‘aspiring’ and saying ‘aspiring EA’. (Unless you actually are a non-EA who’s aspiring to become one.) I’m not an ‘aspiring effective altruist’. It’s not that I’m hoping to effectively optimize altruistic goals someday. It’s that I’m already trying to do that, but I’m uncertain about whether I’m succeeding. It’s an ongoing bet, not an aspiration to do something in the future. ‘Aspiring rationalist’ is better, but it feels at least a little bit artificial or faux-modest to me—I’m not aspiring to be a rationalist, I’m aspiring to be rational. I feel like rationalism is weight-training, and rationality is the goal. If people are unhealthy, we might use ‘health-ism’ to refer to a community or a practice for improving health. If everyone is already healthy, it seems fine to say they’re healthy but weird to say ‘they’re healthists’. Why is it an ism? Isn’t it just a fact about their physiology?
How would you feel about the creation of a Sequence of Shortform Feeds? (Including this one?) (Not a mod.)
Comment
Sure
I can’t speak for Rob but I’d be fine with my own shortform feed being included.