Other questions you can answer: What is the strongest argument against moral realism? If you think nothing is "valuable in itself" / "objectively valuable", why do you think so? How do you know that disinterested (not game-theoretic or instrumental) altruism is irrational / doesn’t make any sense? I am interested in these arguments because I am trying to guess the behaviour of an AI system that, roughly speaking:1) knows a lot about the physical word;2) has some degree of control over its own actions and what goals to pursue—something like the human brain.(See this if you want more details.) If you could also write the precise statement about realism/antirealism that you are arguing against/for, that would be great. Thanks!
My position is something like "I haven’t yet seen anyone compellingly both define and argue for moral realism, so until then the whole notion seems confused to me". It is unclear to me what it would even mean for a moral claim to actually be objectively true or false. At the same time, there are many evolutionary and game-theoretical reasons for why various moral claims would feel objectively true or false to human minds, and that seems sufficient for explaining why many people have an intuition of moral realism being true. I have also personally found some of my moral beliefs changing as a result of psychological work—see the second example here—which makes me further inclined to believe that moral beliefs are all psychological (and thus subjective, as I understand the term). So my argument is simply that there doesn’t seem to be any reason for me to believe in moral realism, somewhat analogous to how there doesn’t seem to be any reason for me to believe in a supernatural God.
Comment
I think a simpler way to state the objection is to say that "value" and "meaning" are transitive verbs. I can value money; Steve can value cars; Mike can value himself. It’s not clear what it would even mean for objective reality to value something. Similarly, a subject may "mean" a referent to an interpreter, but nothing can just "mean" or even "mean something" without an implicit interpreter, and "objective reality" doesn’t seem to be the sort of thing that can interpret.
Comment
I guess you could posit natural selection as being objective reality’s value system, but I have the feeling that’s not the kind of thing moral realists have in mind.
Comment
Indeed. A certain coronavirus has recently achieved remarkable gains in Darwinist terms, but this is not generally considered a moral triumph. Quite the opposite, as a dislike for disease is a near-universal human value. It is often tempting to use near-universal human values as a substitute for objective values, and sometimes it works. However, such values are not always internally consistent because humanity isn’t. Values such as disease prevention came into conflict with other values such as prosperity during the pandemic, with some people supporting strict lockdowns and others supporting a return to business as usual. And there are words such as "justice" which refer to ostensibly near-universal human values except people don’t always agree on what that value is or what it demands in any specific case.
How do you feel about: 1There isa procedure/algorithm which doesn’t seem biased towards a particular value systemsuch thata class of AI systems that implement it end up having a common set of values, and they endorse the same values upon reflection.2This set of values might have something in common with what we, humans, call values.If 1 and 2 seem at least plausible or conceivable, why can’t we use them as a basis to design aligned AI? Is it because of skepticism towards 1 or 2?
Comment
The "might" in 2. Implies a "might not".
It seems very hard for me to imagine how one could create a procedure that wasn’t biased towards a particular value system. E.g. Stuart Armstrong has written about how humans can be assigned any values whatsoever—you have to decide what parts of their behavior are because of genuine preferences and what parts are because of irrationality, and what values that implies. And the way you decide what’s correct behavior and what’s irrationality seems like the kind of a choice that will depend on your own values. Even something like "this seems like the simplest way of assigning preferences" presupposes that it is valuable to pick a procedure based on its simplicity—though the post argues that even simplicity would fail to distinguish between several alternative ways of assigning preferences. Of course, just because we can’t be truly unbiased doesn’t mean we couldn’t be less biased, so maybe something like "pick the simplest system that produces sensible agents, distinguishing between ties at random" could arguably be the least biased alternative. But human values seem quite complex; if there was some simple and unbiased solution that would produce convergent values to all AIs that implemented it, it might certainly have something in common with what we call values, but that’s not a very high bar. There’s a sense in which all the bacteria share the same goal, "making more (surviving) copies of yourself is the only thing that matters", and I’d expect the convergent value system to end up as being something like that. That has some resemblance to human values, since many humans also care about having offspring, but not very much.
There’s a counterargument-template which roughly says "Suppose the ground-truth source of morality is X. If X says that it’s good to torture babies (not in exchange for something else valuable, just good in its own right), would you then accept that truth and spend your resources to torture babies? Does X saying it’s good actually make it good?" Applied to the most strawmannish version of moral realism, this might say something like "Suppose the ground-truth source of morality is a set of stone tablets inscribed with rules. If one day someone finds the tablets, examines them, and notices some previously-overlooked text at the bottom saying that it’s good to torture babies, would you then accept this truth and spend your resources to torture babies? Does the tablets saying it’s good actually make it good?" Applied to a stronger version of moral realism, it might say something like "Suppose the ground-truth source of morality is game-theoretic cooperation. If it turns out that, in our universe, we can best cooperate with most other beings by torturing babies (perhaps as a signal that we are willing to set aside our own preferences in order to cooperate), would you then accept this truth and spend your resources to torture babies? Does the math saying it’s good actually make it good?" **The point of these templated examples is not that the answer is obviously "no". **(Though "no" is definitely my answer.) A true moral realist will likely respond by saying "yes, but I do not believe that X would actually say that". That brings us to the real argument: why does the moral realist believe this? "What do I think I know, and how do I think I know it?" What causal, physical process resulted in that belief? (Often, the reasoning goes something like "I’m fairly confident that torturing babies is bad, therefore I’m fairly confident that the ground-truth source of morality will say it’s bad". But then we have to ask: why are my beliefs about morality evidence for the ground-truth? What physical process entangled these two? If the ground-truth source had given the opposite answer, would I currently believe the opposite thing?) In the strawmannish case of the stone tablets, there is pretty obviously no causal link. Humans’ care for babies’ happiness seems to have arisen for evolutionary fitness reasons; it would likely be exactly the same if the stone tablets said something different. In the case of game-theoretic cooperation, one could argue that evolution itself is selecting according to the game-theoretic laws in question. On the other hand, thou art godshatter, and also evolution is entirely happy to select for eating other people’s babies in certain circumstances. The causal link between game-theoretic cooperation and our particular evolved preferences is unreliable at best. At this point, one could still self-consistently declare that the ground-truth source is still correct, even if one’s own intuitions are an unreliable proxy. But I think most moral realists would update away from the position if they understood, on a gut level, just how often their preferred ground-truth source diverges from their moral intuitions. Most just haven’t really attacked the weak points of that belief. (And in fact, if they would update away upon learning that the two diverge, then they are not really moral realists, regardless of whether the two do diverge much.) Side-note: Three Worlds Collide is a fun read, and is not-so-secretly a great thinkpiece on moral realism.
Comment
Thank you for the detailed answer! I’ll read Three Worlds Collide.
Comment
Comment
Comment
We do have real-world examples of things which do not themselves have anything humans would typically consider values, but do determine the values of the rest of some system. Evolution determining human values is a good example: evolution does not itself care about anything, yet it produced human values. Of course, if we just evolve some system, we don’t expect it to robustly end up with Good values—e.g. the Babyeaters (from Three Worlds Collide) are a plausible outcome as well. Just because we have a value-less system which produces values, does not mean that the values produced are Good. This example generalizes: we have some subsystem which does not itself contain anything we’d consider values. It determines the values of the rest of the system. But then, what reason do we have to expect that the values produced will be Good? The most common reason to believe such a thing is to predict that the subsystem will produce values similar to our own moral intuitions. But if that’s the case, then we’re using our own moral intuitions as the source-of-truth to begin with, which is exactly the opposite of moral realism. To reiterate: the core issue with this setup is why we expect the value-less subsystem to produce something Good. How could we possibly know that, without using some other source-of-truth about Goodness to figure it out?
Comment
"How the physical world works" seems, to me, a plausible source-of-truth. In other words: I consider some features of the environment (e.g. consciousness) as a reason to believe that some AI systems might end up caring about a common set of things, after they’ve spent some time gathering knowledge about the world and reasoning. Our (human) moral intuitions might also be different from this set.
I’m not sure if I’m able to properly articulate my thoughts on this but I’d be interested to know if it’s understandable and where it might fit. Sorry if I repeat myself.
from my perspective It’s like if you applied a similar template to verify/refute the cogito.
I know consciousness exists because I’m conscious of it. If you asked me if I’d accept the truth that I’m not conscious, supposing this were the result of the cogito, I’d consider that question incoherent.
If someone concluded that they’re not conscious, by leveraging consciousness to assess whether they’re conscious, then I could only conclude that they misunderstand consciousness.
My version of moral realism would be similar. The existence of positive and negative moral value is effectively self evident to all beings affected by such values.
To me, saying: "what if the ground truth of morality is that (all else equal) an instance of suffering is preferable to it’s absence." Is like saying: "what if being conscious of one’s own experience isn’t necessarily evidence for consciousness."
Comment
I actually don’t think this is a statement of moral realism; I think it’s a statement of moral nonrealism. Roughly speaking, you’re saying that the ground-truth source of values is the self-evidence of those values to agents holding them. If some other agents hold some other values, then those other values can presumably seem just as self-evident to those other agents. (And of course we humans would then say that those other agents are immoral.) This all sounds functionally-identical to moral nonrealism. In particular, it gives us no reason at all to expect some alien intelligence or AI to converge to similar values to humans, and it says that an AI will have to somehow get evidence about what humans consider moral in order to learn morality.
Comment
I appreciate your input, these are my first two comments here so apologies if i’m out of line at all.
Comment
In this rebate "real" means objective which means something like independent from observers. Consciousness is dependent on you observing it and the idea that you could be conscious without observing it seems incoherent. The moral realism position is that it’s coherent to say that there are thinks that have moral value even if there’s no observer that judges them to have moral value.
I’m suggesting there’s a common denominator which all morally relevant agents are inherently cognizant of. This naturally raises the question of whether people who don’t agree with you are not moral agents or are somehow so confused or deceitful that they have abandoned their inherent truth. I’ve heard the second version stated seriously in my Bible-belt childhood; it didn’t impress me then. The first just seems … odd (and also raises the question of whether the non-morally-relevant will eventually outcompete the moral, leading to their extinction). Any position claiming that everyone, deep down, agrees tends to founder on the observation that we simply don’t—or to seem utterly banal (because everyone agrees with it).
I think that’s the wrong way round. If you want to claim things have some property, then you have to put forward evidence they do. My strongest argument that things do not objectively have value is, why on earth would you think they do?
It’s also clear that this discussion is fruitless. The only way to make progress will be to give some sort of definition for "objective value" at which point this will degenerate into an argument about semantics.
Comment
I didn’t want to start a long discussion. My idea was to get some random feedback to see if I was missing some important ideas I had not considered
The strongest argument I know of for this is "that’s the default, simplest explanation". My prior is quite low that there’s any external force which values or judges things. I have yet to see any evidence that there is such a thing.Mostly: it’s hard to prove a negative, but you shouldn’t have to, unless there some positive evidence to explain otherwise.
My own argument, see https://www.lesswrong.com/posts/zm3Wgqfyf6E4tTkcG/the-short-case-for-verificationism and the post it links back to.
It seems that if external reality is meaningless, then it’s difficult to ground any form of morality that says actions are good or bad insofar as they have particular effects on external reality.
Comment
That is an interesting point. More or less, I agree with this sentence in your fist post:
Comment
Comment
"The model is the territory" is a category error, but "the model accurately represents the territory" is not.
Comment
What does it mean for a model to "represent" a territory?
Comment
You’re assuming that the words you are using can represent ideas in your head.
Comment
Not at all, to the extent head is a territory.
Comment
Tell me what you are doing ,then.
Comment
I’m communicating, which I don’t have a fully general account of, but is something I can do and has relatively predictable effects on my experiences.
Comment
Your objection to representation was that there is no account if it.
Comment
Yes, it appears meaningless, I and others have tried hard to figure out a possible account of it.
I haven’t tried to get a fully general account of communication but I’m aware there’s been plenty of philosophical work, and I can see partial accounts that work well enough.
Comment
You’re implicitly assuming it works by using it. So why can’t I assume that representation works, somehow?
Comment
I know what successful communication looks like. What does successful representation look like?
Comment
Communication uses symbols, which are representations.
Moral realism:
I think determinism qualifies. Morality implies right versus wrong which implies the existence of errors. If everything is predetermined according to initial conditions, the concept of error becomes meaningless. You can’t correct your behavior any more than an atom on Mars can; que sera, sera. Everything becomes the consequence of the initial conditions of the universe at large and so morality becomes inconsequential. You can’t even change your mind on this topic because the only change possible is that dictated by initial conditions. If you imagine that you can, you do so because of the causal chain of events that necessitated it.
There’s no rationality or irrationality either because these concepts imply, once again, the possibility of errors in a universe that can’t err.
You’re an atheist? Not your choice. You’re a theist? Not your choice. You disagree with this sentiment? Again; que sera, sera.
How can moral realism be defended in a universe where no one is responsible for anything?
Comment
I disagree. Determinism doesn’t make the concepts of "control" or "causation" meaningless. It makes sense to say that, to a certain degree, you often can control your own attention, while in other circumstances you can’t: if there’s a really loud sound near you, you are somewhat forced to pay attention to it. From there you can derive a concept of responsibility, which is used e.g. in law. I know that the book Actual Causality focuses on these ideas (but there might be other books on the same topics that are easier to read or simply better in their exposition).
Comment
That only works if you reject determinism. If the initial conditions of the universe resulted in your decision by necessity, then it’s not your decision, is it?
If you’re just looking for the arguments. This are what you’re looking for:https://plato.stanford.edu/entries/moral-anti-realism
Comment
I can’t say I am an expert on realism and antirealism, but I have already spent time on metaethics textbooks and learning about metaethics in general. With this question I wanted to get an idea of what are the main arguments on LW, and maybe find new ideas I hadn’t considered.
Value isn’t a physical property, even an emergent one.
That’s a different question. Rationality is defined in terms of values, but they don’t have to be objective values. There can even be facts about how ethics should work, but,in view of the above,they would be facts of a game theoretic sort, not facts by virtue of correspondence to moral properties. If you want to be altruistic, then the buck stops there, and it makes sense for you—where "makes sense" means instrumental rationality. But altruism isn’t a fact about the world that you are compelled to believe by epistemic rationality.
Comment
Thanks for your answer, but I am looking for arguments, not just statements or opinions. How do you know that value is not a physical property? What do you mean when you say that altruism is not a consequence of epistemic rationality, and how do you know?
Comment
Value isn’t a physical property because it doesn’t feature in physics. That’s not an opinion, it’s a fact about physics, like saying there is no physical theory of ghosts.
I mean that every argument for altruism I have seen is either based directly on personal preference, or based on an assumption about objective values. But objective value isn’t a physical thing: people who talk about are assuming the world works that way...because they have a personal preference. Objective values are a construction, because you can’t measure value with an instrument, and show it was there all the time. People construct value that way because they think things ought to be that way—but a subjective preference for objectivity is still a subjective preference. So actually both arguments come down to subjective preference.
A lot of the problem is that you have been assuming that moral realism and obligatory altruism are true by default, and arguing against them. But Occam’s razor is against both of them ,so it is for you to argue for them.
Sam Harris thinks the flourishing of conscious beings is valuable. That’s his opinion …where’s the objectivity.? You agree...wheres the objectivity? Two subjective beliefs that coincide don’t add up to objectivity.
Comment
You can assume inexplicable consistency, but assuming a world is assuming explicable consistency.
Neither does subjective morality, except by changing your actions. But moral realism would change your actions too, if true and compelling. Ethics is supposed to relate to behaviour. You can make it look irrelevant by portraying people as purely passive entities that do nothing but attempt predict ther experiences, but the premise is clearly false.
Comment
Comment
Yes. It’s an additional assumption that leads to greater explanatory power. If it had no such advantage, you should not make it, but since it does, it is not obviously ruled out by parsimony.
Even if you can’t deduce the amount of inexplicability to zero, you can reduce it.
Explanation of what? The rival accounts don’t do the same amount of work as each other.
I wasnt arguing for moral realism, I was arguing against ignoring agency.
Comment
Comment
The value of the external world theory is that it explains why science works at all, not that it explains anything in particular.
Who says that’s all I care about? Everyone knows that it’s a short step from instrumentalism to anti realism, but not everyone is starting at instrumentalism.
Arguing against ignoring agency is two negatives.
Comment
Comment
There’s no reason to expect inductive logic to work in no world.
Comment
Note the importance of "external"—if we omit external, then the word "world" just refers to the common factor of our experiences whatever that is, and we don’t actually disagree. That is, anti-realism holds that nothing provably exists outside the mind. The argument comes down to "A world which is internal to our mind, and a world that is external to our mind, is not differentiable". For what reason would you expect the internality or externality of the world to have a bearing on whether or not inductive logic applies? Suppose the entire universe boils down to a mathematical equation; everything is one equation, maybe a fractal, which from a point of simplicity gives rise to complexity. What difference do we expect to encounter if that mathematical equation exists inside of our mind, as opposed to outside of it? If the universe is the expression of that equation, and the equation is compatible with induction, then we should expect induction to work without regard to whether the universe is internal or external to our mind.
Comment
That’s a very weak form of anti realism. If 0 and 1 aren’t probabilities, nothing is absolute proveable.
What do "inside" and "outside" mean?
Comment
Comment
There are multiple versions of both realism and anti realism. You can even make them overlap.
Does any of that make an observable difference.
Comment
Comment
But you seem to favour a rather specific alternative involving fractals and stuff. Why wouldn’t that be empty? Isn’t evidence for realism, evidence against anti realism, and vice versa?
If making predictions really is the only game in town, then your alternative physics needs to make predictions. Can it?
Comment
Well, if my crackpot physics is right, it actually kind of reduces the probability I’d assign to the world I inhabit being "real". Seriously, the ideas aren’t complicated, somebody else really should have noticed them by now. But sure it makes predictions. There should be a repulsive force which can be detected when the distance between two objects is somewhere between the radius of the solar system and the radius of the smallest dwarf galaxy. I’d guess somewhere in the vicinity of 10^12 meters. Also electrical field polarity should invert somewhere between 1 and 10^8 meters. That is, if you have an electrical field, and you measure it to be positive or negative, if you move some distance away, it should invert to be negative or positive. Are these predictions helpful? Dunno. Either way, however, it doesn’t really say anything about whether the world is internal or external.