An interesting paradox arise if one tries to apply Aumann’s theorem in a world of widespread disagreement. The problem is especially apparent in considering the stronger principle which Eliezer calls the Modesty Argument, which claims not only that rational, mutually-respecting people should not agree to disagree, but that in general one should be agreeable and allow oneself to be easily persuaded in any argument.
I’ll introduce it with one of my favorite jokes:
This is a story about advice, and any story about advice becomes a story about a village rabbi. Two men came to see the rabbi of their village.
The first one said, "Rabbi, I have a pear tree in my yard. My father planted it, I keep it watered and don’t let the chickens peck on its roots. One of its branches hangs over my neighbor’s wall. So what do I see yesterday but my neighbor standing there eating one of my pears. This is theft, and I want him to pay me damages."
The rabbi nodded his head. "You’re right, you’re right."
The other neighbor said, "Rabbi, you know that I have seven children–without my garden to feed them, how would I manage? But that tree casts a shadow where nothing will grow. So yesterday, when I go out to dig some potatoes, a pear from his tree falls and hits me right on the head. How am I hurting my neighbor if I eat it? And doesn’t he owe me something for his tree’s blocking sunlight?
The rabbi thought and then said, "You’re right, you’re right."
Meanwhile, the rabbi’s wife had been hearing all this. "How can you say, ‘You’re right’ to both of these men? Surely one of the men is right, and the other is wrong!"
The rabbi looked unhappy. "You’re right, you’re right."
Suppose that I am somewhat neutral on a given issue and I am going to argue with one of two partisans on the issue, who each have opposing views. Applying the Modesty Argument it might seem that I should be prepared to be persuaded to the view of whichever one of them I interact with. It is true, in talking to one of them, that we both know that other people exist with opposing views, but since this person has not been persuaded by this knowledge, he must have good reason to hold to his particular side. There is no reason a priori to suppose that my beliefs on this are any better founded than his, especially since we stipulated that it was a position on which I was relatively ignorant. So it is plausible, given that we reach agreement, and given his long-standing commitment to his position, that our resulting agreement will generally be favorable to his side of the argument.
The problem is that I can equally well reason that if I were to interact instead with the person who takes the opposite view, I would be just as convinced to take that other side. This evokes a rather comical image of sequential interactions with the two partisans, in which I am first convinced of one side and then the other. It seems that someone following the Modesty Argument too literally will find himself in the position of the poor rabbi in the joke.
The problem and paradox is that this behavior is irrational. You can’t be in a position where you have an expectation that you can gain some information that will move your opinion in a predictable direction. Just knowing this fact about the situation should be enough to move your opinion already. Robin has what I think is a formal proof of this in his paper, Disagreement is Unpredictable (not sure I understood it correctly though). For a rational person, every new piece of information is a surprise in the sense that he can’t predict which way it will change his opinion, on the average. When I post these messages here I have no idea whether I will be more or less convinced of my opinion as a result of the feedback I receive. That is how it always must be.
Therefore I can’t be in a position where I know if I talk to person A I will be convinced that he is right, and if I talk to person B I will be convinced that A is wrong. A practitioner of the Modesty Argument who finds himself in this situation is doing something wrong.
So there are two questions: what is wrong with the argument above that puts us in the shoes of the rabbi; and what in fact should a rational and modest person do in a world full of disagreement?
First, I seem to have gone wrong when I concluded that if I argued with person A, I would come to agree with him. As I just pointed out, it is irrational to have an expectation that gaining information will predictably change one’s opinion. Instead, it must be the case that I am completely uncertain about how my opinion will change if I talk with A. Given that we will agree, it must be my expectation that A is as likely to change his mind as that he will convince me. Now, I argued above that this is unlikely given A’s long-standing position as a partisan in the dispute. Therefore I have to conclude that A does not meet the conditions for Aumann’s result; he is not rational, or honest, or at least he assumes that those he disagrees with lack these traits. In that case I cannot apply Aumann’s theorem and cannot act in accordance with the Modesty Argument in my discussions with A or B.
It’s interesting to consider what would happen if A and B are in fact both rational and honest, but both distrust whether their counterpart shares these properties. This is a stable configuration, and it’s rational for A and B to disagree in this situation; but of course they lack the mutual respect which is generally associated with saying that they agree to disagree. Now suppose A and B both trust me as an "honest broker", and I trust them. Then I can’t disagree with A, and I can’t disagree with B, once I interact with them. What will happen is that when I interact with A, we two will agree; and when I interact with B, we two will agree. In this way, A and B will come to agreement by both agreeing with me (I might have to repeat the interaction a few times). This again illustrates how odd is the persistence of disagreement in the world, among people who claim to be rational and honest.
But realistically, disagreement does persist, which leaves the question of how a rational person should behave. Unfortunately, I don’t think that Aumann’s theorem or the Modesty Argument offers much guidance. It’s not rational to adopt a stance where you are easily persuaded by everyone you talk to. There’s no particular reason, just because you are talking to a proponent of one side, to believe him, given that you know there are plenty of people around who believe the opposite. It’s not that you think you are better (smarter, more knowledgeable, more rational) than he is; it’s just that you assume there are many equally smart, knowledgeable and rational people on the other side. In the end, you have to make your decision on controversial issues using other grounds than the irrationality of disagreement.
Clearly the problem here is that A does not like B and B does not like A because they are competing for resources. Rationale doesn't play a part in the Rabbi's answers, as he doesn't even stop to think fully and properly before saying 'you're right, you're right', even to his wife. He just takes the easy way out and agrees with them all.
Firstly, not only are they both wrong as a socialised or communist doctrine may hold that "All property is theft" but the wife is also wrong, because both men can be right, and wrong, at the same time.
There's no case to answer here. The Rabbi is only capable of agreeing, and does not have the linguistic tools to articulate the reality of the situation.
Sorry Hal. R.I.P. btw: Thank You for RPOWs!! :)
pdf23ds - It may seem reasonable to think that way, but really it's not. Consider a simpler case where there is someone so smart that whenever you interact with him, he can convince you of anything. In fact he can convince you that A is true and then turn around and convince you that A is false. Then it might seem that you can reasonably have an expectation that after you interact with him, your opinion on a subject will change in favor of whatever position you know in advance that he will advocate.
But eventually you should realize that since he has this ability, even though he provides a convincing argument in favor of his position, you know that he could provide an equally convincing case for the other side. This knowledge should discredit his argumentation and cause you to reject his position, no matter how persuasive it seems to be on the surface. It's not reasonable to be convinced of something when you know that there is just as compelling an argument against it, even though you don't know what that argument is.
To use the possible-world formulation, if you know that there is information that will cause you to have belief X on a matter, then you know that you exist in a set of possible worlds where belief X is reasonable, and therefore you should have belief X now.