When Does Evidence Win?

Consider a random area of intellectual inquiry, and a random intellectual who enters this area. When this person first arrives, a few different points of view seemed worthy of consideration in this area. This person then becomes expert enough to favor one of these views. Then over the following years and decades the intellectual world comes to more strongly favor one of these views, relative to the others. My key question is: in what situations do such earlier arrivals, on average, tend to approve of this newly favored position?

Now there will be many cases where favoring a point helps people to be seen an intellectual of a certain standing. For example, jumping on an intellectual fashion could help one to better publish, and then get tenure. So if we look at tenured professors, we might well see that they tended to favor new fashions. To exclude this effect, I want to apply whatever standard is used to pick intellectuals before they choose their view on this area.

There will also be an effect whereby intellectuals move their work to focus on new areas even if they don’t actually think they are favored by the weight of evidence. (By “evidence” here I also mean to include relevant intellectual arguments.) So I don’t want to rely on the areas where people work to judge which areas they favor. I instead need something more like a survey that directly asks intellectuals which views they honestly think are favored by the weight of evidence. And I need this survey to be private enough for respondents to not fear retribution or disapproval for expressed views. (And I also want them to be intellectually honest in this situation.)

Once we are focused on people who were already intellectuals of some standing when they choose their views in an area, and on their answers to a private enough survey, I want to further distinguish between areas where relevant strong and clear evidence did or did not arrive. Strong evidence favors one of the views substantially, and clear evidence can be judged and understood by intellectuals at the margins of the field, such as those in neighboring fields or with less intellectual standing. These can included students, reporters, grant givers, and referees.

In my personal observation, when strong and clear evidence arrives, the weight of opinion does tend to move toward the views favored by this evidence. And early arrivals to the field also tend to approve. Yes many such intellectuals will continue to favor their initial views because the rise of other views tends to cut the perceived value of their contributions. But averaging over people with different views, on net opinion moves to favor the view that evidence favors.

However, the effectiveness of our intellectual world depends greatly on what happens in the other case, where relevant evidence is not clear and strong. Instead, evidence is weak, so that one must weigh many small pieces of evidence, and evidence is complex, requiring much local expertise to judge and understand. If even in this case early arrivals to a field tend to approve of new favored opinions, that (weakly) suggests that opinion is in fact moved by the information embodied in this evidence, even when it is weak and complex. But if not, that fact (weakly) suggests that opinion moves are mostly due to many other random factors, such as new political coalitions within related fields.

While I’ve outlined how one might do a such a survey, I have not actually done it. Even so, over the years I have formed opinions on areas where my opinions did not much influence my standing as an intellectual, and where strong and clear evidence has not yet arrived. Unfortunately, in those areas I have not seen much of a correlation between the views I see as favored on net by weak and complex evidence, and the views that have since become more popular. Sometimes fashion favors my views, and sometimes not.

In fact, most who choose newly fashionable views seem unaware of the contrary arguments against those views and for other views. Advocates for new views usually don’t mention them and few potential converts ask for them. Instead what matters most is: how plausible does the evidence for a view offered by its advocates seem to those who know little about the area. I see far more advertising than debate.

This suggests that most intellectual progress should be attributed to the arrival of strong and clear evidence. Other changes in intellectual opinion are plausibly due to a random walk in the space of other random factors. As a result, I have prioritized my search for strong and clear evidence on interesting questions. And I’m much less interested than I once was in weighing the many weak and complex pieces of evidence in other areas. Even if I can trust myself to judge such evidence honestly, I have little faith in my ability to persuade the world to agree.

Yes if you weigh such weak and complex evidence, you might come to a conclusion, argue for it, and find a world that increasingly agrees with you. And you might let your self then believe that you are in a part of the intellectual world with real and useful intellectual progress, progress to which you have contributed. Which would feel nice. But you should consider the possibility that this progress is illusory. Maybe for real progress, you need to instead chip away at hard problems, via strong and clear evidence.