How To Fund Prestige Science

How can we best promote scientific research? (I’ll use “science” broadly in this post.) In the usual formulation of the problem, we have money and status that we could distribute, and they have time and ability that they might apply. They know more than we do, but we aren’t sure who is how good, and they may care more about money and status than about achieving useful research. So we can’t just give things to anyone who claims they would use it to do useful science. What can we do? We actually have many options.

A relatively easy case is science that might be useful for improving a product or service in the near future. In this case providers of such goods or services can have good incentives to find the best people and to give them good incentives to help improve their offerings. At least this can work well if researchers can keep their improvements secret within their organizations for a long enough duration.

A related case is where we can generally distinguish discrete inventions (e.g., creations, insights, or techniques) that have commercial value, and where we can identify which small group is responsible for creating each one. In this case, we can set up a system of intellectual property, wherein we give each inventor a property right to control the use of their invention. This can create incentives to develop commercially-valuable inventions that are hard to keep secret for long, and this works even when providers are bad at identifying who to hire how to create such improvements. This does, however, impose substantial transaction and enforcement costs.

Another relatively easy case is where we have something particular that we want accomplished, like figuring out how to find a ship’s longitude at sea. At least this is easy when only one small group is responsible for each accomplishment, we can later verify both that the accomplishment happened and identify who caused it. In this case we can offer a prize to whomever causes this accomplishment. Prizes were actually a far more common method of science funding centuries ago, during and soon after the scientific revolution.

Another very common method from centuries ago was to subsidize complements to scientific research, complements not very useful for other purposes. Long ago science patrons would subsidize scientific journals, libraries, meeting places, equipment, etc. Similarly, today governments often offer research tax credits. This method requires that someone else also fund science via better targeted incentives. For example, long ago many aristocratic scientists funded their own research.

Sometimes what we want are not particular accomplishments, but accurate estimates on particular questions, and increases in that accuracy over time. For example, we might want to know if most dark matter is made of axions, or if adopting the death penalty reduces the crime rate. We want the best current estimate on such things, and then want to learn more over time. For questions in areas where there’s a decent chance that we eventually know and agree on the answers, subsidized prediction markets can both induce accurate-as-possible current estimates, and robust incentives to increase that accuracy over time, even when credit for such increases should be spread widely over thousands of contributors. It is enough that each trader is reward for moving the price toward its final more-accurate position. Combinatorial versions can allow added info on any one question to propagate efficiently to update all the other questions. These methods haven’t been used much so far, but their potential remains large.

For many topics and areas of science, however, the above methods seem awkward and difficult to apply. If these methods were all we had, then of course we’d make do as best we could. But in fact we have another general way that we use to promote science, one that actually gets disproportionate status and attention. Students, media interviewers, and grant funders all crave public affiliation with prestigious scientists, and in effect pay scientists and their institutions for such affiliation. Scientists and the larger world collectively create prestige rankings of papers, people and institutions, including schools, journals, media, and patrons. In this context, individual scientists try to do impressive things, including scientific research, to gain prestige. They also play politics, in order to slant the prestige consensus in their favor.

This prestige system has many problems. For example, mutual admiration societies often form to reinforce each others’s prestige votes on each other, even when these groups don’t actually seem very impressive to outsiders. Even so, this system has long been big and influential, and it can claim substantial credit for many of our most famous and lauded scientific achievements.

In the rest of this post I will make a proposal that I hope can improve this general prestige system. I don’t propose to replace the other better grounded ways to fund science that I described above. If you can use one of them, you probably should. But the demand for prestige affiliation is strong, and may never go away, so let’s see if we can’t find a better way to harness it to encourage scientific research.

I don’t propose to turn the focus of this prestige system away from prestige; that seems a bridge too far. I propose to instead turn its attention toward a better proxy of true prestige: careful future historian evaluations of deserved prestige. Today, we make crude prestige judgments based on past accomplishments and past prestige judgments by others. But we should see these judgments as noisy and error-prone, relative to what future historians could produce if many good ones spent a lot of time looking at any one paper, person, or institution. When science makes progress, we learn better what science claims to believe, and can then better credit prestige to those who advanced such claims.

In particular, I propose that we create speculative markets which estimate the future prestige of each scientific paper, person, project, and institution, and that we treat market prices socially as our main consensus on how prestigious are such things. The historians who make these judgments will be themselves evaluated by yet further-future historians. Let me explain each of these in turn.

For each scientific paper, there is a (perhaps small) chance that it will be randomly chosen for evaluation in, say, 30 years. If it is chosen, then at that time many diverse science evaluation historians (SEH) will study the history of that paper and its influence on future science, and will rank it relative to its contemporaries. To choose this should-have-been prestige-rank, they will consider how important was its topic, how true and novel were its claims, how solid and novel were its arguments, how influential it actually was, and how influential it could have been had it received more attention.

Different independent groups of SEH using different approaches and unaware of each other might be used, with their median choice becoming the official rank. Large areas of related papers, etc. might be judged together at the same time to reduce judging costs. Future SEH can similarly be asked to rank individual people, projects, or institutions.

Assets are created that pay if a paper is selected to be evaluated, and that pay in proportion to the key prestige parameter: some monotonic function (perhaps logarithm) of the rank of that paper relative to its contemporaries. As the number of contemporary papers is known, the max and min of this prestige parameter is known. Similar assets can be created regarding the prestige of a person, project, or institution.

Using these assets, markets can be created wherein anyone can trade in the prestige of a paper conditional on that paper being later evaluated. Yes traders have to wait a long time for a final payoff. But they can sell their assets to someone else in the meantime, and we do regularly trade 30 year bonds today. Some care will have to be taken to make sure the base “cash” assets that are bet are stable, but this seems feasible.

The prices of such markets are visible and recommended as best current estimates of the prestige of such things. Markets can also be created that estimate prestige conditional on favorable treatment by current institutions, to advise the choices of such institutions. For example, one can estimate the prestige of a paper conditional on it being published in a particular journal, or the prestige of a person condition on that person be hired into a particular job. A journal might publish the submissions expected to be most prestigious, given publication.

Scientific institutions should be embarrassed if market price estimates of prestige differ greatly from other markers of prestige, such as jobs, awards, publications, grants, etc. They should work to reduce these differences, both by changing who gets jobs, grants, etc., and by trading to change market prices. In addition to the usual funding of journals, jobs, grants, libraries, etc., science funding could now also take the form of subsidizing historian judging, subsidizing market trading via market makers, and financing hedge funds to trade in these markets. Science funding could also support the collection of whatever data historians say is useful for making later evaluations. This may include prediction market price histories on the chance that key experiments would replicate, that key claims are true, etc.

The SEH who judge papers, etc. from 30 years ago should themselves be judged by possible re-evaluation attempts by SEH another 30 years into the future, and so on to infinity. That is, there should be market prices estimating how much future SEH would agree with the evaluations of particular current SEH, and those prices should be used to rank and choose SEH today. Ideally most SEH are young and expect to be alive in 30 years when they may be judged, personally have a lot of money and status riding on that judgement, and there’s a big chance (say 1/3) of that happening. Assets for all future evaluations could be currently available to trade, and it should be a scandal to see clear differences in market prestige estimates for the same thing evaluated at different future times. Of course other historians who do ordinary history research could be judged in the usual way.

The approach I’ve just described assumes a common prestige ranking of topics and people across all space and time. In contrast, some funders may want to promote science within a topic area or nation without subjecting their choices to uncertainty about the prestige that generic future SEH will assign their topic area or nation. For example, someone might want to fund work on cancer in the US regardless of how important future SEH judge US cancer to be. If so, one might vary the above approach to tie current choices to market estimates of future historians rankings relative to a particular chosen topic area or nation. Generic SEH can probably produce such rankings at a relatively small additional cost, if they are already doing a general evaluation of some paper, person, project, or institution.

Okay, I’ve outlined a pretty grand vision here. What needs to happen next to explore this idea? One very useful exercise would be to hire historians to try and evaluate and rank random scientific work from 30 to 300 years ago. We’d like to see how such rankings vary with time elapsed, topic areas, effort levels, and different random historian teams assigned. This would give us a much better idea of what timescales and effort levels to try in such a system.

(The 30 year time duration is for illustration purposes. The duration should probably be tied to overall rates of economic and scientific change, and thus get shorter if such rates speed up.)

Added 13Dec: I discuss more why this is feasible here.