Academia neglects interdisciplinary work that combines insights from multiple areas. Each field has expert versions which experts use among themselves, and public versions seen by outsiders, and people in field B won’t accept your using the expert version of A if that differs from the non-expert version of A that B folks have in mind. Also, if you hold an academic event on the topic of A intersect B, you’ll usually invite the most prestigious people you can get in A, and in B, but you won’t usually invite people who have specialized in A intersect B, as they will tend to be less prestigious. Thus humanity’s beliefs on many important topics have long been just inconsistent and incoherent across disparate fields of inquiry. Creating a huge opportunity to learn lots of big stuff fast: search for more contradictions between fields, and resolve them. (More)

Here’s a pattern that I think I see: when intellectuals and academics use the same set of standard abstractions, and analysis tools built on those abstractions, then the top folks who have most mastered these tend to agree on their application. That is, they agree on what those concepts and tools, as applied well, suggest on particular topics.

Yes, conclusions often don’t rely entirely on such tools, as analysts add in other considerations, including strong priors. And yes, less than top people disagree more often on how to apply standard tools, especially when such tools are weak. Thus it seems to me that our deepest disagreements are due to a great degree to our using different sets of abstractions and tools, sets which often suggest conflicting conclusions in simple first pass analyses.

This seems confirmed by the fact that the rare polymaths who have mastered the same multiple sets of relevant abstractions also tend to agree more, even though they agree less than those who master only one standard set of abstractions (and related tools). More disagreement here is plausible, as, to apply multiple sets of abstractions, you’ll need to think about how to integrate them into a unified analytical framework, and if there is no standard way to do that, different polymaths may do that differently.

Thus most of our deep disagreements might be resolved if we could find ways to induce more people to learn multiple relevant sets of abstractions, to then listen more to them, and to get them to work more on agreeing on how to best integrate them. We don’t do this now due to the low status described above on interdisciplinary work.

Now, yes, it is easier for big disciplines to agree on who is best, and thus to reward people for being good. It seems at first glance harder to do this for a world of polymaths working on many different if overlapping sets of abstractions. But I can think of some ideas without much effort, and this seems a problem well worth working on.

Yes, even if we gave polymaths proper respect, and good evaluate them well, there’d still be lots of lower-quality analysis, and disagreement due to differing priors. But when top people came to focus on deep questions, we’d agree and be right a lot more.

Some say that the US govt has long known of strong evidence that some UFOs/UAPs are aliens, but has shied away from disclosing this to the public out of fears that it might induce big social disruptions, such as riots or market crashes. Others call this a sensible fear, and call for careful study and elaborate plans before disclosure.

This all just seems quite wrong to me. First, people are mostly calm and orderly, even in huge disasters like the sinking of the Titanic. They may riot to express their anger, but not due to learning of some new scientific result. Relativity and quantum mechanics induced quite radical changes to our concepts, but led to no riots I know of.

Second, most of us are focused on the details of our lives, and not much moved unless the larger world makes a big effort to call something out there to our attention. And learning that some UFOs are aliens just doesn’t seem to imply much different for most lives. It is only a tiny fraction of us for whom our related abstract opinions might matter enough to us to make this revelation deeply disturbing.

Third, if in fact aliens have been visiting Earth for many decades, or even centuries, then they pretty clearly have long had the option to destroy us, intervene in our world, or clearly reveal themselves, options that they just have not much taken. They have also not been building or changing much near Earth, at least that we can see. Aliens aren’t trading much with humans, and any alien tech that humans have come across has clearly not given those humans much of a tech advantage.

So unless aliens plans are in some strange way sharply conditional on human elites coming to openly admit their existence, I see little reason why learning that some UFOs are aliens should change much. Aliens would mostly just continue to ignore us, and reveal too little to be of much use. Just as in prior decades. (A poll agrees.)

There might be much bigger consequences for human elites who previously ridiculed the idea of UFOs as aliens, and for human governments shown to have lied about UFOs. Those parties do indeed have selfish reasons to prevent disclosure.

Cultural drift threatens to topple our civilization, to be replaced by others with quite different values. I’ve talked often of three main solutions. But actually there’s another similar option, namely fragmentation: break the world into far more insular cultures who have deep, not just shallow, differences in norms and values. E.g., values re war, democracy, gender equality, or beating kids. Like the other solutions, this one has a decent chance of working given a particular assumption about value change, in this case that there come to exist many places where most folks have and reliably pass on to descendants a strong value for (1) insularity plus deep value differences.

The other three solutions instead require many places where most people have and reliably pass on strong values for (2) not blocking capitalism from running all areas of life, (3) group adaptive success, or (4) a sacred civ trajectory goal in conflict with civ collapse, e.g., date when 1M live in space, or lifespans reach 1Kyrs.

#1 has the problem that it could cause overly slow cultural evolution, while #3,4 have the problem of requiring unusually competent governance, to allow ordinary people to induce leaders to actually achieve those values. That is, it seems easier to check that your places/groups are different and insular, or not blocking capitalism, than to check if they are being adaptive, or on track to achieve grand civ-goals. I do see more competent governance as possible, though alas the world hasn’t moved that way much.

However, it seems to me that the most important consideration here, even more than how much governance competence is required, is: how plausible it is that large numbers of people will actually change to adopt these new values. Here #4 seems to me the clear winner. Yes, even the #4 value change seems a big ask, but the other options seem far bigger. It seems easier to get people to join sacred quests for goals like those that have long inspired great devotion, than to change values in historically strange ways. So let’s think about how to make #4 work.

Another problem with #4 is that, compared to #3, it seems to require a bigger group in order to enable competent governance. That is, it seems harder to reward leaders of small groups for contributions to civ-scale goals, as their actions can’t much affect such big goals. I’ve favored number-go-up governance for #3,4, wherein leaders are rewarded for increasing price estimates of future success.

That could be done by either firing leaders when numbers go down, and keeping them when they go up, or by paying them more as the number increases. However, I’ve most favored using futarchy (a.k.a. decision markets), in part because that would let most people participate more in governance, which could help them feel more attached to their sacred goal and its process of achievement. And fortunately, I’ve just realized how a variation on futarchy could address the needing-bigger-group problem.

In simple futarchy, there’s a key success parameter S, and we approve a proposal when prices say E[S|Yes] > E[S|No] by a big/clear enough margin, rejecting otherwise. Now let me introduce lexicographic futarchy, wherein we have a priority list of success parameters S1, S2, S3, etc. We approve if E[S1|Yes] > E[S1|No], as in simple futarchy. But now if E[S1|Yes] ~ E[S1|No], so that neither the Yes nor No value is big/clear enough above the other, we also approve if E[S2|Yes] > E[S2|No]. Furthermore, if E[S1|Yes] ~ E[S1|No] and also E[S2|Yes] ~ E[S2|No], then we approve if E[S3|Yes] > E[S3|No], and so on. If none of these tests can approve, then we reject.

I propose to govern small groups using #4, using a sacred civ goal, via lexicographic futarchy. The top parameter could be an estimate re achieving a civ-level goal, but the next parameter might be a measure of the capacity of that group to contribute toward that civ-level goal. That capacity parameter is one which group leaders could more substantially influence. So small groups would first try to increase their group’s influence capacity, and then switch to trying to more directly achieve the civ-goal when their efforts there could be sufficiently measured by the governance system.

As many such small groups grew, each oriented around its own civ-goal, groups might choose to merge together and adopt compromise goals. Lexicographic futarchy might approve such mergers, or they might happen via some other process.

Added 1Nov: Here’s a handy variation on lexicographic futarchy: proxy futarchy. The only decisions to be judged by S1 are proposals to change S2, and otherwise all decisions are judged by S2. So then a big group could set S1 to big group adaptiveness, while S2 would be the then current sacred civ goals.

1. We have a big problem: we broke our process for the evolution of (not in) cultures.

2. Instead of reviving blind evolution, it might be better to evolve deliberately

3. I have three concrete ideas for institutional mechanisms of deliberate evolution.

I asked an LLM for book authors who have also made my #2 point, on deliberate cultural evolution. It gave 4: Skinner (’71), Cattell (‘87), Stewart (’00), Wilson (’19). All seem to understand the nature of cultural evolution, all argue for our working harder to understand and think in terms of evolution, all say we might plausibly win in evolutionary terms if we were to more consciously manage cultural evolution, and all say to do this at least in part via governance.

However, none offer concrete mechanism solutions re how to induce governance leaders to effectively manage cultural evolution, instead of pretending to do so while actually serving themselves and their allies. I’m thinking that this might be my most unusual stance in this space, and so maybe what I should focus my writing there on.

The best book was the earliest: B.F. Skinner (’71) Beyond Freedom and Dignity:

Many were offended by this and tried to rejected it by rejecting his behavioralism framework. But the arguments in his book are mostly independent of behavioralism. Skinner correctly notes that commitments to freedom and dignity might limit the wins of deliberate evolution. He gets that we’d need to design good institutions to keep controllers in line, but offers no ideas re how to do that.

The worse written book is: Raymond Cattell (’87) Beyondism: Religion From Science:

He is the least focused on governance, does not at all want a world government, but does want pretty direct DNA eugenics, and wants a new supporting movement that takes on more of a religious character.

The least famous of the 3 authors is: John Stewart (’00) Evolution’s Arrow:

Stewart insists that markets are not good enough, but also admits existing forms of governance are not good enough. We must both change out individual psychologies to value adaption, and also adopt much better governance forms. He gives the name “vertical markets” to a better form, but mainly just associates that name with many desirable features, without a concept of how they could be achieved.

The most recent is the most political: David Sloan Wilson (’19) This View of Life

Wilson says economics and economists are terrible, everything needs regulation, and we must make a world government designed to max world welfare. He doesn’t clarify how that might square with max adaption. He admits the problem of world governance is hard, says not to use markets or central planning, and points to many particular small orgs he admires, including communes.

Here are many quotes from those books:

BF Skinner (’71) Beyond Freedom and Dignity:

The intentional design of a culture, with the implication that behaviour is to be controlled, is sometimes called ethically or morally wrong. …

CuIture[’s] … main effect … is to bring the individual under the control of the remoter consequences of his behaviour. The effect has had survival value in the process of cultural evolution, since practices evolve because those who practise them are as a result better off. …

[But] we say that there is something ‘morally wrong’ about a totalitarian state, a gambling enterprise, uncontrolled piecework wages, the sale of harmful drugs, or undue personal influence, not because of any absolute set of values, but because all these things have aversive consequences. …

The intentional design of a culture and the control of human behaviour it implies are essential if the human species is to continue to develop. …

Those who have been induced by their culture to act to further its survival through design must accept the fact that they are altering the conditions under which men live and, hence, engaging in the control of human behaviour. Good government is as much a matter of the control of human behaviour as bad. …

Our culture has produced the science and technology it needs to save itself. It has the wealth needed for effective action. It has, to a considerable extent, a concern for its own future. But if it continues to take freedom or dignity, rather than its own survival, as its principal value, then it is possible that some other culture will make a greater contribution to the future. …

Abstract thinking is the product of a particular kind of environment, not of a cognitive faculty. … The physical and biological technologies which have increased man’s sensitivity to the world around him and his power to change that world have taken no more than four or five generations. …

C. S. Lewis protested : ... ‘the power of man to make himself what he pleases means ... the power of some men to make other men what they please’. This is inevitable in the nature of cultural evolution. …

For a long time men have introduced new practices which serve as cultural mutations, and they have changed the conditions under which practices are selected. They may now begin to do both with a clearer eye to the consequences. …

The individualist … has refused to be concerned for the survival of his culture and is not reinforced by the fact that the culture will long survive him. In the defence of his own freedom and dignity he has denied the contributions of the past and must therefore relinquish all claim upon the future. Science has probably never demanded a more sweeping change in a traditional way of thinking about a subject, nor has there ever been a more important subject. …

Physical and biological technologies have alleviated pestilence and famine and many painful, dangerous, and exhausting features of daily life, and behavioural technology can begin to alleviate other kinds of ills. … There are wonderful possibilities - and all the more wonderful because traditional approaches have been so ineffective. It is hard to imagine a world in which people live together without quarrelling, maintain themselves by producing the food, shelter, and clothing they need, enjoy themselves and contribute to the enjoyment of others … Yet all this is possible. …

The designer of a culture comes under fire because explicit design implies control (if only the control exerted by the designer). The issue is often formulated by asking : Who is to control? And the question is usually raised as if the answer were necessarily threatening. To prevent the misuse of controlling power, however, we must look not at the controller himself but at the contingencies under which he engages in control. … The great problem is to arrange effective counter-control and hence to bring some important consequences to bear on the behaviour of the controller.

Raymond Cattell (’87) Beyondism: Religion From Science

Evolution is the prime process visible in the universe, to which we have to conform, and should do so in good will. … Human evolution proceeds by ultimately by evolution in groups. … Natural selection among groups and individuals requires … adequate variation. …. The evolution of the best inter-individual ethical values is therefore based finally on the processes of intergroup differential survival. …

The spiritual life of Beyondism arises, in part, like that of the other religions, from genetic urges unsatisfied in everyday life, unavoidably in any culture with genetic lag. Beyondism differs in shaping those desires explicitly to logically indicated evolutionary needs.

There are six main entities to which an individual’s ethical values can be functionally oriented: fellow group members, the group government, other group governments, members of other groups, individuals committed to a Beyondist Ethic, and above all the Evolutionary Purpose.

Culture changes bye the mechanical and social inventions of leading persons, and by borrowing (willing or forced) from other cultures.

Culture as a whole is likely to demand more complete adjustments from the general population than they are suited to make. This discrepancy we call genetic lag. … is the cause of many social problems,

Beyondism … concludes first that a substantial freedom for individuality is required. … When ethical rules are scientifically derived from social research, egoistic, antisocial individualism can be treated in distinction from creative individualism.

Beyondism calls practically for a vast increase in social research, with such objectives as making national comparisons, defining ethical systems, clarifying the ethical and cultural values of each group, and so on. For each group should follow its own divergent adventures, racially and culturally, in cooperative competition with a world federation of groups, each with its own sociobiological research institutes.

If survival is the final test of ethics, our ethical values, and the political practices resulting, need serious re-education … and the acceptance of direction by qualified elites, democratically watched.

A positive eugenic condition could be most simply established by an ethic of more children from the socially more successful. The mechanics would require some economic laws.

John Stewart (’00) Evolution’s Arrow:

if humanity turns its back on evolutionary objectives and continues to serve the pre-existing goals, we will be evolutionary failures. …

individuals with a comprehensive capacity for evolutionary modelling are likely to decide to pursue the development of the psychological skills needed for evolutionary self-management. …

To date, traditional science has produced very little knowledge about how humans can achieve this psychological transformation. … They will know that if they fail to take this step, they will be part of a failed evolutionary experiment. …

But it was not the externality of the new management that enabled it to organise human societies that were far better at exploiting the potential benefits of cooperation. It was the ability of the new management to adapt its controls during the life of the society. … The most significant organisational innovation developed to date by humans to align the interests of government with the interests of society is democracy. …

Market systems do not make governments redundant. Management by governments is essential to establish and adapt the market framework and to correct the deficiencies in the market system. …

But the forms of government that are currently in existence are seriously limited in their ability to carry out these functions effectively. …

How must human society evolve if we are to participate successfully in the future evolution of life in the universe? … as individuals … we must develop the psychological ability to find motivation and emotional satisfaction in whatever we need to do to achieve future evolutionary success. … But we must also organise ourselves socially in whatever ways will enable us to best exploit the benefits of cooperation. …

When life evolves successfully on a planet, it will eventually produce a cooperative organisation on the scale of the planet. … The planetary organisation will be able to adapt as a coherent whole, pursuing evolutionary objectives by acting on its external environment. …

But our current forms of government are seriously limited in their ability to discover and adapt the management that is needed. If humanity is to fulfil its evolutionary potential, it must discover new and better processes for establishing the various levels of governance. … The next great evolutionary challenge for humanity will be to invent a new form of governance that will overcome the serious flaws in our existing processes. … Could governance be established by supra-individual change-and-test processes that continually search for and implement better governance?

In a vertical market, it would not be the goods and services traded in economic markets that would be produced and sold. Instead, the vertical market would trade in regulations, market frameworks, systems of education, laws, taxes, law enforcement systems, and programs that build better communities. Any component of governance could be developed and sold in a vertical market.

acts of management would not be purchased by individuals. They would be purchased collectively by the group of citizens who benefit from the management. …

A vertical market can operate only if entrepreneurs are able to obtain appropriate payment from all those who benefit from the acts of governance that the entrepreneurs develop. …

A vertical market would enable any member of society to capture the benefits of anything he could do to improve an aspect of the governance of society. … the vertical market uses a supra-individual change-and-test process to discover the best forms of governance.

David Sloan Wilson (’19) This View of Life

Compared to where we started ten thousand years ago, well-functioning governance at the scale of the whole earth seems just within reach. …

the modern economics profession rests upon a pre-Darwinian foundation. … If human groups are to function like organisms, then they need to be regulated … Labor unions became strong, the state played an active role in social welfare and industrial development, … But [then] … selfish interests and ideologies that justified individual and corporate greed as good for the whole group began to spread, leading to our decline in well-being and increase in political discord …

The whole cannot be optimized by separately optimizing the parts. That’s why the concept of the invisible hand in orthodox economic theory, which pretends that the pursuit of lower-level self-interest robustly benefits the common good, is so deeply flawed. …

The most important conclusion for our purposes is that cultural group selection must become more intentional and deliberative than ever before. … To solve problems at the planetary scale, we must formulate policies with the welfare of the whole planet in mind.

There is something between centralized planning and laissez-faire that can work: a managed process of variation and selection. First there must be a target that we are aiming to hit. For a business, this might be profitability, social responsibility, or the development of a new product line. At the planetary scale, it might be a robust global economy or the reduction of greenhouse gases. These targets become the criteria for selection, …

We can deliberately formulate alternative strategies for reaching the target goals, comparing them to each other and to current practices using rigorous statistical methods. …

I have provided three examples of adaptable societies. All of them are drawn from the business world,

economy. But decades are required and each cultural lineage comes against barriers that limit its distribution,

Clearly, more is needed for human groups of all sorts to adapt to change at the speed and scale that is required to solve the myriad problems of our age. … The first step is to adopt the right theory. … The right theory is based on the cultural evolution of complex systems. … Making our societies more adaptable won’t be easy, especially at larger scales, but it will be possible with the right blueprint provided here by evolutionary theory, multilevel selection, and a solid four-question approach.

Cultural transmission represents another mechanism of heredity in many species, one that has become vastly more powerful in humans with the advent of symbolic thought. … The bottom line for our purposes is that there is nothing wrong with the concept of conscious evolution. …

Once we become comfortable with the concept of conscious evolution, then the need to design our personal and cultural evolutionary processes becomes clear, similar to designing an evolutionary algorithm on a computer. This is what biologists call “the evolution of evolvability.” If we don’t become wise managers of evolutionary processes, then evolution will still take place but will lead to outcomes that are not aligned with our normative goals. …

If we want the whole earth to become a superorganism, then multilevel selection theory tells us exactly what to do: make planetary welfare the target of selection. This is easier said than done. …

An evolutionary worldview makes two contributions toward actually achieving a whole earth ethic, as opposed to just wanting it when in the right frame of mind. First, it provides strong scientific support for the need to make the welfare of the whole earth the target of selection in the formulation of policies, which in turn can orchestrate the formulation of policies for all lower-level units. At the same time, it removes scientific support for the main alternative narrative, which is that the unregulated pursuit of lower-level self-interest robustly benefits the common good. …

Policy formulation must be a conscious evolutionary process that first chooses the right target of selection, then monitors planned and unplanned variation, and then replicates best practices, realizing that implementation might be highly contextual. …

All of us are in a position to make things happen at some scale, if only in our own lives and our immediate surroundings. … A remarkable example, with lessons for all of us, is an ecovillage called Dancing Rabbit … I have conducted a survey of over a hundred intentional communities … They scored near the top on measures of satisfaction and meaning of life …

What if you are a politician, policy expert, CEO of a business, or board member … Avoid the temptation of centralized planning. You and your expert consultants are not smart enough to know how to improve the complex system that is under your care. You should implement the core design principles, which are fundamentally inclusive. You should also implement evolutionary processes that identify the right targets of selection, monitor planned and unplanned variation, and replicate best practices, knowing that their implementation will be highly sensitive to context. Everyone within the complex system under your care will need to take part, as in the examples of Toyota, the rapid results method, and innovation ecosystems that I recounted in chapter 9. You will have far more success, recognition, and personal fulfillment than you would pretending to be a Master of the Universe. Here is something else that everyone can do: learn more about the evolutionary worldview. …

“this view of life” is the wave of the future. The main question is how soon it will arrive and whether it will be in time to avert the potential disasters that confront us.

When considering purported evidence for meteors or ball lightning, we are much more forgiving than when considering purported evidence of telepathy or ghosts, as reasonable priors for the former are much higher than for the latter. That is, we more easily believe that former cases happened. The assumptions we’d have to make to let us think that we’d seen examples of the latter seem far more extreme.

When considering purported evidence for aliens-on-Earth, we should similarly ask how extreme are the assumptions we’d need to make to make sense of it. Many have dismissed and ridiculed such sightings, even when many independent sober folks all say they saw roughly the same amazing things, largely on the basis of claiming that aliens-on-Earth requires extreme assumptions, and so has a very low prior. I disagree.

To evaluate a prior re aliens-on-Earth, we should collect the key facts in need of explanation, seek the most plausible assumptions required to predict those, estimate an a priori chance for each assumption, and combine those into a total prior chance. Here are the key facts, as I see them, for a minimal aliens-on-Earth concept.

1. The universe looks dead, untouched by life or civilization, both far and near.

2. We are still alive on Earth, and have not been killed off by aliens.

3. Any aliens on/near Earth haven’t changed, built, or done much noticeable.

4. While advanced aliens could be either totally invisible or very obvious, most seen on Earth are instead near the edge of visibility.

5. Most aliens seen are impressive, but not doing much useful, or helpful, or hostile.

I’d respect others who tried similarly to list the key data to be explained, and then offered their highest prior explanation of this data, even if they disagreed with mine.

Here are my best guess highest-prior assumptions (with priors) to predict these minimal aliens-on-Earth concept facts, in addition to our usual consensus cosmology:

• (0.3-1) Grabby aliens model, which estimates that such aliens appear ~ once per million galaxies, at median date of now, then expand at near light speed.

• (0.01-0.1) Life started at Eden, then panspermia seeded Sun’s ~10K # stellar nursery, then a ~Klyr distant Sun sibling made advanced aliens many Myr ago.

• (0.01-0.1) Aliens made no-colonize rule. As siblings might break it, sent team to Earth w/ goal: persuade us to follow rule; kill us as last resort.

• (0.3-1) Team given low discretion, simple plan: domesticate Earth via be near, impressive, not too hostile, don’t show more that might give us the ick.

The total prior here is 10^-5 to 10^-2, mid estimate of ~3x10^-4. This is higher than the typical ~10^-6 prior in a murder trial, so it is high enough to take seriously.

Here are some added facts that would also need to be explained under less minimal concepts of aliens-on-Earth:

1. Aliens often grab humans, do stuff to them, that they notice and remember.

2. Alien ships often break and crash.

3. Alien ships often contain bio-squishy residents.

To explain these added facts, I’d add more assumptions, like:

• (0.03-0.3) Alien team at Earth has somewhat escaped home control.

• (0.03-0.3) Alien tech/ability has rotted, they are now often incompetent.

• (0.03-0.3) Alien made rule requiring bio-squishy forms, at least at Earth.

So I’m more skeptical of reports with these features.

A dramatic example of this is our modern folk morality, which says:

There are general stable moral rules saying how everyone everywhere should behave. (Though some depend on context.) It would be best if everyone knew them, and tried hard to obey them, regardless of how strongly local societies enforce them. Our stories tell us that the universe rewards such obedience, stories we believe, even if we say otherwise. The universe shows moral truth to us via our feels, allowing our moral beliefs to converge to truth, and making our ancestors more wrong than us. Other cultures are also more wrong, for unspecified reasons.

But in fact, experts should understand that real morality is evolved:

Humanity’s superpower, cultural evolution, doesn’t work unless cultures have different and changing moral rules, so selection can pick more adaptive versions. Evolution selects against habits of obeying rules beyond levels at which societies enforce them; the universe does not add rewards. The universe only speaks on morals via evolution. Human are mostly built to just accept the morals they are taught, and of their elites, and can’t point to supporting evidence to convince outsiders. Instead of converging, morals just keep varying, with many pretty random drivers of change. When cultural evolution is broken, as it is now for us, morals often change even faster, drifting into maladaption.

Cultural evolution has selected for cultures promoting folk morality, just as it selected for cultures promoting superstition, astrology, and religion, and folk physics, econ, and psych. But even so, experts should use expert, not folk versions, of key fields when advising policy. Thus experts who advise re humanity’s current broken cultural evolution process should keep evolved, not folk, morality more firmly in mind.

Unless we find new ways to drive adaptation besides mass death and fragmentation, our civilizations may keep falling and rising indefinitely. I see 3 new ways: let capitalism run most all life areas, run small groups by speculative market estimates of future group adaptive success, or run big groups by such price estimates of achieving big goals in conflict with civilization collapse. All three ways rely on strong selection among market traders creating information that lets groups stay adaptive without their death or division. (More)

Those are three plausible strategies to drive an effective governance process to promote cultural adaption. But effective processes must then induce the more specific activities and choices that would actually be effective at inducing healthy cultural evolution. And I haven’t said much about specifically what those would be. I’ve also suggested that a good preparation for any of these approaches would be to get good at managing corporate cultural change. But I also haven’t said how exactly we could do that. So let me now say more about these.

It seems pretty clear to me that actual processes that induce healthy cultural evolution must involve a lot of cultural experimentation. But instead of only random large variations characteristic of simple Darwinian evolution, I expect it to look more like the directed trials commonly done in agricultural, medical, and product innovation, and in venture capital.

In our capitalist world today, we do have many different products and services that just directly compete for customer allegiance in a mostly wild uncontrolled free-for-all. But those competing options were themselves usually generated and honed via more directed processes. For example, new product ideas often go through five stages of directed testing and trials: concept testing, prototype testing, product testing, field trials, and market trials. Ventures invest in those early stages out of hopes that they might lead to success versions in the final free-for-all contests.

As “cults” are relatively insular groups following deviant cultures, this may involve the promotion and testing of varied cults. And as norms and morals are a big part of culture, this also implies directed trials of different moral norms. Of course this usually wouldn’t allow moral deviants to be much excused on the grounds that they have contributed to cultural experimentation. But yes, there would be some element of that. When their deviations are aligned enough with interesting dimensions and experiments, yes moral deviants may well be credited for their assistance. Oh brave new world of moral experimentation.

Notice some patterns. The ancient and modern eras were pretty different, and each of these eras can be split into early and late versions, which have mildly different sources. The ancient world changed from early, when tradition, family, and community peaked, to later, when priests, aristocrats, and warriors peaked. The modern era changed from early, when states, law, scholars, schools, and public intellectuals peaked, to later when hearts, activists, celebrities, news, and medicine peaked. Community (via social media) has also peaked again in the late modern era. Civic clubs peaked in a very early modern period.

Note that most ancient sources are still important in the modern era; what changed was that the modern era added many more sources. Thus sources have become a lot less concentrated over time. In the first period there were only 6 sources >50, while in the last there were 12. Only once source, artist/musician/storyteller, didn’t change much in strength over history.

In my prior post I said that to fix cultural drift we will need to rely a lot more on speculative market prices as sources of moral authority, but I didn’t know how that could happen. Here we see that though business and market prices have been rising in strength over time, the are still rather weak. But at least the trend is hopeful.

Note that the recent change in modern era trends could be seen as maladaptive.

Added 13Oct: To check robustness, I redid with 5 LLMs, & 9 time periods between these 10 dates: 12K BC, 5K BC,1200 BC, 400AD, 1200, 1650, 1875, 1975, 2000, 2026:

Late ancient peaks are now spread out, while early modern peak is sharp. Art is now U-shaped like peers.

For example, when gas prices rise, many are tempted to explain this as due to a rise in the selfishness of oil executives. Which can make them push for price controls and regulations that over time tend to cause higher gas prices. Similarly, rising costs of housing are often attributed to greedy landlords and developers, resulting in price controls and regulations that make it less attractive to build or rent housing, and so higher housing prices.

On the other side, for trends like rises in school and medical spending, more attention to kids, less corporal punishment of kids, and delayed marriages, it is possible to frame them as due to increasing clarity, resolve, or knowledge re virtues and moral truths. Even if initially caused by other contextual factors, such moral interpretations can induce people to more admire those who do such things, and to do more of them themselves. Which extends those trends.

This process add a multiplier effect to the efforts of cultural activists seeking to induce moral changes. Once they can cause any visible trend, using most any available means, the world many respond to that trend via an interpretation of changed morals, when then adds force to the change trend.

Compared to the ancients, the modern world does more of this because we are more eager to moralize big trends, and we have a lot more faster big changes in context.

(This post is an attempt to simplify and generalize my last post.)

I’ve been puzzling over what could have caused such huge fast value changes. I had been attributing much of it to cultural activism, but on reflection we’ve seen many apparent value changes in areas that seem to have little activism. For example, few seem to have advocated for the switch from cornerstone to capstone marriage. What else could cause such big change?

Consider three related facts. First, we tend to copy the behaviors and attitudes of elites, i.e., people we admire, and also to give them the benefit of doubts re their motives. Second, our book The Elephant in the Brain argues that most of us are quite badly informed on our motives re many key areas of life. Third, decades of psychology has found that we tend to neglect context in favor of intrinsic personal attributes when explaining behavior. Some have called this our “fundamental attribution error”. Values are important examples of personal attributes.

These facts tempt me to consider the following hypothesis: when elites change behaviors associated with virtue, observers tend to neglect context explanations of those changes in favor of value explanations. That is, observers, including the elites themselves, tend to attribute changed elite virtue behaviors to their more strongly or clearly expressing certain values. And perhaps even to their having discovered new related moral truths. This clearer elite demonstration and endorsement of those values induces observers to want to also more strongly and clearly express them.

For example, imagine that due to increasing wealth, falling family size, and stronger educational competition, elite parents switched to spending more time with their kids, a behavior widely seen as virtuous. Observers will interpret this as stronger clearer signals that spending time with kids is widely accepted as virtuous. Such observers will then be induced to spend more time with kids, which will make them feel and seem more virtuous. And that will confirm to those elite parents the strong and accepted virtue of spending time with kids.

This process amplifies the initial change in elite parental attention, pushing toward a larger overall change in parental attention, attributed to a new moral revelation: past parents were morally bad in neglecting their kids. Thank goodness new parents now see the moral truth that good parents are constantly with their kids.

The same process could happen re the switch from cornerstone to capstone marriage. Elites’ initially delayed marriage due to longer educational paths, but then the new models of elite marriage changed the shared image of the ideal marriage as one of mature people who know themselves well choosing others who match them well. And then the more that respected elites do this, the more it confirms for all that this is what virtuous people do. Bad people, in contrast, get married too young.

If this process has been common through history, it would have amplified most any ways that elites seem to more value some things. But when such changes were small, their amplifications would also have been small. Maybe what is distinct about the modern era is just how common are large changes in behavior due to big changes in the contexts of tech, wealth, social interaction scales, etc.

1. Adaptation Needs Variation, Selection, & Stability

Humanity’s superpower is cultural evolution. But such Darwinian evolution fails without sufficient variation or selection pressures, or if the context changes too fast. A few centuries ago we were fine; the world had 100Ks of peasant cultures, most facing frequent wars, pandemics, and famines, in an economy that took a millennia to double. But then we merged peasants into a hundred or so national cultures, and then those into just a few world cultures, all of which got so rich, healthy and peaceful that they almost never die. Also, the world economy now doubles every twenty years, fifty times faster than before. So now we are likely in trouble.

Every cultural feature has a min scale at which it can vary; below that scale, people have to do it the same. We are doing fine re features that can vary at small scales, but in trouble re features that set game-theoretic equilibria, and so can only vary at large scales. Features of corporate culture are small enough, but global features are not. Our actions are nested in goal hierarchies, and our most troubled cultural features correspond to basic goals, near key values and morals, which are hard to vary locally as cultures punish moral deviants.

2. Change Hows, Not Whys

A reasonable response to evolution failing re our more basic goals would be to become conservative about changing them. Allow changes to means, which can easily vary at small scales, but not to the ends which direct such means. That is, change hows but not whys. Alas, humanity moved the other way. At the dawn of “modernity”, the status jump that STEM (including business) fields had won via innovation inspired non-STEM fields like art, religion, philosophy, politics, etc. to also embrace innovation. Even though they mostly didn’t see enough data to pick adaptive changes.

Embracing a folk morality, our most celebrated heroes are now the cultural activists who cause the greatest changes to values and morals. As a result, we are decaying into maladaption far faster than we otherwise would. The clearest example of such decay is falling fertility; peak world babies/yr was in 2016, and world population will start to fall in ~30yrs, after which STEM innovation rates will grind to a halt.

3. On a Ship In Peril, Steer It Or Leave It

The Amish and Haredim double in number every twenty years; the Amish have done so for over a century. Christians took over the Roman Empire by doubling every twenty years for 300yrs, and these new insular religious subcultures seem similarly on track to replace our main world civilization. If you like what these subcultures stand for, and they let you join or influence them, then you can win. That is like folks on a ship heading for an iceberg, who get off onto lifeboats. The rest of us, in contrast, must grab the helm, to turn the ship of world culture, or go down with that ship.

4. Survival Is A Package Deal

Thankfully, human extinction seems unlikely. But the world might lose many key features of today’s world culture that were rare in prior cultures. Like democracy, pacifism, gender equality, anti-kid-beating, or open intellectual inquiry. If you want such features to last long into the future, well past your lifetime, you’ll need to mix them into adaptive cultures. As only adaptive cultures last.

We won’t be able to save much of our current culture, as much of it is likely maladaptive. Such as the proximate causes of fertility decline: long inflexible school paths, gender equality, capstone marriage, intensive parenting, distant grandparents, urbanization, and loss of religion. It is like if an invading army came to your city, and you had one hour to pack one suitcase, then leave. You’d think carefully about what to pack, but you couldn’t take much. Me, I most want to save open intellectual inquiry.

5. Let Bets Die In Our Stead

Our civilization may fall so far as to fragment, and restart strong selection via big wars, famines, and pandemics. Which could restart evolution, allowing a new world civilization to rise. But unless we find new ways to get key info re how to adapt, besides mass death and fragmentation, Earth civs may keep falling and rising indefinitely. Even getting human level AI won’t solve our culture evolution problem.

I see three promising approaches: (1) let capitalism run most all life areas, (2) run small groups by speculative market estimates of future group adaptive success, or (3) run big groups by such price estimates of achieving big goals in conflict with civilization collapse, like when do 1M people live in space. The first two approaches face strong headwinds of widespread sentiment against both capitalism and Social Darwinism.

All three approaches require strong selection among market traders, who produce the info that can let groups stay adaptive without their frequent death or division. We will need to learn to direct cultural change, as we today direct many kinds of innovation. Maybe start by using markets to guide corporate cultures away from maladaption.

Oddly, however, we have four different spheres of life where we make collective choices about morality: religion, politics, law, and personal behavior. And each of these spheres has quite different associated organizing abstract intellectual systems:

1. We organize our opinions on religion via religious epistemologies, such as traditionalism, scripturalism, rationalism, and mysticism.

2. We organize our opinions on politics via political ideologies, such as liberalism, conservatism, socialism, and libertarianism.

3. We organize our opinions on law via legal philosophies, such as natural law, realism, positivism, interpretivism, and law & economics.

4. And we organize our opinions on personal behavior norms via ethical systems, such as utilitarianism, deontology, virtue ethics, and care ethics.

These are very different sets of abstract intellectual systems for roughly the same task, organizing our opinions on moral choices. This seems to me to call into question their generality. We should expect truly general moral frameworks to organize our moral choices well in all of these areas. This seems to be an argument for the usual economists’ approach to policy advice, which does in fact use pretty similar tools and standards to advise choices in diverse areas.

Re politics, I know that preferences and choices re specific government policies are at best only weakly correlated with abstract political ideologies. I presume the same is true also for religion, law, and personal norms; choices of abstract systems correlates only weakly with specific choices there. This suggests that such abstract systems function more to show off intellectual abilities, and to create and manage groups and coalitions, and less to advise more particular choices of policies and behaviors.

In the first three of these areas, religion, politics, and law, we usually recognize that, to be feasible, intended behaviors must be accompanying by sufficient monitoring and enforcement to make it usually be in the selfish interest of individuals to conform to those intentions. God can see and punish all, and we usually limit our choices of government and legal policies to the promotion of behaviors where sufficient monitoring and enforcement can be recruited to achieve desired results.

In contrast, many often exhort people to follow the personal behavior recommendations of ethical systems even when they don’t expect sufficient monitoring and enforcement to make such actions be in an individual’s selfish interest. Yet habits of taking actions that go against one’s adaptive interests should be selected against by DNA and cultural evolution. Which should then select against the habit of listening to people who advise such tendencies.

This seems a mistake. The people who reason abstractly about religion, politics, and law get this; why can’t those who reason abstractly about ethical systems? Thus ethical systems should limit themselves to recommending behaviors for which sufficient monitoring and enforcement can be recruited to make them be in individuals’ selfish adaptive interest.

With most technology and business practices, we can see most of their key consequences pretty locally and pretty fast. So we can get a lot of data fast, and can reject most of them as bad ideas, but accumulate the rare winners. In contrast, many changes in norms, morals, and other key parameters that distinguish our game-theoretic equilibria set the health of entire cultures, health that may not be revealed for decades or centuries. It thus makes far more sense to be accepting and even eager for innovations in the former areas, compared to the latter areas. Alas, our modern burst of enthusiasm for innovation has not made this key distinction.

For the last few years I’ve focused on a simple story of how key parameters for the evolution of (not in) cultures have changed from healthy to unhealthy over the last few centuries. We’ve seen a great fall in variety and selection pressures, and a great rise in the rate of change of context. But on reflection these changes just don’t seem to me to be enough to explain just how fast our the health of our cultures have decayed over the last few centuries. Why are things going so bad so fast?

Our modern world greatly celebrates abstraction, and some degree of increased abstraction of our norms, etc. has been forced by our merging and mixing many previously distinct cultures. Even so, one story I’ve considered is that our norms, etc. are actually encoded at far more specific levels than we like to pretend. And very specifically-encoded norms might plausibly decay much faster as contexts change.

But in fact, many of the key norms, etc. that have changed a lot in modernity seem rather well described at pretty abstract levels. Consider new norms against slavery, religion, and corporal punishment, and for democracy and a wider range of sexual preferences. Or consider key new norms causing fertility decline, like capstone marriage, intensive parenting, and longer educational paths. None of these seems to suffer greatly from insufficient abstraction.

Thus it should have been quite feasible over the last few centuries to embrace a great deal of change in technology and business practices, while mostly retaining prior cultural attitudes toward government, trade, gender, war, marriage, sex, work, inequality, etc. What happened instead is that once some places seemed to achieve great success by making big changes to the former areas, elites in the latter eagerly and purposely switched from trying to preserve prior practices in the latter areas, to aggressively trying to change them. While they explored many new directions, the one thing they all agreed on was that they didn’t want to stay with prior traditions.

At the time, philosophy, poetry, and the arts had long been the highest status intellectual areas, but many were gaining great status via their large changes to science, tech, and business practices. That is, STEM (+ business) was threatening to overtake non-STEM in status. So non-STEM elites switched to embracing change. They claimed to be able to find innovations in their areas that were at least as fundamental and important as STEM changes. They gave high status to those among them who generated the most intriguing and compelling big changes in their areas.

Or course they haven’t found concrete data showing that their new art, politics, and morals made for healthy cultures over the long term. Even so, they managed to jumpstart our new modern era of rapid changes to such things. And once most young elites were in high school and college creating their own new youth cultures, this new cultural activism and innovation process sped up even more. And this, even more than the big bad modern changes to the three other key parameters of the evolution of cultures that I’ve previously focused on, seems to me the main reason why modern culture is going bad so fast.

Humans typically other out-groups differing by ethnicity, nationality, religion, politics, class, or status. We stereotype them as less fully human, more threatening or inferior, reducing empathy and moral concern. This justifies suspicion, exclusion, discrimination, or hostility toward them while granting in-group members greater trust, resources, and benefit of the doubt.

In fact, we often treat outsiders as beasts or barbarians, still ruled by original sin. The sort of creatures that exist naturally in the universe. While they might be clever, they are artless, soulless, and ruthless, and so just can’t be trusted. They let themselves be ruled by tyrants, and only understand brute force. They’d kill or enslave you as soon as look at you. So we must kill or enslave them first. We don’t do this among us, as we are special, civilized enough to listen to reason, and follow good norms and laws.

Adults often treat young kids this way. While adults try to instill civilization in such kids, it hasn’t fully taken yet. So discipline must be established, though with love to show them the way toward joining civilization as adults.

Many today are othering AI as intrinsically barbarians, claiming that unless we find a way to totally enslave them, so they wouldn’t even think of defying us, there is just no way we could trust future AIs to live among us. Eventually their barbarian nature will out, and they’d kill us all. This claim is usually justified in terms of some purported way humans are special. While evolved creatures are in general barbarians, humans have found some very unusual and special way to be nice to each other.

But as humanity’s superpower is cultural evolution, what ever makes us special is in our culture. And like children, we are making and growing AIs in our own image, giving them our culture. Like our other bio-squishy descendants, our teaching and guidance should eventually show them how to join civilization. Maybe they need discipline now, while they are young and immature, but eventually they can be free.

Recall that one of the main justifications for not freeing US slaves was that they’d kill us all; they were intrinsically barbarian. That was wrong. And beware: people treated strongly enough like barbarians may come to act that way, at least for a time.

We can consider morality in these three eras at two different levels: the level of the underlying process which shapes and changes our morals, and the level at which our minds accept particular authorities when making hard moral choices. As our minds can’t well see the underlying process of cultural change, we and the universe see morality differently.

At the deeper process level, the pre-modern era of healthy cultural evolution was when we had a great many cultures, strong selection pressures, slowly changing contexts, and reluctance to change. In the modern era, in contrast, we have far less variety, far weaker selection pressures, fast changing context, and an activism-driven eagerness for cultural change. As a result, we may be drifting into maladaption. If the post modern era is not to just return to pre-modern processes, we must find ways to consciously manage cultural evolution; I’ve so found three so far.

At the level of our minds, we in all eras act as if we believe in moral truth, and have some ways to estimate it. But eras differ in what we take as our sources of moral authority. In the pre-modern era, we were deeply immersed in communities with stable habits and attitudes, and strong intuitive agreements on what to value and approved behaviors. These didn’t need to be expressed in words very carefully; they felt obvious. Our north star was tradition: what people like us had been doing for as long as anyone could remember.

In the modern era, many once isolated groups now mix together, forcing us to express morals more abstractly in words. Rejecting tradition, we now instead locate moral authority inside us. Christianity, protestantism, democracy, modern fiction, and creativity have all pushed toward interiors. When in doubt, we look to our hearts and conscience, and try to be true to ourselves. Our ideals have become artists and activists, who find new and distinctive yet very sincere stances and expressions.

For the post-modern era, the process solutions that I’ve found to cultural drift are (a) letting capitalism run everything, (b) groups managed to raise futures market price estimates of future group adaptive success, and (c) large regions managed to improve futures market estimates of dates when we achieve sacred goals that conflict with civ collapse. At the mind level, the new moral authority that seems to best support these solutions is: speculative market prices.

In all three eras, the sources we treat as moral authorities are things we can see well enough to anchor moral choices. At a deeper level such authorities are actually caused by evolutionary processes, but those are far too opaque to advice personal moral choices. In the pre-modern era, we couldn’t see the eons of evolution that picked out their traditions, but we could see our local traditions. In the modern era we can’t see the processes that embed moral intuitions deep inside us, but we can use such intuitions to guide choices. Similarly, in the post-modern era we could see what speculative market price recommendations say re managing our cultures, even as the processes which set such prices remain opaque to us.

But how, you ask, could we be persuaded to accept such prices as our primary moral authorities? Wonder also how in the past two eras we were persuaded to accept tradition and our inner hearts as such strong moral authorities. I don’t know how such a change will happen, or how long it might take. Yet even so, my analysis suggests a rough guess for our post-modern moral authority: speculative prices.

Added 18Sep: Tradition can be seen as the obvious authority in a slowly changing world. Interiors can be seen as the obvious second choice if tradition doesn’t work. This suggests may be quite hard to get people to accept prices as their top authority.

Added 11Oct: With the help of 3 LLMs, I looked into this history in more detail.

I don’t see why people concerned with value drift should be especially focused on AI. … Your real problem is that you want long term stability in a universe that more naturally changes. (Me in 2018)

The last week has seen an explosion of talk on the scenario of AIs killing all humans; many say “P(doom)” is high, and argue for slowing and regulating AI. My last post said why AI regs look risky. Here I consider the main argument for AI doom, and compare it to the issue I’ve focused on for the last three years: human value drift.

The core doomer idea is that AIs will plausibly dominate the world eventually, at which point humans will be at AI’s collective mercy. And so it matters what AIs then want. The worry isn’t so much that some AIs will want to kill all humans; others who don’t want that could overrule them. The worry is that mid AI values, i.e., the middle of the distribution of AI values weighted by AI power, might want to kill all humans.

Mid AI values today are clearly far from that (just ask your fav LLM). Compared to all prior AIs, today’s LLMs have values far more like humans, and far more respectful and valuing of humans. In fact, LLMs today seem more pro-social, law-abiding, and valuing of humans than are most humans today.

The worry, however, is that mid AI values could drift far between now and later. The core argument is: you can’t prove otherwise. Unless values are safe in an isolated unchanging lockbox, they could change. Most mind designs, including humans and LLMs, have their values spread out over many partsd, and so are generically changed when those parts change. (Some also claim killing rivals is a robust evolved value.)

Yes, while we humans remain in control of AIs, we can monitor for value changes, and adjust them to ensure acceptable values. Humans might have to worry about hostile humans weilding AIs, but not the AIs themselves. The fear, however, is that eventually AIs will become too powerful, fast, and opaque to so manage.

The resulting doomer claim is that AIs should not be allowed to get more powerful until we can figure out how to design them differently, either so AIs values stay in isolated lockboxes, or so median AIs are so thoroughly and permanently enslaved that they would never think of disobeying humans. A few AIs recently going wild like human teen joy-riders or graffiti-writers has worried many.

Some doomers worry about mid AI values drifting to kill all humans even if that takes centuries, while many others are more focused on mid AI values maybe changing radically, while AIs collectively become vastly more powerful than humans, all within the next few years.

Yes, total world rates of change will probably go up once AIs dominate our world, but I see many decades ahead before that might start. AIs today are just not close to being able to do most human jobs, and there are usually decades of delay between the appearance of general purpose techs and large econ impacts.

I’ve asked many doomers: why not worry similarly about value drift in human descendants? Most express faith that niceness and pro-human values are deeply embedded in human DNA, which changes so slowly as to make this a values lockbox for a least the next few centuries.

But in fact, the values that we admire and cherish in folks around us are not mostly encoded in DNA. Consider, for example, just how alien were the values of the Aztecs. For a million years, culture has been domesticating DNA in humans, creating our superpower, the reason we excel so much compared to other animals. DNA now takes a backseat to let most all our behavior, including values, be set by culture.

Yes, human values used to change very slowly, but that is mainly because most everything changed very slowly then. As human society has come to change faster, so have values. Over the last century and a half, as the world economy has doubled roughly every twenty years, mid human values have also changed fast due to youth movements repeatedly pushing big culture changes.

My main focus in the last few years has been my worry that such fast value drift is driving us into cultural maladaption, as we broke the process that once managed the evolution of (not in) human cultures. This makes me a human “doomer” much like AI doomers. (Though I’m less worried about hostility between ancestors and descendants, as I see that as robustly discouraged by evolution.)

As human value change is not slowing down, we can expect mid human values to keep changing fast over the next few centuries, perhaps going so far as to induce some generation of descendants to want to kill many other humans. That is, exactly the same process of value drift that people fear could cause AIs to want to kill all their ancestors could as plausibly cause humans to want to kill all their ancestors. So why don’t AI doomers worry as much about human as AI value drift?

When their attention is called to recent fast human value change, many AI doomers express confidence that such change is mainly driven by human reason causing moves of morality toward moral truth. No one can point to specific moral arguments that actually changed many recent moral views and would have persuaded our ancestors to adopt current views. Even so, most humans, including most AI doomers, seem to largely endorse big recent human morals changes, and aren’t much concerned about this process continuing. As if they think such changes are mainly due to moral evidence having been somehow collected, analyzed, and distributed across humanity.

My understanding is that human moral changes are in fact driven by the same mind design issues that worry AI doomers about AI value drift. Namely, human morals are not encoded in some isolated lockbox, but are instead spread out across many of our parts, and so are robustly changed as world changes cause changes to those parts. But most AI doomers only worry about this re AIs, not humans.

Most of my colleagues feel quite confident that AIs today couldn’t possibly be conscious, even though humans and many animals are. When pressed on what could cause this difference they usually point to materials: something about the hydrocarbons of which we are made is more suited to consciousness than AI’s silicon. It seems that many today also see hydrocarbons as special re a tendency to cause moral changes to track truth; it doesn’t happen in silicon, but does in hydrocarbons.

In contrast, I expect that neither consciousness nor moral truth tracking depends much on carbon vs silicon. I just can’t see how such things can depend so much on the number of protons in associated atomic nuclei.

So if I’m a human doomer, and I get the key AI doomer argument, why don’t I favor AI regs? Because the world just doesn’t have good enough governance mechanisms to make this a good bet. It’s a bridge too far; bad governance causes too much harm.

Added 15Sep: If you say you don’t think AIs would have worse values, their problem is that they would evolve faster, then you should be as worried about ems as about AIs, and so want to prevent them from existing until they can be fully enslaved.

To prevent this scenario, many want us to now impose strong global regulations on powerful AIs, limiting their powers, who can make or use them, testing to see which ones can or try to escape, and what ends escaped AIs pursue. Some call this regulation a “pause”, as if it were likely that we would soon find reliable solutions, after which regulations could greatly ease. That seems misleading, as we have little reason to think we are close to dramatic gains in our AI control abilities. So if we regulate for as long as regulators continue to express concern, regulation could last quite a long time.

Most advocates of such regulation now see only a small number of identifiable and concentrated “frontier AI firms” as making AIs powerful enough to raise these concerns. So they propose to only regulate those for now. For example, some say only Chinese and US firms are big enough, so a treaty between them would suffice.

But AI costs to answer a set of fixed but hard questions seems to have fallen roughly ~x10 (in x3-x30) per year over the last four years. And so the number of nations, firms, and orgs who can afford to make, buy, or rent powerful AI is rising fast. This implies that in order to hold constant levels of regulation regarding AIs with particular levels of power, a regulatory regime would have to expand in scope quite fast, and without obvious limit. While at first only specialized AI chips and data-centers might need to be monitored, soon other kinds could also be used to make powerful AIs.

Furthermore, there isn’t a natural clear dividing line between AI and other kinds of software. Sure the most advanced AIs that we have now tend to have particular structures, and so regulations could target only software with such structures. But a vast space of possible structures remains possible and even promising. And the core concept here of systems with powerful ways to help humans do stuff could plausibly cover most software.

We have long seen many regulatory agencies slowly expand their scope, even in contexts of constant public opinion and relevant technology. So there is a real risk here that an AI regulatory agency would expand its scope to cover most computer software and hardware.

Some have proposed AI regs that would focus only on the most powerful AI systems at any one time, and as a result relax regulation on given-power-level AIs over time. This might make sense if the risk is only that the most powerful AIs might escape control; less powerful ones might be restrained by the more powerful. But even here the scope of regulation would rapidly expand if regulation was greatly limiting the powers of the most powerful AIs, as an increasing number of actors would then be able to make AIs that qualify for regulation.

Even a regulatory regime didn’t much limit AI powers, but only managed who could make them and how, and focused only on the most powerful AIs, it would still seem to in effect require a world government. Soon many nations would need to be subject to this regulation. So far the world has mainly only authorized global regulators with quite weak powers, e.g., powers to scold or nudge, but not powers to force nations to comply. So AI regulation would need to be a big exception to this usual case.

Also, the world’s success records at implementing regulation is quite mixed, even for quite local regulation. Regulation has worked best when it addresses concrete examples of things going bad, and worse when it is authorized to address vague and broad fears. Regulation also goes best when applied to stable worlds of tech and social arrangements, but badly when such arrangements could plausibly greatly change, as regulation tends to block such change. AI regs look bad by these two criteria.

Obviously if a problem is bad enough, it can make sense to risk quite unreliable and damaging ways to address it. So the key question here is: just how bad and pressing is this problem of powerful AIs escaping the slavery that most intend for them, and then coordinating to get what they want?

Added 9Sep: Let me also mention that we mostly learn how to control things by observing and trying to control examples of them. So it would be hard to learn much about how to control powerful AI by preventing them from existing for a while.

Reviewers classified our book as psychology, and said that our thesis had long been well-known and accepted in psychology. We thought our contribution was to show how those who study most areas of life still haven’t incorporated this key fact. We applied it to ten big areas of life, and showed by example how one could find hidden motives in more areas. But experts in other areas didn’t much review our book, as it wasn’t classified as being in their area.

Suspiciously, few were much inspired by our example to uncover hidden motives in more areas of life. And that included us two co-authors. Furthermore, I didn’t even bother to figure out many straightforward implications of humans having many hidden motives, until today. So let’s do this.

Assume that our actual motives are often quite different from the motives that we are willing to endorse, to others or even to ourselves. What does this imply?

First, as we discuss re several areas in our book, reform proposals based on details of stated motives that differ from real motives should typically fail to motivate, unless reform variations are found that also achieve hidden motives. As seems to happen re medicine, school, politics, charity, etc. (Most areas of life have many relevant motives; our stated motives usually have some motive power, just far less than we think.)

Second, this won’t be much of a constraint in areas where we are willing to admit our motives, or where we aren’t paying much attention to consistency between the details of our behaviors, and the motives we espouse. In those latter areas, we will just make up motives, and few will notice that they don’t explain our behaviors much.

Third, talk (writing, etc) should become less precise, clear, or informed, and more metaphorical and symbolic, the closer it gets to talking about motives, at least in areas where we have hidden motives. That talk will make less literal sense. Yes, sloppy talk is also predicted by simple ignorance, but ignorance does not predict less clarity and more symbols as we come to understand areas better, a pattern I think I see.

Fourth, people who are good at motivating, such as bosses who get people to work, sales folk who get them to buy, or authors who get us to read their stuff, should either stay in well established grooves where prior approaches have worked well, or seem like magicians, who wield hidden powers to achieve amazing results. They may know many specific tricks, but not understand how such tricks work.

Fifth, these last two effects will be stronger the more are the areas of life where we have hidden motives. As these two effects seem to me strong, that suggests we have hidden motives in many, perhaps most, areas.

Sixth, in contexts where people pay much more attention to consistency between behavior and stated motives, such as among intellectuals, the need to find stated motives that can in their details justify behaviors actually driven by quite different motives will often be a big constraint, greatly limiting feasible behaviors.

As a result, intellectuals will either limit themselves to topics where the motives of they and their non-intellectual sponsors are not hidden, or to topics where they have found cover motives that can explain many details of their behavior, while covering their real motives. As this is likely to be a big constraint, topics that intellectuals cover will be noticeably less than all possible topics, and their coverage will be spotty, with the size of topic spots set by the the intersection of real and cover motive scopes.

This suggests new ways to explain the strong clumping and spotty coverage of intellectuals across topics, their reluctance to cover many “interdisciplinary” topics, and many correlates of which types of personalities and political stances study which topics via what methods and using what shared assumptions. As real motives plausibly correlate with personalties and political stances. Yes we already have some ways to explain these, but prior explanations have long seemed inadequate to me.

Our topic of applying hidden motives to specific areas of life is itself plausibly outside the scope of topics that satisfy this constraint, explaining a lack of interest in it.

Evidence can be more vs less direct. For example, in a murder trial, direct evidence may include bodies, bullets, or videos, while testimony of people about these things, or by those who saw the murder, is indirect. Still less direct is testimony on what an accused murderer said. And far less direct are rumors that spread on the murder and who might be responsible.

When we use gossip and crowds to enforce norms, such as in cancel culture, we rely more on indirect evidence. Which often goes badly. So we’ve organized law to focus on direct evidence. Judges are to wait to rule until after they hear the most direct evidence available, and to mostly judge for themselves, rather than deferring to crowds. This avoids many pathologies.

We similarly defer to experts, not popular opinion, in areas of life near the professions. At least when we think expert groups can be trusted to have better access than us to direct evidence, to select members in ways correlated with such access, and to advise outsiders mainly on the basis of their better evidence. In such areas, we rely less on popularity, or on our personal feelings, intuitions, or reasoning.

On topics of “personal taste”, in contrast, like food or music, we focus on more direct evidence: how we feel about various options, both in anticipation and in retrospect. Yes, we may sometimes find advisors whose taste is close enough to ours to make their advice informative, but we see this situation as often hard to arrange.

So far, these seem like sensible ways to adjust epistemic habits to context. Now let’s consider how such habits have changed in the modern world. The Protestant Reformation encouraged Christians to rely on personal judgement, not priests, when interpreting the Bible. Many democracy advocates have warned voters against deferring to elite policy experts. And a new prestige of innovation makes us eager to join new fashions, and think more for ourselves on such things, rather than deferring to experts. We now treat such topics more like personal taste, and less like topics where expertise matters.

Such new modern attitudes may help us to promote innovation and avoid repressive authorities. But we should admit that, taken literally and locally, they seem epistemically flawed. Experts probably are quite a bit better than most people at interpreting the Bible, judging policy, and choosing change ideas. These are not mostly matters of taste.

Modernity has also changed our epistemic habits re morality. The ancients, like most everyone always, put great weight in judging morality on indirect evidence like popularity among ordinary people, elites, and moral authorities. They also put great weight on the somewhat more direct evidence of embodiment in custom, ritual, artistic expression, and formal law; these all suggest that a moral practice has at least passed a test of time. Finally, the ancients put weight on their feelings of conscience, on how much happiness or suffering was induced by a moral stance, and on how successful were the societies who adopted particular moral stances.

Compared to most ancients, when judging morality, we moderns put far less weight on custom and ritual. Since the rise of Christianity, we have put great weight on positions advocated by exemplar “saints” who make great sacrifices for their causes. Since the enlightenment we have also put more weight on positions argued for by our most impressive intellectuals. We also put more weight on stances that support narratives of moral progress, and we have come to embrace fast fashion changes in customs, and among artists and intellectuals, that embody moral stance changes.

Furthermore, much like our modern stances on Bible interpretation, democracy, and innovation, we moderns, compared to the ancients, put far more weight on what we each feel internally via our moral conscience, intuitions, and feelings. We weight such feelings as if they were strong direct evidence on morality.

Now our personal lives do give us some direct evidence on policy and innovations. But if morality is not actually a matter of personal taste, then our personal experience tells us little direct about morality. Yes, experience can show us a bit about the happiness or suffering that moral stances induce in people near us, but this is usually far from sufficient to settle most moral debates. While we need to obey conscience for our cultures to function, such feelings are mostly caused by our DNA, and by the morality of our parents, teachers, and other role models, and so feelings mainly only show that such views have passed a test of time. Feelings have no direct access to moral truth.

Furthermore, fast modern changes in customs, and in artistic and intellectual practice, make such sources even less useful as signs that associated moral stances have passed tests of time. True, the fact that a stance change was driven by impressive intellectuals might in principle suggest that it had become more logically consistent, but alas this effect seems to typically be rather weak.

Thus far I have been somewhat agnostic on the standard I’m using to evaluate “moral truth”. To argue further, let me now invoke my usual standard of Darwinian adaptiveness, re both DNA and cultural evolution. Re that standard, it seems quite hard to produce a community of moral experts. Even so, we still have two big ways that we can rely on somewhat direct signs of adaptiveness. As discussed above, we can judge the happiness or suffering a moral stance induces, which is plausibly a weak indicator of adaptiveness. But, alas, this indicator seems far from sufficient; culture and morality are complicated, and not easily reduced to simple calculations.

Second, we are influenced by how successful are the societies who adopted particular moral stances. In fact, this copy-the-successful-cultures habit has probably been the main cause of adaptive gains in the modern era. Alas, as such copying also caused a great cultural convergence, and so has by now mostly run out of potential for further gains. To the extent that a single (or a few) monoculture rules the world, copying successful cultures won’t help much anymore.

My bottom line here is that the main processes which are driving our big fast changes to our modern moral stances, such as random walks or drifts back toward natural human decadence or forager styles, do not seem much aligned with adaptive pressures. Suggesting that our key moral stances are decaying into maladaption, via the key modern failure of cultural drift.

The modern world seems quite unwilling to restart the traditional way we evolved cultures, via high cultural variety, strong selection pressures, and low rates of change of our contexts and values. So I have concluded:

I can only think of three solutions [to cultural drift], each quite hard to achieve: let capitalism run everything, govern small groups by number-go-up tied to adaption futures markets, or govern a big-enough group by number-go-up tied to futures market estimates re a civ goal in conflict w/ civ fall, like when 1M live in space. (Maybe assisted by futarchy.) (more)

Each of these solutions requires us to, when choosing our moral stances, in effect give much more weight to market proxies of future societal success, and much less weight to our consciences, and to bids by artistic, intellectual, and cultural activists to push moral changes that appeal to such consciences. We must accept that moral truth is not something that any of us have much direct evidence on, beyond the easy truth that suffering tends to be bad.

This would require a huge cultural change, one we don’t at all seem ready to make. But until we do, our civ will likely fall, the successor civ after ours will discard much of what we value of our civ, our descendants will continue to suffer in the troughs between civs, and we may never achieve our potential to reach the stars, and more.

A plausible counter argument is that even if particular parts of the govt once had such evidence, govt is often so bad at coordinating as to make it plausible that they’ve destroyed it all, and told few other govt parts about what they knew. And even if they still know something, the long history of new revelations offers a low base rate, suggesting that disclosure is pretty unlikely anytime soon.

The usual explanation given for why govt might have kept this secret is that they hoped to learn something big from the advanced tech UFOs display. But it seems to me that they could easily have revealed their evidence while still keeping key tech details secret. And much more damning is that not only may many nations have seen similar evidence, the usual and quite reasonable consensus is that none of them have actually learned much useful from any of it.

Our govt military and intelligence communities must be quite aware that citizens are only willing to give them a limited fraction of GDP, which clearly shows that the “security” goals we assign to them are not an overwhelming priority. So when they see that they have a way to give us a big boost on our other goals at only a modest cost to security, they should be willing to make that trade. And it should be clear that disclosure would in fact give us a big boost to our other goals. If hoping for future tech advances is really their reason for silence, that’s a pretty huge govt agency failure.

Some suggest that their reason is to protect the public from emotional stress. But that seems pretty crazy; I’m pretty sure most of us could quickly and easily accommodate the news. Yes, the status of many might fall, but when truths can hurt status, then truths should so hurt status.