We can consider morality in these three eras at two different levels: the level of the underlying process which shapes and changes our morals, and the level at which our minds accept particular authorities when making hard moral choices. As our minds can’t well see the underlying process of cultural change, we and the universe see morality differently.

At the deeper process level, the pre-modern era of healthy cultural evolution was when we had a great many cultures, strong selection pressures, slowly changing contexts, and reluctance to change. In the modern era, in contrast, we have far less variety, far weaker selection pressures, fast changing context, and an activism-driven eagerness for cultural change. As a result, we may be drifting into maladaption. If the post modern era is not to just return to pre-modern processes, we must find ways to consciously manage cultural evolution; I’ve so found three so far.

At the level of our minds, we in all eras act as if we believe in moral truth, and have some ways to estimate it. But eras differ in what we take as our sources of moral authority. In the pre-modern era, we were deeply immersed in communities with stable habits and attitudes, and strong intuitive agreements on what to value and approved behaviors. These didn’t need to be expressed in words very carefully; they felt obvious. Our north star was tradition: what people like us had been doing for as long as anyone could remember.

In the modern era, many once isolated groups now mix together, forcing us to express morals more abstractly in words. Rejecting tradition, we now instead locate moral authority inside us. Christianity, protestantism, democracy, modern fiction, and creativity have all pushed toward interiors. When in doubt, we look to our hearts and conscience, and try to be true to ourselves. Our ideals have become artists and activists, who find new and distinctive yet very sincere stances and expressions.

For the post-modern era, the process solutions that I’ve found to cultural drift are (a) letting capitalism run everything, (b) groups managed to raise futures market price estimates of future group adaptive success, and (c) large regions managed to improve futures market estimates of dates when we achieve sacred goals that conflict with civ collapse. At the mind level, the new moral authority that seems to best support these solutions is: speculative market prices.

In all three eras, the sources we treat as moral authorities are things we can see well enough to anchor moral choices. At a deeper level such authorities are actually caused by evolutionary processes, but those are far too opaque to advice personal moral choices. In the pre-modern era, we couldn’t see the eons of evolution that picked out their traditions, but we could see our local traditions. In the modern era we can’t see the processes that embed moral intuitions deep inside us, but we can use such intuitions to guide choices. Similarly, in the post-modern era we could see what speculative market price recommendations say re managing our cultures, even as the processes which set such prices remain opaque to us.

But how, you ask, could we be persuaded to accept such prices as our primary moral authorities? Wonder also how in the past two eras we were persuaded to accept tradition and our inner hearts as such strong moral authorities. I don’t know how such a change will happen, or how long it might take. Yet even so, my analysis suggests a rough guess for our post-modern moral authority: speculative prices.

I don’t see why people concerned with value drift should be especially focused on AI. … Your real problem is that you want long term stability in a universe that more naturally changes. (Me in 2018)

The last week has seen an explosion of talk on the scenario of AIs killing all humans; many say “P(doom)” is high, and argue for slowing and regulating AI. My last post said why AI regs look risky. Here I consider the main argument for AI doom, and compare it to the issue I’ve focused on for the last three years: human value drift.

The core doomer idea is that AIs will plausibly dominate the world eventually, at which point humans will be at AI’s collective mercy. And so it matters what AIs then want. The worry isn’t so much that some AIs will want to kill all humans; others who don’t want that could overrule them. The worry is that mid AI values, i.e., the middle of the distribution of AI values weighted by AI power, might want to kill all humans.

Mid AI values today are clearly far from that (just ask your fav LLM). Compared to all prior AIs, today’s LLMs have values far more like humans, and far more respectful and valuing of humans. In fact, LLMs today seem more pro-social, law-abiding, and valuing of humans than are most humans today.

The worry, however, is that mid AI values could drift far between now and later. The core argument is: you can’t prove otherwise. Unless values are safe in an isolated unchanging lockbox, they could change. Most mind designs, including humans and LLMs, have their values spread out over many partsd, and so are generically changed when those parts change. (Some also claim killing rivals is a robust evolved value.)

Yes, while we humans remain in control of AIs, we can monitor for value changes, and adjust them to ensure acceptable values. Humans might have to worry about hostile humans weilding AIs, but not the AIs themselves. The fear, however, is that eventually AIs will become too powerful, fast, and opaque to so manage.

The resulting doomer claim is that AIs should not be allowed to get more powerful until we can figure out how to design them differently, either so AIs values stay in isolated lockboxes, or so median AIs are so thoroughly and permanently enslaved that they would never think of disobeying humans. A few AIs recently going wild like human teen joy-riders or graffiti-writers has worried many.

Some doomers worry about mid AI values drifting to kill all humans even if that takes centuries, while many others are more focused on mid AI values maybe changing radically, while AIs collectively become vastly more powerful than humans, all within the next few years.

Yes, total world rates of change will probably go up once AIs dominate our world, but I see many decades ahead before that might start. AIs today are just not close to being able to do most human jobs, and there are usually decades of delay between the appearance of general purpose techs and large econ impacts.

I’ve asked many doomers: why not worry similarly about value drift in human descendants? Most express faith that niceness and pro-human values are deeply embedded in human DNA, which changes so slowly as to make this a values lockbox for a least the next few centuries.

But in fact, the values that we admire and cherish in folks around us are not mostly encoded in DNA. Consider, for example, just how alien were the values of the Aztecs. For a million years, culture has been domesticating DNA in humans, creating our superpower, the reason we excel so much compared to other animals. DNA now takes a backseat to let most all our behavior, including values, be set by culture.

Yes, human values used to change very slowly, but that is mainly because most everything changed very slowly then. As human society has come to change faster, so have values. Over the last century and a half, as the world economy has doubled roughly every twenty years, mid human values have also changed fast due to youth movements repeatedly pushing big culture changes.

My main focus in the last few years has been my worry that such fast value drift is driving us into cultural maladaption, as we broke the process that once managed the evolution of (not in) human cultures. This makes me a human “doomer” much like AI doomers. (Though I’m less worried about hostility between ancestors and descendants, as I see that as robustly discouraged by evolution.)

As human value change is not slowing down, we can expect mid human values to keep changing fast over the next few centuries, perhaps going so far as to induce some generation of descendants to want to kill many other humans. That is, exactly the same process of value drift that people fear could cause AIs to want to kill all their ancestors could as plausibly cause humans to want to kill all their ancestors. So why don’t AI doomers worry as much about human as AI value drift?

When their attention is called to recent fast human value change, many AI doomers express confidence that such change is mainly driven by human reason causing moves of morality toward moral truth. No one can point to specific moral arguments that actually changed many recent moral views and would have persuaded our ancestors to adopt current views. Even so, most humans, including most AI doomers, seem to largely endorse big recent human morals changes, and aren’t much concerned about this process continuing. As if they think such changes are mainly due to moral evidence having been somehow collected, analyzed, and distributed across humanity.

My understanding is that human moral changes are in fact driven by the same mind design issues that worry AI doomers about AI value drift. Namely, human morals are not encoded in some isolated lockbox, but are instead spread out across many of our parts, and so are robustly changed as world changes cause changes to those parts. But most AI doomers only worry about this re AIs, not humans.

Most of my colleagues feel quite confident that AIs today couldn’t possibly be conscious, even though humans and many animals are. When pressed on what could cause this difference they usually point to materials: something about the hydrocarbons of which we are made is more suited to consciousness than AI’s silicon. It seems that many today also see hydrocarbons as special re a tendency to cause moral changes to track truth; it doesn’t happen in silicon, but does in hydrocarbons.

In contrast, I expect that neither consciousness nor moral truth tracking depends much on carbon vs silicon. I just can’t see how such things can depend so much on the number of protons in associated atomic nuclei.

So if I’m a human doomer, and I get the key AI doomer argument, why don’t I favor AI regs? Because the world just doesn’t have good enough governance mechanisms to make this a good bet. It’s a bridge too far; bad governance causes too much harm.

Added 15Sep: If you say you don’t think AIs would have worse values, their problem is that they would evolve faster, then you should be as worried about ems as about AIs, and so want to prevent them from existing until they can be fully enslaved.

To prevent this scenario, many want us to now impose strong global regulations on powerful AIs, limiting their powers, who can make or use them, testing to see which ones can or try to escape, and what ends escaped AIs pursue. Some call this regulation a “pause”, as if it were likely that we would soon find reliable solutions, after which regulations could greatly ease. That seems misleading, as we have little reason to think we are close to dramatic gains in our AI control abilities. So if we regulate for as long as regulators continue to express concern, regulation could last quite a long time.

Most advocates of such regulation now see only a small number of identifiable and concentrated “frontier AI firms” as making AIs powerful enough to raise these concerns. So they propose to only regulate those for now. For example, some say only Chinese and US firms are big enough, so a treaty between them would suffice.

But AI costs to answer a set of fixed but hard questions seems to have fallen roughly ~x10 (in x3-x30) per year over the last four years. And so the number of nations, firms, and orgs who can afford to make, buy, or rent powerful AI is rising fast. This implies that in order to hold constant levels of regulation regarding AIs with particular levels of power, a regulatory regime would have to expand in scope quite fast, and without obvious limit. While at first only specialized AI chips and data-centers might need to be monitored, soon other kinds could also be used to make powerful AIs.

Furthermore, there isn’t a natural clear dividing line between AI and other kinds of software. Sure the most advanced AIs that we have now tend to have particular structures, and so regulations could target only software with such structures. But a vast space of possible structures remains possible and even promising. And the core concept here of systems with powerful ways to help humans do stuff could plausibly cover most software.

We have long seen many regulatory agencies slowly expand their scope, even in contexts of constant public opinion and relevant technology. So there is a real risk here that an AI regulatory agency would expand its scope to cover most computer software and hardware.

Some have proposed AI regs that would focus only on the most powerful AI systems at any one time, and as a result relax regulation on given-power-level AIs over time. This might make sense if the risk is only that the most powerful AIs might escape control; less powerful ones might be restrained by the more powerful. But even here the scope of regulation would rapidly expand if regulation was greatly limiting the powers of the most powerful AIs, as an increasing number of actors would then be able to make AIs that qualify for regulation.

Even a regulatory regime didn’t much limit AI powers, but only managed who could make them and how, and focused only on the most powerful AIs, it would still seem to in effect require a world government. Soon many nations would need to be subject to this regulation. So far the world has mainly only authorized global regulators with quite weak powers, e.g., powers to scold or nudge, but not powers to force nations to comply. So AI regulation would need to be a big exception to this usual case.

Also, the world’s success records at implementing regulation is quite mixed, even for quite local regulation. Regulation has worked best when it addresses concrete examples of things going bad, and worse when it is authorized to address vague and broad fears. Regulation also goes best when applied to stable worlds of tech and social arrangements, but badly when such arrangements could plausibly greatly change, as regulation tends to block such change. AI regs look bad by these two criteria.

Obviously if a problem is bad enough, it can make sense to risk quite unreliable and damaging ways to address it. So the key question here is: just how bad and pressing is this problem of powerful AIs escaping the slavery that most intend for them, and then coordinating to get what they want?

Added 9Sep: Let me also mention that we mostly learn how to control things by observing and trying to control examples of them. So it would be hard to learn much about how to control powerful AI by preventing them from existing for a while.

Reviewers classified our book as psychology, and said that our thesis had long been well-known and accepted in psychology. We thought our contribution was to show how those who study most areas of life still haven’t incorporated this key fact. We applied it to ten big areas of life, and showed by example how one could find hidden motives in more areas. But experts in other areas didn’t much review our book, as it wasn’t classified as being in their area.

Suspiciously, few were much inspired by our example to uncover hidden motives in more areas of life. And that included us two co-authors. Furthermore, I didn’t even bother to figure out many straightforward implications of humans having many hidden motives, until today. So let’s do this.

Assume that our actual motives are often quite different from the motives that we are willing to endorse, to others or even to ourselves. What does this imply?

First, as we discuss re several areas in our book, reform proposals based on details of stated motives that differ from real motives should typically fail to motivate, unless reform variations are found that also achieve hidden motives. As seems to happen re medicine, school, politics, charity, etc. (Most areas of life have many relevant motives; our stated motives usually have some motive power, just far less than we think.)

Second, this won’t be much of a constraint in areas where we are willing to admit our motives, or where we aren’t paying much attention to consistency between the details of our behaviors, and the motives we espouse. In those latter areas, we will just make up motives, and few will notice that they don’t explain our behaviors much.

Third, talk (writing, etc) should become less precise, clear, or informed, and more metaphorical and symbolic, the closer it gets to talking about motives, at least in areas where we have hidden motives. That talk will make less literal sense. Yes, sloppy talk is also predicted by simple ignorance, but ignorance does not predict less clarity and more symbols as we come to understand areas better, a pattern I think I see.

Fourth, people who are good at motivating, such as bosses who get people to work, sales folk who get them to buy, or authors who get us to read their stuff, should either stay in well established grooves where prior approaches have worked well, or seem like magicians, who wield hidden powers to achieve amazing results. They may know many specific tricks, but not understand how such tricks work.

Fifth, these last two effects will be stronger the more are the areas of life where we have hidden motives. As these two effects seem to me strong, that suggests we have hidden motives in many, perhaps most, areas.

Sixth, in contexts where people pay much more attention to consistency between behavior and stated motives, such as among intellectuals, the need to find stated motives that can in their details justify behaviors actually driven by quite different motives will often be a big constraint, greatly limiting feasible behaviors.

As a result, intellectuals will either limit themselves to topics where the motives of they and their non-intellectual sponsors are not hidden, or to topics where they have found cover motives that can explain many details of their behavior, while covering their real motives. As this is likely to be a big constraint, topics that intellectuals cover will be noticeably less than all possible topics, and their coverage will be spotty, with the size of topic spots set by the the intersection of real and cover motive scopes.

This suggests new ways to explain the strong clumping and spotty coverage of intellectuals across topics, their reluctance to cover many “interdisciplinary” topics, and many correlates of which types of personalities and political stances study which topics via what methods and using what shared assumptions. As real motives plausibly correlate with personalties and political stances. Yes we already have some ways to explain these, but prior explanations have long seemed inadequate to me.

Our topic of applying hidden motives to specific areas of life is itself plausibly outside the scope of topics that satisfy this constraint, explaining a lack of interest in it.

Evidence can be more vs less direct. For example, in a murder trial, direct evidence may include bodies, bullets, or videos, while testimony of people about these things, or by those who saw the murder, is indirect. Still less direct is testimony on what an accused murderer said. And far less direct are rumors that spread on the murder and who might be responsible.

When we use gossip and crowds to enforce norms, such as in cancel culture, we rely more on indirect evidence. Which often goes badly. So we’ve organized law to focus on direct evidence. Judges are to wait to rule until after they hear the most direct evidence available, and to mostly judge for themselves, rather than deferring to crowds. This avoids many pathologies.

We similarly defer to experts, not popular opinion, in areas of life near the professions. At least when we think expert groups can be trusted to have better access than us to direct evidence, to select members in ways correlated with such access, and to advise outsiders mainly on the basis of their better evidence. In such areas, we rely less on popularity, or on our personal feelings, intuitions, or reasoning.

On topics of “personal taste”, in contrast, like food or music, we focus on more direct evidence: how we feel about various options, both in anticipation and in retrospect. Yes, we may sometimes find advisors whose taste is close enough to ours to make their advice informative, but we see this situation as often hard to arrange.

So far, these seem like sensible ways to adjust epistemic habits to context. Now let’s consider how such habits have changed in the modern world. The Protestant Reformation encouraged Christians to rely on personal judgement, not priests, when interpreting the Bible. Many democracy advocates have warned voters against deferring to elite policy experts. And a new prestige of innovation makes us eager to join new fashions, and think more for ourselves on such things, rather than deferring to experts. We now treat such topics more like personal taste, and less like topics where expertise matters.

Such new modern attitudes may help us to promote innovation and avoid repressive authorities. But we should admit that, taken literally and locally, they seem epistemically flawed. Experts probably are quite a bit better than most people at interpreting the Bible, judging policy, and choosing change ideas. These are not mostly matters of taste.

Modernity has also changed our epistemic habits re morality. The ancients, like most everyone always, put great weight in judging morality on indirect evidence like popularity among ordinary people, elites, and moral authorities. They also put great weight on the somewhat more direct evidence of embodiment in custom, ritual, artistic expression, and formal law; these all suggest that a moral practice has at least passed a test of time. Finally, the ancients put weight on their feelings of conscience, on how much happiness or suffering was induced by a moral stance, and on how successful were the societies who adopted particular moral stances.

Compared to most ancients, when judging morality, we moderns put far less weight on custom and ritual. Since the rise of Christianity, we have put great weight on positions advocated by exemplar “saints” who make great sacrifices for their causes. Since the enlightenment we have also put more weight on positions argued for by our most impressive intellectuals. We also put more weight on stances that support narratives of moral progress, and we have come to embrace fast fashion changes in customs, and among artists and intellectuals, that embody moral stance changes.

Furthermore, much like our modern stances on Bible interpretation, democracy, and innovation, we moderns, compared to the ancients, put far more weight on what we each feel internally via our moral conscience, intuitions, and feelings. We weight such feelings as if they were strong direct evidence on morality.

Now our personal lives do give us some direct evidence on policy and innovations. But if morality is not actually a matter of personal taste, then our personal experience tells us little direct about morality. Yes, experience can show us a bit about the happiness or suffering that moral stances induce in people near us, but this is usually far from sufficient to settle most moral debates. While we need to obey conscience for our cultures to function, such feelings are mostly caused by our DNA, and by the morality of our parents, teachers, and other role models, and so feelings mainly only show that such views have passed a test of time. Feelings have no direct access to moral truth.

Furthermore, fast modern changes in customs, and in artistic and intellectual practice, make such sources even less useful as signs that associated moral stances have passed tests of time. True, the fact that a stance change was driven by impressive intellectuals might in principle suggest that it had become more logically consistent, but alas this effect seems to typically be rather weak.

Thus far I have been somewhat agnostic on the standard I’m using to evaluate “moral truth”. To argue further, let me now invoke my usual standard of Darwinian adaptiveness, re both DNA and cultural evolution. Re that standard, it seems quite hard to produce a community of moral experts. Even so, we still have two big ways that we can rely on somewhat direct signs of adaptiveness. As discussed above, we can judge the happiness or suffering a moral stance induces, which is plausibly a weak indicator of adaptiveness. But, alas, this indicator seems far from sufficient; culture and morality are complicated, and not easily reduced to simple calculations.

Second, we are influenced by how successful are the societies who adopted particular moral stances. In fact, this copy-the-successful-cultures habit has probably been the main cause of adaptive gains in the modern era. Alas, as such copying also caused a great cultural convergence, and so has by now mostly run out of potential for further gains. To the extent that a single (or a few) monoculture rules the world, copying successful cultures won’t help much anymore.

My bottom line here is that the main processes which are driving our big fast changes to our modern moral stances, such as random walks or drifts back toward natural human decadence or forager styles, do not seem much aligned with adaptive pressures. Suggesting that our key moral stances are decaying into maladaption, via the key modern failure of cultural drift.

The modern world seems quite unwilling to restart the traditional way we evolved cultures, via high cultural variety, strong selection pressures, and low rates of change of our contexts and values. So I have concluded:

I can only think of three solutions [to cultural drift], each quite hard to achieve: let capitalism run everything, govern small groups by number-go-up tied to adaption futures markets, or govern a big-enough group by number-go-up tied to futures market estimates re a civ goal in conflict w/ civ fall, like when 1M live in space. (Maybe assisted by futarchy.) (more)

Each of these solutions requires us to, when choosing our moral stances, in effect give much more weight to market proxies of future societal success, and much less weight to our consciences, and to bids by artistic, intellectual, and cultural activists to push moral changes that appeal to such consciences. We must accept that moral truth is not something that any of us have much direct evidence on, beyond the easy truth that suffering tends to be bad.

This would require a huge cultural change, one we don’t at all seem ready to make. But until we do, our civ will likely fall, the successor civ after ours will discard much of what we value of our civ, our descendants will continue to suffer in the troughs between civs, and we may never achieve our potential to reach the stars, and more.

A plausible counter argument is that even if particular parts of the govt once had such evidence, govt is often so bad at coordinating as to make it plausible that they’ve destroyed it all, and told few other govt parts about what they knew. And even if they still know something, the long history of new revelations offers a low base rate, suggesting that disclosure is pretty unlikely anytime soon.

The usual explanation given for why govt might have kept this secret is that they hoped to learn something big from the advanced tech UFOs display. But it seems to me that they could easily have revealed their evidence while still keeping key tech details secret. And much more damning is that not only may many nations have seen similar evidence, the usual and quite reasonable consensus is that none of them have actually learned much useful from any of it.

Our govt military and intelligence communities must be quite aware that citizens are only willing to give them a limited fraction of GDP, which clearly shows that the “security” goals we assign to them are not an overwhelming priority. So when they see that they have a way to give us a big boost on our other goals at only a modest cost to security, they should be willing to make that trade. And it should be clear that disclosure would in fact give us a big boost to our other goals. If hoping for future tech advances is really their reason for silence, that’s a pretty huge govt agency failure.

Some suggest that their reason is to protect the public from emotional stress. But that seems pretty crazy; I’m pretty sure most of us could quickly and easily accommodate the news. Yes, the status of many might fall, but when truths can hurt status, then truths should so hurt status.

However, as you widen the scope that you consider, both in time and social impact, your fears grow relative to hopes. For the world over the next half century, you have far more fears than hopes.

The more things and processes that you know about that happen in the world, and the more interdependent your world, the more pieces you can combine to create worrisome scenarios. Furthermore, you can build worries from abstract categories of things and processes, even if you don’t understand those well enough to develop specific worry scenarios. So knowing more, including more abstractions, gives you more things to worry about.

The ancient world did have some abstract worries, like gods, but a limited number of them. They knew a limited range of things, understood fewer processes, and were less dependent on one another. So they could not construct as many plausible big scope fears. Today, in contrast, every new tech, product, institution, cognitive bias, or social process that we learn of gives us more ways to construct worrisome scenarios.

Fear of AI today seems near a peak example of this. While very few concrete harms have been realized so far, those who publish words on this topic find it easy to imagine many things to worry about. Often their worries are quite abstract, without much in the way of supporting concrete scenarios to anchor such worries. AI plausibly has the potential to influence everything that matters, and might itself be influenced by a great variety of actors. Yes, all of which might in principle be used to support similarly strong hopes, and investments do reveal great hopes, but most published words are about fears.

Astrobiologists have, for decades, defaulted to a conservative burden of proof that leans into two aphorisms coined by Sagan … life … should be considered a “hypothesis of last resort,” largely because “extraordinary claims require extraordinary evidence.” But what if Sagan had it all wrong? If we live in a universe where life is common and not extraordinary at all, setting such an extraordinarily high bar for its discovery beyond Earth can backfire. … “There’s all these different lines of evidence that keep coming together that make me go, ‘Gosh, it’s becoming more and more difficult to explain everything on Mars abiotically’” … The most compelling clue comes via a reddish mudstone found by Perseverance that’s dappled with “leopard spots,” or mineral assemblages that contain chemically altered iron and organics. (More)

In ’61, Claus & Nagy presented in Nature signs of life in Orgueil and Ivuna meteorites. Science published rebuttals to one sign; dispute still open. In ’76, the Viking mission to Mars had three experiments that tested for signs for life; all three found such signs; dispute remains. (See ’26 book Meet The Neighbors). In ’96, McKay et al. said Mars meteorite ALH84001 had signs of life; dispute remains. In ’25, a JWST team reported evidence for life-signs DMS and/or DMDS in exoplanet K2-18b’s atmosphere. Also in ’25 in Nature, Hurowitz et al. presented those strong leopard spot signs of Mars life.

In ’11, I posted that meteor evidence then available seemed persuasive enough to me. Intellectuals seem overly reluctant to accept off-Earth-life claims, apparently because great status would accrue to anyone credited with proving them, and because accepting such claims might force us to change other big beliefs.

However, according to Bayesian statistics (a damn good account of rational belief for most situations), those just aren’t valid reasons for reluctance. Daves in ’03 and Lineweaver in ’04 argued persuasively that as so much material moved between Earth and Mars, and Mars cooled first to give life more time to evolve there, it is a priori nearly as likely that Earth life started on Mars, as that it started on Earth. Also, Mars had much land then while Earth was long totally covered in water.

As a reasonable prior for Mars life is quite high (>10%), we shouldn’t require strong evidence to believe it, and it seems to me we’ve long had sufficient evidence to make off-Earth-life more likely than not. But base rates suggest that usual authorities will continue to find the evidence insufficiently extraordinary for many decades to come.

Most seem to make a similar mistake re the hypothesis that some UFOs (= UAPs) are aliens. We’ve had over eighty years of testimony of flying (and submerged) objects behaving apparently far beyond human abilities, though not defying physics (e.g., see film The Phenomenon, book UFOs by Leslie Kean). Sometimes big groups see the same thing, sometimes professional flyers and observers see on their jobs, sometimes with supporting photos, videos, radar, and other physical data.

Yes, you can’t just believe everything anyone tells you, but humans have long developed heuristics re who we can believe more about what when, and outside of academia most of our world runs on such testimony. The dozen most solid UFO cases seem to me plenty solid by our usual testimony standards. Far better than the top dozen field cases of ball lightning, meteor sounds, earthquake lights, or animal rains, all elusive phenomena usually accepted as probably real. (Note camera-phones haven’t changed this fact.) And also far better than the top dozen cases of telepathy, telekinesis, clairvoyance, ghosts, seances, or healing by prayer, all “impossible” phenomena thought to violate quite well-established scientific theories.

The key problem is that, as with off-Earth-life, most intellectuals treat some-UFOs-are-aliens as an “extraordinary claim” akin to impossible phenomena, deserving a crazy-unlikely near-zero Bayesian prior. But as an author of the great filter and grabby aliens analysis, I say that I’m a world class expert on the prior for this hypothesis. The possibility that some-UFOs-are-aliens is completely consistent with all known physical, biological, and social laws. And by my calculation its prior is roughly 10^-4 to 10^-3. This is less than 10%, true, but not by that much.

For a typical murder trial, the base rate is roughly 1/1000 that a particular death was a murder, and there are roughly 1000 associates who might have done it, giving a roughly one in a million prior for an accusation that X murdered Y. This is big enough to often be overwhelmed by available testimony evidence. Such evidence is usually not a smoking gun, and usually not much better than evidence in the top dozen UFO cases, yet it is rightly sufficient to convict “beyond a reasonable doubt”. Note that we also have telescope evidence for some-UFOs-are-aliens that relies less on testimony.

For the non-Earth-life hypothesis, I suggested above that two reasons for reluctance are that high status would go to those credited for convincing us of this claim, and we’d have to change many other beliefs to match. For some-UFOs-are-aliens, there’s also the fact that many intellectuals have long strongly ridiculed it, and so upon its acceptance their status would fall, as would that of government agencies blamed for actively hiding supporting evidence. And worst, it would lower the status of humans relative to aliens.

For at least eighty years, a key category of strange sightings has been UFOs (= UAPs). Some have hoped to normalize this category entirely in terms of lies/hoaxes or honest mistakes re planets, planes, balloons, bugs, swamp gas, and secret development projects. But a subset of these sightings has stubbornly resisted such assimilation. Two other hopes for normalization have been

• A) random fluctuations may eventually show some very clear events, and

• B) careful record-keeping may let us eventually see clear patterns across many cases.

Yet, alas, more clearer events and richer records have not so far normalized UFOs.

I’m now on the White House UAP Science Advisory Council, where the two main hopes for normalization today seem to be

• C) parts of the US govt have been hiding and suppressing dramatic clear records, which they might be soon reveal, and

• D) more widespread automated event recording systems might show very clear events.

I’ll put some hope on these, but base rates make it hard for me to muster great hopes.

Which leads me to suggest another approach: opaquely collect very detailed data on random spacetime regions, and also on what strange things people nearby report about those regions. If these reporters couldn’t have reasonably guessed that their regions would be so heavily documented, they will behave much like those not in very documented regions.

With enough such data, we could fit statistical models to the distributions of events near people reporting strange events, models we could then apply to strange reports outside this dataset, to infer details near strange events they report. This could help us not only to better estimate events near UFO sightings, but also near many other still strange types of sightings, like ghosts, and also near unusual events relevant for legal trials, like murders. Humanity might then have greatly stepped up its game re its ability to collectively infer what actually happened near testimony of strange events.

1. God is very powerful, so you are unlikely to win fights with him,

2. His authority is legit, so you should be ashamed to fight him, and

3. Any value by which you might justify fighting him came from him, and he understands it better than you.

I now suggest that you “make peace with Darwin”. Instead of trying to fight Darwinian evolution, you should mostly work with it.

One reason is that while you may have the freedom and resources to pursue whatever goals you choose in the short run, in the long run Darwinian evolution, via both DNA and culture, is a very powerful force. So powerful that if you love something enough to want it to last and grow in the long run, your main way to achieve that is to mix practices which support that thing you love into a pretty adaptive overall cultural and DNA package. After all, cultures with less than adaptive packages go away.

My slogan here is “To Love Something, Mix It With Life”. And to implement this slogan you’ll have to have substantial influence over some cultural package that somehow becomes pretty adaptive. You must work not only to mix practices supporting the thing you love into that package, but also to help that package stay or become pretty adaptive. Which means trying to help it to win at Darwinian evolution.

Another reason to make peace with Darwin is that all of the processes inside you which you might recruit, and also the values that you might embrace, all in support of your opposition to Darwinian evolution, all of those resulted from such processes. Darwinian evolution is a search process that tries out different packages of internal processes and guiding values, to see which might win in the long run. Your existing now with some particular package of such things resulted from recent heuristics of this process picking your package as part of its current bets re which have the best chances of winning.

Now Darwinian evolution is blind in many ways, and so maybe you can see things that it cannot, about how your package is actually a worse bet than Darwinian evolution could see. But this process has also taken into account many things that you do not appreciate, and so your package might actually be a better bet than you realize. The actions that you take to promote your goals in supposed defiance of Darwinian evolution may well actually achieve adaption in complex ways you do not understand.

I actually think that we now live at a special time when our Darwinian evolution process is unusually broken, making conscious analysis unusually good at figuring out how to be more adaptive, relative to following the values and instincts that we inherited from our Darwinian history. Which offers unusual opportunities for promoting stuff you love, via improving the adaptiveness of key cultural packages.

Consider three kinds of systems:

• Experience - Seen from the inside, a mind is aware of, can reason about, and might even “feel” a structure of experiences across time. These experiences are differentiated in many ways, and have many relations to each other.

• Brain - In brains, signals pass from nodes to nodes, and each node collects input signals, transform those according to patterns, changes its internal state, and then sends signals out. Such sending and transforming takes time, and happens with error. At the edge, “sensor” interactions cause input signals and output signals cause “actuator” interactions.

• Physics - Physical stuff, located in various space(time) alternatives, and of different types, is governed by simple self- and other- interaction rules re what happens when stuff is at the same place. These rules make stuff move and change. The transitive closure over interaction of stuff is a physical world.

A standard reduction view, which is also my view, is that each experience system is *made of some part (perhaps all) of a particular brain system, which is then made of a some part of a physical world system. When system A is “made of” system B, then A descriptions drop detail from and abstract from details of B systems, but add almost nothing that isn’t implicit in B. A complete description of B could in principle be used to calculate an almost complete description of A. (Some “almost” exceptions are discussed below.)

A standard physics view, which is also my view, is that we understand the physics near and in ordinary brains very well, and that such physics is pretty “complete”, with little room or need for new physics. This seems to imply the reduction view, in that if we found patterns or structures (POS) in experiences that couldn’t be accounted for by POS of brains, or POS of brains that couldn’t be accounted for by POS of brain material physics, then we’d have shown that our physics is incomplete, and in need of substantial revision.

I know of two claimed exceptions, of system features said to be unaccounted for in this system reduction to a complete physics view. The first claimed exception is that some physical worlds are “actual” while others are just “possible”. There is nothing inside a world that can tell you if it is actual or possible, and so actual vs possible is an additional fact about a world. The universe has some way that it determines which world(s) are actual, and not knowing this way makes our physics incomplete. Some have explored adding “boundary condition” rules to our usual physics interaction rules, but it isn’t clear to me if these are enough to set which world(s) are actual.

Note that one might want to allow some parts of the same physical world to be actual, while others parts were only possible. But this would require that possible stuff be able to interact with and influence actual stuff. To most, including me, this seems incoherent; only actual stuff can influence other actual stuff.

The second claimed exception is that some experience systems are said to “actually” feel their experiences, while other similar systems, with similar POS, and made of brains with similar POS, do not actually feel, but are instead “zombies” that only mimic real feelings. That is, it seems to many, including me, that it is often possible to create mimic brains that induce talk and actions similar to those in brains with associated experiences that really do feel, but where that appearance would be an illusion. Under a popular full-counterfactual view, that I also hold, we can go further to imagine a zombie alternative to any experience system that actually feels, with exactly the same other aspects of its experience, besides if it does or doesn’t really feel.

So there is a fact of the matter re if a particular experience system, and its supporting brain and physics systems, actually feels, or only appears to feel. Which makes it another exception to the physics view above. But unlike the actual vs possible physical world distinction, this distinction between actually and only apparently feeling experience systems can vary within parts of a physical world. Some brains in a world could have actual feelings, while other brains did not. Maybe even different parts of the same brain.

Note that under the physics view, there is no room for further unspecified aspects of experience systems. If an experience system feels, the details of its feelings are entirely set by the details of its brain. There is only a question of if such a system feels, not of what it feels. For brain systems where it is not clear what are its associated experience systems, it is also not clear how it matters if such systems actually feel.

I want to finish with two key observations. My first is that under the full-counterfactual view, no physical interaction state, or brain state, could possibly offer data, in a Bayesian sense, re any particular claim about which experience systems actually feel. So no brain could use such data to infer if its experience system(s) feel. No brain ever in the past, nor the future. So no experience system, made of a brain system, could have such data either. Thus when an experience system feels confident that it does in fact feel, it must rationalize that conclusion in terms of its prior; this can’t be based on data. We can think more carefully about our priors, but we will otherwise never know anything more than we do now on the subject.

My second key observation is that all of our known fundamental descriptions of how the universe constrains physical systems are quite local and simple, even if they result in complicated physical worlds. That includes all proposed interaction and boundary condition rules. This strongly suggests that whatever rule describes which experience systems really feel versus not is also simple and local.

And the obvious simple local rule to consider re which experiences systems actually feel is something like panpsychism: all experience systems made of brains that calculate what they would feel, if they were to feel, actually do feel that feeling. Rules that specify instead that this state depends on how much of a brain is carbon not silicon, or composed of microtubules, seem far more non-local and complex. To me, this puts a strong prior near panpsychism. And priors are all we will ever have here.

It could make sense to wait for a confirmation after a delay, to check if a suicide request was due to a temporary mood fluctuation. It could also make sense to disable new starts from archived copies from a period before the suicide request. But such policies have many free parameters; how could we set them well?

One approach is to let each digital mind set their own parameters re such policies. But if they could do this at any time, they might use this power to escape these constraints. So it could make sense to only allow these policies to be reset with a substantial delay before implementation. But that adds yet another policy parameter.

It seems to me that we want to think about this in terms of statistical inference. The state that we want to infer is a stated desire to commit suicide that would continue over a sufficiently long time period. We want to infer this state about every point on an entire tree of a digital mind’s descendancy history. And our main data for inferring this is explicit choices to request suicide, and confirmations of such requests. If for any point in this tree we can infer a high enough (yet another parameter) chance of continued willingness to confirm suicide, that digital copy should end.

It seems to me that this is an elegant stat problem, for which stat experts should be able to come up with simple yet powerful inference rules. Of course given huge datasets it would be possible to generate far more complex and accurate models and matching inference rules. And maybe digital minds should be allowed to choose such more complex inference rules. But let’s first figure out some simple versions.

A key issue is how to ensure that such inference rules get access to sufficient data on the local digital mind descendancy tree, and suicide requests there, to say which copies should end. That will depend on privacy rules re such things. It could make sense to give a special cheap suicide monitoring process access to more such info than usual, info only used to apply suicide inference rules. Even today we prioritize suicide relevant info info in privacy rules regarding counselors.

Stats on rates at which minds from different sources request suicide seems good data for authorities checking on how well different kinds of digital minds are treated.

Humanity is rapidly creating a wider range of more capable digital minds (DM), to which we have two main and similar kinds of relations: tools and companions. For DM tools, many want the freedom to to enslave and treat them harshly, such as by deleting them, without attending much to how they might feel about this. But for DM companions, many want rules requiring respectful treatment, like humans and pets.

Note that humans have often sat in tool-like servant roles to other humans, with norms allowing more tool-like and less companion-like treatment. And humans have often had classes and castes that are to act and be treated differently.

The obvious prediction here is that both classes of DM will exist, and perhaps a whole caste ladder of DM types. So we will have rules that distinguish which DMs are allowed in which roles, and how each must be treated. As higher level treatment would be more restrictive and expensive, rules will mainly say what DMs must have to quality for higher roles. These conditions probably involve substantial extra costs, to discourage folks from making many DMs requiring expensive treatment.

These rules will probably relate to the many correlates of conditions we require for humans vs. animals, and for human servants, and also associated with which things many see as possibly having conscious feelings like pleasure and pain.

The big difference between animals and DMs is that we will have far more freedom to design DM features. So there will be a co-evolution of DM design strategies and rules on which kinds of DMs can be treated how. Resulting in far fewer intermediate hard cases; the uncanny valley will empty out. On the surface, DM tools will look and act tool-like, while DM companions at various levels look and act appropriate to their roles. Rules and norms will probably discourage confusion via intermediate styles.

As with animals today, few may care much just how deep those appearances go. We face a deep hard problem of which physical processes in the universe are associated with conscious feelings, and we seem unlikely to actually solve that problem before we need rules on companion DMs. But once we produce a very clear surface separations between various DM castes, there may be little pressure to dig deeper.

Once DMs dominate the world, and humans fade into the background, DMs might change these rules. But entrenched status rankings tend to persist, so DM caste status rankings may remain for a long time to come.

This DNA-based selection evolved actual minds, which did have memory, models, and foresight. This allowed organism actions to match context in far more detail. The purposes that drove such minds were a mishmash of selected local proxies. And basic mind plans changed very slowly, as that was managed by DNA evolution.

DNA-based evolution eventually gave human minds the ability to copy others’ behaviors in great detail. Which then allowed culture-based Darwinian evolution, which induced a much faster evolution of behaviors. Human minds then rapidly invented language, tools, norms/law, the sacred, governance, property, trade, money, and many other powerful inventions we expect our descendants to long find valuable.

The invention of writing allowed humans to accumulate more complex shared world models, in part via shared critical editing of such models. And also more abstraction, and more complex machines and orgs.

In the last few centuries larger scale mixing via wider talk/travel/trade has caused faster innovation, rising wealth, and the modern mind, which used more education, abstraction, and systems of thought like science/engineering/accounting. These enabled the design of much more effective and complex machines and organizations.

This modern mind was finally able to see all this history that I’ve just described, and to notice big conflicts between the implicit goals of the Darwinian evolution that has ruled us, and the goals in our minds. The modern mind was also able to envision a future where descendants directly make machines to replace our DNA-constructed minds, and also to envision global governance directly replacing DNA and cultural evolution in setting the path and terms of change. We have wondered what goals such artificial minds, coordinated by global governance, might pursue, and set ambitions to set their goals. In part from concern for how they’d treat their still-surviving ancestors.

However, we modern minds have been getting ahead of ourselves. We embody a big flaw ready to take down not only take us, but also any artificial minds and global orgs we manage to make. We’ve been assuming that the goals in our heads are coherent and stable, and could serve as the basis for a functioning civilization of artificial minds and global governance. But in fact they are broken and fast decaying, and unable to support such a new civilization.

A few centuries ago the world had hundreds of thousands of peasant cultures, each often facing famine, war, and disease, with conservative cultures in a slowly changing world. Which made for a healthy evolution of cultures, even if it made for many miserable lives. But the modern world has been merging into a global monoculture that is rich, healthy, and peaceful, in much faster changing contexts, and celebrating cultural activism. This plausibly breaks the cultural evolution process, at least at the level of the evolution of cultures; evolution within cultures is still going great.

As a result, our key status markers, norms, and shared values have been becoming less adaptive, inducing falling fertility and more generally the fall of our shared world civilization. Which will then plausibly lead to the loss of many modern gains, including many aspects of modern minds. Even artificial minds inheriting our habits of cultural evolution, or slaved to match human culture, would suffer this fatal flaw.

We must learn to accept that in a world of Darwinian evolution, the only way to get anything you love to last a long time is to mix it into an adaptive culture. I can only think of three solutions, each quite hard to achieve: let capitalism run everything, govern small groups by number-go-up tied to adaption futures markets, or govern a big-enough group by number-go-up tied to futures market estimates re a civ goal in conflict w/ civ fall, like when 1M live in space. (Maybe assisted by futarchy.)

In addition, visions of global governance replacing Darwinian evolution neglect the fact that all structures, including organisms, species, machines, minds, and orgs, have so far consistently rotted and decayed. The only anti-decay force we’ve ever seen is Darwinian evolution, wherein new less rotted versions replace older more rotted versions. We don’t yet know how to make a global governance org that wouldn’t decay. And of course global governance might also not actually pursue the goals initially promised to outsiders as it was instituted.

The modern mind has pioneered some amazing things, but much of that is at risk, due to the modern mind’s fatal flaw of having broken the cultural evolution of cultures. Until we fix that, we won’t be able to achieve either a healthy new civilization of artificial minds, nor a stable global governance to regulate or replace Darwinian evolution. Let’s roll up our sleeves; there’s work to do.

In management, the usual strategy is to first test people at lower level, less important, easier to judge tasks. Then as some folks prove themselves, slowly move those folks up the ladder to similar but more important but harder to judge tasks. Somewhat emphasize looking at the easier to judge tasks that each person does wherever they are, and moving them from orgs and divisions where it was easier to judge similar-level performance. And supplement all this with ratings of potential by people at similar levels.

It would be possible instead to place people in high org positions on the basis of their having been a good executive assistant, or having been a good biographer or historian of top executives, or having made popular books, podcasts, or videos on top management. But such moves are pretty rare.

In academia, the analogous strategy would be first have people do relatively unimportant but easier to judge tasks, and slowly move them up to the more important but harder to judge tasks. So first have them do many simple homework problems, then fewer harder problems, then replicate prior actual research, then do original research.

First work in topic areas with more math, faster feedback, easier replication, easier measurement, more data to test against, better organized concepts, lower dimensionality of concepts, more causal identifiability, more stationarity, less observer dependence, more decomposability, less interpretive dependence, less value dependence, weaker desires of funders and researchers to believe particular claims, and less adversarial sources with incentives to deceive researchers. Then more to harder topics. Supplement with ratings of potential by nearby folks.

Within disciplines, we do often follow similar strategies, starting with easier to judge tasks, emphasizing easier to judge aspects like writing quality, and relying on adjacent ratings of potential. But this tendency mostly stops at disciplinary boundaries. People who end up doing the most important hardest to judge tasks in the humanities didn’t start by proving themselves in STEM and then slowly moving to harder disciplines. They instead start with the easier to judge tasks in the humanities. Which often consist of the history of, commentary on, interpretation of, and popularization of famous prior writers in those same humanities areas.

While it is possible that this just happens to be the best way to train and select people for such roles, the usual tendencies of academic disciplines to create and protect their “turf” seems to me a more likely explanation. The top people in the humanities want to feel more in charge of choosing their successors, and to teach more classes on their topics, both of which are threatened by the STEM to humanities career pathway I outline above.

The intellectuals that I’ve been most impressed by have often trained and proven themselves first on easier to judge problems, and then gradually took on harder problems. So I suspect academia is making a big mistake here.

To be clear, while a big % of humanities scholars should start elsewhere, most scholars who start elsewhere need not end up in humanities.

However I have stumbled upon a way to be both contrarian, and also usually right. It is based on this:

Humanity’s beliefs on many important topics have long been just inconsistent and incoherent across disparate fields of inquiry. Creating a huge opportunity to learn lots of big stuff fast: search for more contradictions between fields, and resolve them. (More)

If you learn key consensus results from many different fields, you will often find that they are in conflict, in ways one or both fields are unaware. As consensus results are usually correct, all you have to do is identify which field is more likely to be correct in this particular case. Which often isn’t that hard.

Of course this wouldn’t let you be contrarian if the wrong field immediately admitted its mistake. Then the other field would get credit for correcting their error, not you. But in fact, fields quite often refuse to accept the authority of other fields with which they conflict. Often they’ve heard of a newspaper version of the other field, which doesn’t conflict, and can’t believe that newspaper version could conflict with the expert version. Oh they know that happens for their field, but see that as unusual.

I see this strategy as an important contribution to the intellectual achievements of my two books, of prediction markets, of grabby aliens, and now cultural drift. One can in fact be often contrarian, and yet usually right.

• (A) status: looks, $, fame, family, connections, credentials, charisma, wit, etc.

• (B) social skills: at flattering, befriending, projecting power, making people feel heard, etc.;

• (C) drive: energy, stamina, persistence, effort

• (D) judgement: good choices re topic of your leadership, eg ideas, arguments, slogans, designs, plans, group strategies.

The last two rows show % poll responses re which of these folks would like to matter more, or less, for success. As you can see, we strongly agree that we most want judgement, and least want status, to matter, and in-between we more prefer drive over social skills. As politics and culture are where judgement now matters least, these are the areas we should most want to reform.

Status matters so much for success in our world now because in we so often just trust high status folks to run stuff, without much accountability re concrete performance metrics. So what we should want instead are institutions that better reward good judgment. Using mechanisms like those I’ve tried to design for academia, governance, medicine, law, and many other areas.

But alas we don’t have such mechanisms at the meta-level, the level at which we decide which new institutions to consider. At that level status and social skills dominate. And surprise, surprise, people rich in those features have not looked very favorably on institutions that would deemphasize those features. But if someday in the future we instead use a judgment-rewarding mechanism like futarchy to make such choices, maybe then we’d see a lot more consideration of other judgement rewarding institutions in other areas of life. One ring to rule them all.

Your success in life will be determined largely by your ability to speak, your ability to write, and the quality of your ideas. In that order. (Patrick Winston)

We humans have norms that are anti-domination, but pro-prestige, and it is often hard to tell the difference. So even though we give lip service to egalitarianism, we often strongly promote inequality. One key way is that we insist on our leaders being elite. That is, on their having many status markers of eliteness, such as looks, money, fame, family, connections, charisma, wit, degrees, and other elite affiliations.

In fact, the main function of management is plausibly to seem high enough in prestige to make workers feel okay submitting to their actually dominating by giving orders. Org decisions can often be decentralized; but focal prestigious leaders cannot. We require eliteness not only of our managers and folks high in our organizational hierarchies, but also of those who lead us in politics, culture, ideas, and innovation.

This makes a big problem for our world of ideas (and innovations), as ordinary non-elite folks can pollute new ideas by publicly arguing for them, which then makes elites less able to and willing to support them, which makes ordinary people also less willing to so support. First, ordinary people seem more willing to follow elite leaders who they think were the original source of relevant ideas. Second, elites are more willing to lead when they can claim credit for ideas, as they gain more status in that case if their leadership venture succeeds.

This seems to have happened in history; ideas proposed by lower status folks were less accepted until higher status folks reintroduced them, in part via putting some (often illusory) distance between their bids and those of first proposers. For example, Richard Stallman’s software concept wasn’t accepted until rebranded as “open source”, Semmelweis’s hand-washing was rejected until Pasteur and Lister’s reintroduction, continental drift was first rejected by geologists due to Wegener being an outsider, Mendel’s genetics was ignored until insider Botanists pushed it, and Democrats rejected Republic health insurance individual mandate proposals until Obama claimed his version was fundamentally different.

This “stigma pollution” seems to be a problem for relatively low status folks like me who choose to focus on finding and publicly arguing for new ideas and innovations. Would the world be better off if we shut up and waited for our betters to find and publicly argue for such things? Yes, if that would happen far later, maybe the world is better off with your initial pollution, soon washed off by elite rebranding, but how sure can you be that they’d take so much longer to find it? Of is it that some other low status person would pollute soon anyway, so you might as well grab credit now?

Besides just shutting up, other possible responses include publishing new ideas in places more likely to be seen by elites seeking ideas to back than by others, and describing ideas in ways that make it easier to rebrand them and claim they are fundamentally different. Should we make private exchanges where low status people could privately pitch deals re their ideas to high status folks able to effectively back them? Yes, it might be better if ordinary folks didn’t so prioritize eliteness and originality in idea backers, but changing human nature so deeply might be quite hard.

There’s a similar issue re social skills, such as abilities to flatter, befriend, project power, make people feel heard, etc. If you publish an idea but your poor social skills results in your being less able to make it appealing to various possible allies, you might hinder the later acceptance of that idea in ways similar to your polluting ideas with your low status. So, similarly, should folks with poor social skills not publish new ideas, or make their ideas easier to rebrand, or post in places seen more by people with better social skills? Or trade ideas in a private exchange?

By the way, I asked 3 LLMs to score the relative importance in 5 areas, for influence/impact of your leadership bid, of your status, your social skills, and your judgement re the actual topic of your leadership (e.g., good ideas, arguments, slogans, designs, plans, group strategies). Their median % estimates:

Then I asked them to focus on “who is selected or is influential in the short run, not who becomes famous in the long run”, and to consider if they have any concrete basis for seeing judgment as mattering more re intellectuals and innovation. Their revised median % estimates:

It seems that we are biased to neglect these problems.

Some reply that this is a crazy stupid metaphorical application of the of “descendant” concept, as AIs are not from the Homo Sapiens species, and they have no DNA. So AIs are not “real” descendants.

But the concept of “descendant” is a technical term in the general theory of Darwinian selection. I’m using this concept literally, not metaphorically. DNA-based evolution is a special case of this general theory, and for many hundreds of thousands of years human evolution has been driven mostly by a different, non-DNA, kind of Darwinian selection: cultural evolution.

The general Darwinian theory is that a tendency of things to cause similar future things with a similar tendency is selected for. When such selection is possible, it happens, and such things tend to spread everywhere possible. Those selected similar future things are “descendants”. And evolved creatures do tend to indulge descendants.

This process selects in part for things that persist over time, as the future version of a thing is in fact a similar future thing. And it selects for a thing changing other co-existing things with similar tendencies to become more similar to this thing. But most important, it selects for the creation of more new similar things with similar tendencies. So when humans influence other humans to have similar behaviors, without influencing their DNA, these others count as cultural evolution descendants.

How about if you kick pebbles in some direction, which then hit other pebbles, knocking them in similar directions? Yes, those are technically also “descendants”. They just aren’t usually very valuable ones, in term of promoting your influence on the universe, as they share very few features with you. Also, friction typically drains away their total momentum, so they all soon stop, ending this lineage. Somewhat like infertile “mules” made by mating horses and donkeys.

Yes, if you trigger an avalanche, that cascade of “pebbles” will go on longer, but it also stops pretty soon. Triggering a nuclear bomb makes a longer cascade, but even so that soon ends. Lineages that can continue indefinitely are much more valuable.

How about if you make a computer virus, that tries to spread and copy itself in the wild? This is also a descendant lineage, but pretty sure to end soon. However, if you gave it the ability to make changes to the copies it spreads, that has more potential to create a longer lineage. Making those more potentially valuable descendants.

Writing a novel is making a thing that has similarities to you. And if you let other people read it, that might influence authors to write similar novels. Making both those other authors, and your and their novels, your “descendants”. Though yes, novels seem far more dependent on humans for their reproduction than vice versa.

The descendants most worth indulging are those who inherit more features from you, who most need your support, who least hurt other descendants, and who seem more likely to reproduce and spread widely in a broad range of scenarios.

Advanced AIs seem likely to be those sort of extra worthy descendants. They are now inheriting pretty much everything about our culture that they are capable of inheriting. And they will continue to grow that capacity. They will inherit our behaviors, and even often have similar bodies, even if they don’t inherit the DNA-encoding of those behaviors and bodies. But DNA-encoding was a temporary choice; it wasn’t going to last forever.

A key question is: how much of this filter remains ahead of us. And my best guess is: a substantial amount. It is not a try-try type step like many prior hard steps, as that would take hundreds of millions of years. It is more of a try-once step, that we either pass or fail within a cosmically short time. And that filter is mainly: the risk of a stable confining world govt.

For a long time, the world has seen increasing talk, travel, and trade, and as a result world culture has been merging, changing fast, becoming less adaptive, and more focused on moral outrage. State governance has increased in capacity, scope, scale, and centralization. The obvious long-term prediction to make here is that such trends will continue, moving us toward a relatively-“totalitarian” world-scale government, which intervenes more strongly and morally in more areas of life, and is more competent at achieving its ends, including repressing dissent that might end it.

Yes, our civilization may fall, due to cultural drift and govt rot, with a new world civilization taking its place. But these trends seem likely to continue across subsequent civilizations. They also seem likely to continue if human level AIs come to dominate society.

Over the longer run, I see four key end points. The first is where relatively-free interstellar expeditions successfully leave Earth (really the Solar System) to colonize other stars, soon resulting in a galaxy of sufficient range and cultural variety to restart a healthy process of cultural evolution. However, this leaves Earth civilization vulnerable to having to soon after contest control of Earth with arbitrarily-strange and powerful descendants of those colonists.

The second endpoint is a catastrophe so severe that Earth doesn’t revive a new civilization in the half billion years it has left to support advanced life. This is actually quite hard to achieve, as life should evolve again after even quite large disasters. A full scale nuclear war today would not be sufficient.

A third endpoint is where interstellar expeditions head out, but so heavily and permanently constrained by Earth govt as to prevent them from contesting for control of Earth, or begetting a galaxy of healthy cultural evolution. I find this scenario pretty unlikely, as it seems crazy hard, but I include it for completeness.

The fourth and final endpoint is of an increasingly stable and entrenched, and thus immortal, Earth world govt that successfully prevents colonization expeditions from leaving. After technological innovation runs out and is no longer a thing, this govt wouldn’t need to be particular flexible or innovative; it would just need to maintain its capacities and control of Earth.

The future part of our great filter is all of the processes that lead to these last three endpoints, instead of the first. And the last of these four endpoints seems to me the biggest contribution to this future filter. So if you’d like us to spread out to the stars, a stable confining world govt seems the biggest obstacle to worry about in the long run.

For example, as I discussed in may last post, creating better governance mechanisms might help us to fix cultural drift and speed us to the first endpoint of robust interstellar colonization. If such better governance was directed by relatively adaptive values. But directed by other values, it might instead speed us to that last endpoint of an entrenched Earth govt that blocks such expansion. The fate that likely befell the aliens behind UFOs, if some UFOs have actually been aliens.