For years, cybersecurity experts have been warning about the chaos that highly capable hacking bots could usher in. … Claude Mythos Preview appears to represent not an incremental change but the beginning of a paradigm shift. … Perhaps more concerning than the reported capabilities of Mythos Preview is that other companies are not far behind. (More)

Finding bugs was also hard, so the worst flaws stayed hidden, sometimes for decades. It wasn’t a great system. But the difficulty on both sides created a kind of détente that held. Now, thanks to new A.I. tools, anyone can write code. Soon, bad actors could use those same tools to find out what’s wrong with code. The détente is over. (more)

Use strong passwords that are unique across every site, preferably through a trusted password manager. Better yet, when a site offers a passkey, take it. … For accounts without passkeys, use an authenticator app for two-factor authentication, not text messages. Always keep all your software up to date, and uninstall unnecessary apps. (more)

OK, I’m a few weeks late to this party, but not too late to give many of you news: We may soon face a period (a few years?) of greatly reduced software availability.

For many decades, we have known how to write pretty secure software. It takes a bit longer, and security considerations must be central to early design efforts, but it is possible. However, developers have usually been in too much of a rush to market to do this. So most software systems today are riddled with security holes. What has saved them so far is that it takes humans a lot of work to find and exploit such holes.

However, there now exist powerful AI systems that are far better at finding and using such holes. Soon (within a year or two?) many AI firms will have such tools, and they will spread to be widely available. Yes, such AI systems can also work to patch such holes, but computer security experts tell me that the nature of insecure systems is to make it much easier to find and use than to patch such holes. Attack beats defense.

Software firms would then more eagerly rewrite their code to use more secure designs, and AI could help them to do this. But this takes time, and as there isn’t a lot of secure software out there now, AI hasn’t had big datasets ready to help them learn how to do this well. So it will take some time to replace weak with strong software.

So there may soon be a period, starting within a few years, maybe lasting a few years, when most actual software systems can cheaply be hacked. This will make such software firms vulnerable to ransomware, and make customers wary of using their products. Customers, firms, and App stores, will respond by cutting back on what software systems they offer, and by simplifying them by dropping many features.

As our world has come to rely on software for a great many things, it seems quite concerning that we might soon have to make do with substantially less software. How vulnerable are crucial systems like electricity, cars, traffic lights, voting systems, and payment systems? I don’t think we know. Beware the coming Hackastrophe.

Note: such an event would likely make the public much more willing to regulate AI. And if credit card firms get overwhelmed with false sales, that could make crypto more attractive.

Besides participating as suppliers, customers, employees, or targets of such orgs, there are two other key ways we engage such orgs: politics and governance. In politics, we take sides among the different alliances of masters and tasks, struggling for who will dominate. In governance, we try to hold masters accountable for achieving tasks, and seek new better ways to choose, reward, and monitor them.

Low status folks have long been advised to keep their head down and stay out of both politics and governance. Higher status folks, in contrast, are somewhat encouraged to do politics, if they are willing to risk suffering repression when their allies lose. We like democracy as more of us can more safely be political, and thus see ourselves as high status, though politics becomes less safe as political polarization rises.

However, most folks are well advised to stay out of governance, at least when that involves any substantial chance of holding masters more accountable, and thus cutting into their spoils. Masters coordinate to block cuts to their spoils. (Yes, some spoils come via achieving promised tasks, but most don’t.) In contrast, masters don’t mind and even like governance changes that don’t risk stronger accountability. Such as making it more popular, inclusive, decentralized, more intensive participation, etc.

How much should you fear masters displeased by your meddling in governance? Greatly! Org masters, and their allies and wannabes, are the fiercest predators of our world. Smart, energetic, and well-connected, they are wolves in sheep’s clothing, smiling broadly, speaking gently and grandly, but holding their fangs and claws ready in shadows to strike when ready.

Alas, our world has long suffered from poor governance. So much so that for most problems we know how to solve, we don’t actually solve them. We got better enough at governance to allow the modern world to have big orgs, but just barely.

Today, our civilization faces problems so huge that we will mostly likely fall, as did the Roman Empire, to be replaced by insular fertile cultures like the Amish and Haredim. Better governance seems our best hope here, and promising alternatives do exist, ones that can be tested at small scales before deploying on larger scales. Alas such efforts are mainly blocked by spoil-protecting masters. Will enough of us risk their displeasure to force such innovation experiments in time?

Yet, strikingly, few in public discussions present themselves as trying to figure things out together with their convo partners. Such as by posing problems and questions, reframing these to avoid sloppiness, offering alternative options and answers, noting puzzling or contrary consequences, and admitting when one’s prior convo moves are undermined by new points made.

Yes, presenting a figuring-stuff-out-together convo persona often imposes some costs relative to other possible personas. But the more eager that we are to suppress other possible interpretations of their motives, the more eager we should be to pay such costs, to assert our preferred persona.

I have to conclude that while we usually don’t want to directly admit that we seek to impress, sell, or support, we don’t actually much mind observers inferring such motives in us. Few actually have that much respect for people those who try to figure stuff out together.

As an economist, not a lawyer, I write here on public interest, not what is legal.

Like all financial markets, prediction markets can serve many functions, such as moving and cutting risk, collecting and sharing information, and the fun of action and proving yourself in competition. For decades, that risk function was the only one U.S. regulators allowed as a justification, but I’ve long argued for a huge potential info value, far more than what we now realize. I want these markets to grow toward that potential. While I personally don’t mind people having fun, I see that others mind, and we might have to compromise there.

Focusing on that info function, prediction markets have many issues in common with other info institutions, like gossip, academia, and journalism. All info institutions can induce folks to (A) reveal info better kept secret, (B) reveal secrets people promised to keep, (C) waste time and money that could be used productively, (D) make misleading contributions to get favorable treatment, (E) change the world to get favorable treatment, and (F) reward participants unequally.

I admit these are real issues, but I say we should treat the various info institutions similarly, unless we find specific reasons to treat them differently. For example, if you wouldn’t forbid govt employees from talking to reporters, for fear they’ll reveal govt secrets, also don’t forbid them from trading just due to similar fears.

On (A), an example is election-day who-wins predictions, which many say discourage voting. But as the risk of over-regulation here is severe, the first amendment should protect prediction markets as an info institution, especially markets on politics and policy. Just as protests are protected, since there are things you can say via protests you can’t say via mere words, trades should also be protected, as there are things you can say with trades you can’t say via words or protests. Putting your money where your mouth is adds punch to your words.

On (B), orgs have legit interests in keeping secrets, but outsiders often have legit interests in exposing them. Many of history’s most lauded journalism stories were enabled by org leaks. There’s a tradeoff here, and requiring everyone to work to help all orgs keep their secrets goes too far. We have strong rules on the books now re prediction market “insider trading”, but note that such rules for stocks have had limited effects. At public firms announcements, half of the price change happens beforehand, and half of that is from insider trading. We shouldn’t expect prediction market rules to succeed much better, or to result in much worse harms.

On (C), other financial markets already allow as much pure “gambling” as anyone could want, and compared to prior ages we today let people devote great time and money into non-productive fun of many sorts, including news, making risky choices of who to date, and making risky choices of careers like acting, music, or athletics.

On (D), speculative markets are actually far more resistant to manipulation than other info institutions. When traders expect more efforts to manipulate a price, they respond so that prices on average become MORE accurate. Also, in head-to-head comparisons with other info institutions, with the same question, time, participants, and resources, speculative markets have been consistently about as accurate or much more accurate.

On (E), life insurance has big enough stakes and easy enough personal influence that we reasonably regulate it to prevent murder for money. But we see almost no cases of traders successfully sabotaging firms to profit from stock trades; firms seem too hard for individuals to influence compared to the stakes. And when prediction markets have been made on events that individuals can influence, it seems traders have been well aware of this fact, and saw this fact as adding to their fun.

On (F), other info institutions also give unequal rewards for intelligence, education, effort, and good social connections. Yes, we could create amateur-only markets, but few would want to trade there; most want to try their hand competing with the best.

Due to their great info potential, let’s approve prediction markets by default, especially when they can inform topics that matter, and only restrict them when we see clear evidence of harm, applying similar standards and scrutiny as we do for other info institutions.

But what are you the most wrong about where you do have opinions, and where they are consequential for you? Consider seven factors that say when you are BLINDED:

• [B]ound: When you are judged by your group on your confident and unthinking belief in and loyalty to particular claims, you won’t study them well.

• [L]ow-Impact: When you are wrong about factors relevant for collective choices, your vote barely moves them, and so you have little incentive to think about them to make them better.

• [I]ndefinite: When concepts come from a high dimensional space where it seems hard to pin them down, separate them, or to define or measure them.

• [N]on-Connected: When you see relevant concepts as coming from a whole separate realm that has no logical connections to all the usual realms where you know things.

• [D]evalued: When you declare yourself to be largely indifferent to the consequences for you, as something else matters much more to you.

• [E]vidence-Poor: When you actually have little relevant data to draw on, and the best data that you have supporting your opinion is the mere fact that some groups like yours have continued to exist and while holding this opinion.

• [D]ynamic: When the topic is about what changes to be making to your group’s collective choices, either recently or in the near future, the mere fact that your group exists no longer offers even weak evidence for those choices.

The max mistake topic area, with all of these factors, is: the adaptiveness of your morals.

Your group suspects that you are evil if you do not see their morals as obvious, and even suspects you if you had to think to come to agree with them. Morality is a collective choice, where you are punished for deviating, so to have an impact you’d have to change your group’s shared moral opinions. Moral concepts tend to be hard to pin down, and today most see moral claims as sitting in a disconnected realm where all our usual non-moral claims are not relevant.

On the topic of the cultural and DNA adaptiveness of your group’s morality (and norms and status markers), most people say they care much less about the adaptiveness of their morals than about the “moral truth” of their morals. Figuring out theoretically which morals are more adaptive is actually quite hard, and so our best evidence is empirical: which successful societies have had which morals. But the fact that your society seems inclined to change its morals lately in a particular direction is far weaker evidence for the adaptiveness of that direction.

The topic where you most need careful thought is also where your community most punishes such thought. This is our big blind spot on which our civ will likely fall.

I asked ChatGPT (5.5) and Claude (4.7) to give percentile estimates for the median level who judges who are the very best intellectuals, for the West in various years. They gave median 99%,99.5% for year 1000, median 96%,97% for year 1750, median 93%,90% for 1900, and median 88%,80% for 2025.

We have thus seen an increasing populism in who among us judges who are our very best intellectuals. Which is plausibly a source of intellectual decay. Especially as it is often noted that we usually find it hard to distinguish between mental quality levels above our own.

April 25, 1996, thirty years ago today, I first posted my best idea: decision markets, i.e., speculative markets that advise specific decisions by estimating decision-conditional outcomes. A.k.a., “futarchy” as applied to governance. It’s not my deepest, grandest, beautiful, or hardest won insight, just the one with the biggest expected impact.

My idea was a simple combo of two other long-well-known ideas.

The first prior idea I built on is that speculative markets do quite well at aggregating info. This was explored in theory (Emory 1896, Gibson 1889, Bachelier 1900) and in data (Cowles 1933, Working 1934). Even so in 1996, US regulators in practice only allowed risk-hedging, not info aggregation, as an “economics rationale” to allow markets to exist. (The allowed “price discovery” rationale was tied to helping other markets hedge risks.)

In 1984 I left grad school in physics and philosophy of science at U Chicago to go to Silicon Valley to do AI research, and on the side work with Xanadu, trying to invent the World Wide Web. Around 1988 I first started to have doubts about the Xanadu vision of reforming public convo by making criticism easy to find, and wondered what else we could do instead. So I started to think and write about the big potential of making speculative markets to aggregate info on far more topics. Like most everyone who first enters this space, I was first thinking mainly in terms of markets on the usual topics we see in mass media, punditry, and public policy debates.

The second prior idea I built on is that info is mainly valuable by informing specific decisions. For many centuries we’ve seen calculations of the value of certain specific info for specific decisions. And then we developed more general theory (Ramsey 1928, Hosiansson 1931, Blackwell 1951, Savage 1953, Schlaifer 1959). At Caltech social science grad school 1993-1997, I learned decision theory and the standard value of info calculation. Then wondering where speculative markets could add the most info value, ~1996 I realized that this would likely come from markets estimating specific outcomes given specific decision choices.

As I was one of the first to write on the big potential of prediction markets, many who entered this space over the years approached me. At which point I usually pitched this decision market concept. Which usually pushed them away, as they were focused, as I was initially, on those mass media and punditry topics. But I have doggedly persisted.

Most all innovations combine simple elegant ideas with messy details that make those ideas work. Mine is no different. To find the right messy details, one needs concrete trials and experiments trying different detail versions. It has been hard to find orgs willing to do this, as org decision making is usually quite political. But in the last few years we’ve thankfully started to see some trials.

As an econ professor who specialized in governance, I can assure you that the world is greatly structured by the fact that we typically have pretty incompetent governance. Imagine a governance that, when assigned a goal, would reliably achieve that if it is in fact feasible. This would radically reshape our whole world. (Yes, even if we soon get powerful AIs.) As decisions markets plausibly enable such competent governance, this is why I estimate their expected impact to be so very great.

Many parameters correlate simply with this high-to-low goal axis. For example, lower-level goals and actions tend to take less space, time, and other resources. They are less likely to conflict with other goals, and more likely to be time-consistent. They are more easily evaluated for success, better described by simple abstractions, more reliably controlled, and more easily optimized by hill-climbing. They seem more observable, reversible, and substitutable, give faster feedback, and are more easily automated.

However, other related parameters depend on this key high-to-low goal axis in less simple ways; they instead peak at some mid-level, and fall away from that in both directions. For example, we have more conscious awareness of, give more conscious attention to, and make more deliberate choices re mid level goals. We can more clearly articulate them and their relations to other goals, and we can more easily teach others to manage them. People coordinate with each other more here, and our blame, credit, norms, and laws focus more here. There is more cultural variety of behaviors at these mid levels; other behaviors are more set by DNA.

A noteworthy exception is that such mid-peaking parameters often peak at much higher levels in large for-profit orgs, and in other large orgs, like militaries, with strong incentives tied to concrete goals. Such orgs often can and do articulate, measure, credit, and blame the behaviors of top people who manage high-level goals.

A simple interpretation of these patterns is that cultural evolution of coordinated behaviors faced a key tradeoff. Let me explain.

As thinking and talking takes time, there is a lowest level of goals and actions where we can discuss them as we choose and do them, so that such talk greatly influences those actions. While humans can and do watch and learn details of others’ behaviors that are at much lower-levels, we mostly do this non-verbally and unconsciously.

However, to enforce norms, including the norms that say that we should keep our promises, we humans need to be able say to others in sometimes-verifiable words what we and others have or have not been doing lately. So that we can complain about such actions, and recruit others to exert social pressures toward norm enforcement. To defend ourselves against such accusations, our conscious minds were created to manage key stories of what we’ve been doing lately and why.

So cultural evolution got into the habit of having us think and talk consciously about goals near this lowest-articulable level, and also to notice, copy, and teach chunks of behavior near these levels. And in addition, we mostly manage our norms, status markers, and key coordination mechanisms near such levels. As this cultural evolution process is pretty random and uncoordinated, efforts to abstract these norms and chunks most naturally expressed at these mid levels into higher level goals don’t usually achieve much clarity or coherence. Also, we seem reluctant to explicitly name cultural adaption itself as a big higher level goal.

So why didn’t we instead define and manage our social coordination using much higher levels goals? The simple correlations above say that such higher goals would tend to be less modular, less observable, and less easily described using abstractions. Making it harder for us to see and describe them, and to enforce norms about them.

However, with the invention of money and for-profit orgs, the world has now found new ways to use modular observable goals at quite high-levels. When we allow such orgs to manage key areas of life, they have shown remarkable abilities to effectively coordinate our behaviors. The problem is that, in many minds at least, their wider use would violate other key norms that we have inherited from cultural evolution.

Notice that cultural natural selection of individual behaviors seems insufficient to evolve better norms and status markers, as these are features of key game-theoretic equilibria, where individual deviations are punished. We need instead to have collective deviations of entire cultures, i.e., units with much stronger internal than external conformity pressures.

Alas this process has been greatly hindered in the last few centuries by decreasing variety and selection pressures, and increasing rates of environmental change and internal cultural drift. Which is plausibly causing such norms to decay, plausibly leading to civ collapse and replacement in a century or two.

Added 25Apr: There is a theory of “basic level categories” saying we have a natural level of abstraction across our concepts in general. HT.

For money, my dad was a programmer, and my mom made presentation graphics for a finance firm. On the side they were missionaries, a pastor, and a writer. When she could retire from making money, my mom became a writer full-time, contributing to 30 of the 275 Chicken Soup books; ~20M people have probably read one of her essays there.

My two bothers were most recently a court bailiff and a pool cleaner for money, and on the side a musician and pastor. Their wives were a sales clerk and a legal secretary. My wife was a clinical social worker and her brother was a govt lawyer. ChatGPT (5.4) and Claude (4.7) estimate ~45-55, 50-60 as percentile ranks for this family overall in terms of job prestige.

I’m now a university professor, and my two sons are a programmer and an investment banker, so the three of us together get ~75,90 percentile estimates. Making my family solidly middle class, and me and my sons upper middle.

Workers often face a conflict between how their job has been define by the world and their training, and what their managers tell them to do on that job. Usually people succeed more when they accept boss framings, and higher class folks more tend to have this and other more successful habits.

Both LLMs say that this also happens more specifically in academia, where there’s a conflict between the job defined as intellectual process, i.e., helping the world better understand key abstract topics, and the job defined as what it takes to get prestige and resources. Lower class folks tend more to pursue that first definition. My class background is substantially lower than that of most academics, and I fit this pattern, as I see my job more in terms of intellectual progress, less in terms of resources and prestige.

At my mom’s funeral, I was reminded that such events involve much praising of the dead on various metrics. Which raises the question: what do you aspire to be praised on at your funeral, and in future historical mentions? It also a meta question: why don’t we write periodic essays on what we are trying to achieve in our lives, so that at our funerals folks can discuss how well we achieved our stated goals? Yes of course they could also discuss how well we achieved their other goals for us, but our own goals also seem quite relevant.

I would of course prefer that, at my death and after, and even well before, people praise me for all the usual virtues. But compared to others, I put a much bigger weight on intellectual progress. I want people to say, because it’s true, that I helped the world gain insight on important neglected potent topics. Important because that’s what matters, neglected because it is far easier to find big insights on those topics, and potent because the big win is when others build on my insights, and integrate them into larger shared systems, as part of a long process of civilization accumulating insight. And myself having insights isn’t that valuable compared to communicating them in ways so let others see and build on them.

At my funeral, please do ask yourselves how well I did at this.

So CEOs block the application of futarchy to firms. You might think that investors could just force CEOs to use futarchy, if that would max investor gains. But investors also can’t seem to prevent the adoption of poison pills, which also cut investor gains. It seems we must accept that top managers have power sufficient to induce firm outcomes that don’t max profits. Investors do not in fact fully control firms.

Okay, then what if we flip this script, and set decision markets to the task of directly achieving the selfish managerial ends that likely drive managerial power politics? Create a metric of the total success of an individual manager over their future career, and then make advisory power-futarchy markets that estimate this personal success given key choices under that manager’s power. And to discourage sabotage, give everyone who that might be able to act to greatly hurt this success a positive stake in that success, a stake they aren’t allowed to trade to below zero.

Would this supercharge power politics, via better informing political strategies? Plausibly this would improve both offensive and defense political choices, and also make political info more symmetric. Managers could less often win via strategies that rely on rivals not noticing their plans until too late. So might power-futarchy actually cut harms from firm politics? Maybe, relative to the alternative of no markets at all, helping managers have successful careers also on average helps firms to max profits.

Of course such markets may advise top managers to not create power-futarchy markets to aid their subordinates several levels below them. Such markets might even say to instead give such subordinates futarchy markets tied to key firm or division outcomes. If so, that might usefully limit the scope of power-futarchy. Yes, this might over time undermine support for power-futarchy, but maybe not before current managers achieved great success from it.

Some kinds of power politics strategies may be hindered by open markets estimating their power effectiveness. But we needn’t have such markets regarding all possible managerial choices. Though, yes, the choice to not create such a market on some key choice might be taken as a bad sign about the politics behind that choice. No doubt there would be many new tricks to be found when playing power-futarchy.

Our dominant world culture hates tribes, but loves factions, especially factions who we see as “down”. We hate groups who disagree with world elite consensus on school, medicine, democracy, gender equality, sexual freedom, legal due process, rules of just war, and norms of good parenting. And we hate tribe supporters for their self-favoritism and habitual hostility toward outsiders. About these things we see our dominant world tribe as just right, and the others evil.

But we love factions within this main tribe who embrace distinct symbols, and who fight for tribe norm reforms. We call this love “tolerance”. At least we love factions who we can plausibly see as “down” relative to “up” rivals. (We presume “up” illicitly hurts “down”.) We hate “up” faction members who promote their factions, and accuse them of actually representing hated tribes. We moderns tend to channel our instinctive human tendencies to be tribal into our factional conflicts. Not noticing how we need tribes far more than factions.

The big problem is that in history our moral norms and status markers came mostly from cultural group selection acting on tribes, not factions. By crushing all but one dominant tribe, we now mostly block such evolution from preventing the decay of shared norms, or their adaption to changing context. We now see this most clearly in the decay of norms supporting fertility, but such decay is plausibly also happening across all our key norms. Selection acting instead on factions can’t do this remotely as well. If such decay continues long, our civilization will fall, to be replaced by others.

Unfortunately, we find it hard to see this problem, as our moral norms and status markers seem to us as just obviously true, and thus good bases for any analysis. In contrast, we can see and appreciate fights among factions, as we can frame these in terms of our “obvious” shared norms. But that doesn’t help much to ensure that the winners of faction fights are more adaptive.

Instead of trying to repress competing tribes, as we usually do, we might try to instead promote them. But even that seems quite insufficient, as the main underlying reason that the world has over centuries been merging toward one big tribe is the increasing ease of distant trade, travel, and talk. Such merging has achieved great scale economies of production and innovation, and a great reduction in conflict harms, such as via war, due to increasingly shared norms. Most people really like having a world community with shared norms..

There are a few today, like the Amish and Haredim, who care enough to treat themselves as distinct tribes, and are willing to forgo many gains of world cultural integration to achieve this. Such folks insulate themselves culturally from the large world, and so are the folks today whose descendants are mostly likely to replace our dominant world culture. But few groups today are this devoted to becoming tribes. Most of the folks today interested in cultural variety, like “network state” folks, are not remotely this devoted, and so have little chance of creating new tribes.

I can see only three ways for our main world civ, which I treasure in many ways, to avoid being replaced like this. The first solution is to somehow greatly raise the status of tribes, relative to factions. Convince the world to fragment into far more tribes, not just factions. Tolerate and even encourage groups having quite deviant views on democracy, gender equality, etc. to favor themselves and isolate from outsiders.

The second solution is to leave the world mostly integrated into one big tribe, but to find new ways to control and govern how key moral norms and status markers are changed to become more adaptive. Such as via competent governments held strongly accountable to increase adaption futures estimates, or via using a competent futarchy to pursue sacred adaption-achieving goals like when a million people live in space.

The third solution is to vastly increase the role of for-profit orgs in setting our moral norms and status markers. The evolution of firm cultures has long been quite healthy, as firms form quite distinct groups facing strong capitalists selection pressures. And for-profit orgs competing to give customers key numbers and observable outcomes have quite consistently improved on such outcomes. Each area they came to control, such as buying and running governments, or paying parents to make kids they could in effect sell, would likely become more adaptive.

As you can plainly see, these are all big long-shots. Our situation is quite desperate. And not likely to get better until a lot more people start to think about it.

You may have heard that such *group selection* never happens, but that’s wrong. Not only do most cultural evolution scholars see group selection as a key force, group selection also seems to important in DNA evolution, where species are groups. The fact that more species today descended from fragmented habits like rivers, coral reefs, and rainforests, where habitats were smaller, suggests that group selection of species has actually mattered more for DNA than individual selection within species.

I’ve said previously that healthy cultural evolution for stuff like norms status markers depends on four key parameters: enough cultural variety, strong enough selection pressures on cultures, slow enough internal cultural drift, and slow enough rates of environmental change. But I have to admit that this first “variety” parameter is a sloppy way to talk about it. Counting the number of cultures would make sense if, as with species for DNA, there was only one clear scale at which people are joined into cultural groups. But in fact cultural behaviors cluster together at many different scales.

However, I’ve been doing some reading, and have found that for decades cultural evolution scholars have had a less-sloppy substitute concept: “network structure”. If you look at the details of who people interact with, and who they are likely to copy their behaviors from, the shape of the network of such ties matters a lot for cultural group selection.

For example, the network feature that most promotes group selection seems to be “modularity”, roughly how many more ties there are within clusters, compared to between clusters. It also matters how similar are people within clusters, how much overlap there is between interaction and emulation networks, how well prestige tracks adaptiveness, how much conformity pressure there is for a behavior, and how much that behavior effects visible outcomes that people care about.

Each different type of behavior can have its own different network, and its own different coordination scale, requiring group selection at that cluster scale or above in order to select adaptive versions of that behavior. But it seems clear that relevant scales for many kinds of behaviors have greatly increased over the last few centuries, greatly reducing the effective “variety” for the purposes of cultural evolution. And this is plausibly cutting the effect strength of group selection, likely enough to cause net maladaptive change to our norms and status markers.

Power Fights - Most stories are about conflict, and so most big stories are about fights. And long ago, most big stories (e.g., Illiad) focused on powers and alliances fighting within worlds that were relatively stable, especially re tech, and within a context of stable morals. As those didn’t change much, stories didn’t care much about them.

The simplest stories of this type focused on one particular fight, with a start, middle and end. More complex stories, on longer timescales, might depict a sequence of fights with relative peace in between. Even more complex versions might have old powers leave, new powers enter, and changing alliances between powers.

Moral Fights - Starting with religious stories, but then spreading to most centuries ago, the sides in fights acquired stronger moral colors. These fights were not just about power (i.e., dominance) but also moral persuasion (i.e, prestige). The simplest versions had good heroes fight bad villains (e.g., Lord of the Rings). More complex versions had many fighting sides, or all sides seeing themselves as good.

Some moral fight stories have a small group of activists trying to spread their new moral view to a wider world. A common feature here is that the world at story end likely has more or less good morality, depending on who wins the fights.

Unstable Tech - Our modern world often has tech and business changing fast on the timescales of big fights. Tech changes often favor particular sides of fights, and can call into question common assumptions in prior moral positions. Many science fiction stories highlight how tech changes can influence who wins, and how they can force one to reconsider basic moral commitments.

The simplest such stories present a world with quite different tech to ours, but where that tech doesn’t change much during the story (e.g., Dune). This helps readers see how tech differences might translate to fight and moral differences. More complex stories focus on one particular big tech change (e.g., Frankenstein), and show that one change affects who wins in fights, and key moral categories. The most complex stories show long fights in the context of a long history of many big tech changes.

Unstable Morals - I’ve lately become unhappy with science fiction, as I came to understand the basics of cultural evolution. Science fiction’s big or fast changing tech, even with shifting powers and alliances over centuries, are usually set in the context of quite stable morals. Yet in fact over the last century or so key values, norms, and morals have changed about as fast as tech, and due to pretty random and plausibly out-of-control cultural evolution. A similar failure happens when historical fiction sets characters with modern values as heroes against villains with old-style values.

So I’d like to see authors try to write big stories, of whole civilizations over long timescales, that more realistically depict cultural instability. Yes it can be comforting to see key characters long continuing to fight for the same shared moral causes, even as their powers, alliances, and tech change greatly. And it can be disturbing to see key morals changing as fast as tech, and nearly as arbitrarily. But the switch to Unstable Tech type stories similarly resulted from the disturbing realization that fast changing tech often upended our conflicts. And we seem to have managed that switch okay.

Traditional societies have discouraged, regulated, and banned many kinds of pleasures. Such as sleep, idleness, fancy or plentiful food, fancy clothes, travel, humor, music and dancing, gossip and small talk, drugs and intoxication, fiction, gaming, gambling, bragging, gossip, fighting, spanking, and many forms of sex including prostitution. They feared such pleasures distracting from work and piety.

Our world still bans many things, but pleasure isn’t usually a central consideration; we are far more indulgent and approving of pleasure. Yet we still do ban a few pleasures, including recreational drugs, dogfighting, corporal punishment, loan sharks, dwarf-tossing, gambling, and sex that is paid or with minors. Drugs, dogfighting, dwarf-tossing, corporal punishment, and loan sharks seem to be about physical harms, and also shame and empathy. Sex has long evoked deep complex opaque feelings.

But sports bets don’t involve shame, physical harm, or deep opaque feelings. We mostly approve of sports, and of people putting lots of time and energy into playing and watching sports. And sports bets complements those activities, making them more interesting, engaging, and better informed.

Yes, we dislike money all else equal, but we let money touch many adjacent areas. Yes, sports bets can waste time and money, but so do a great many allowed pleasures. Yes, they involve risk, but we let people take big risks in deciding who to date, and in longshot careers like acting, music, or athletics. Yes, sports bets resolve faster, but you can bet just as fast and big in ordinary financial markets. Yes, bookies once charged high fees, but new markets have far lower fees.

I guess I lean toward explaining banned sports bets as just a random exception to our usual historical trend, which seems a weak but good sign re how long we’ll let these new sports betting markets continue to be legal. Not my thing, but I usually don’t mind others having fun via their things.

Added 14Apr: Many point to the possibility of commitment problems, where people are tempted in the moment to do stuff they would want to commit ahead of time not to do. But it isn’t that hard to set up commitment mechanisms, and when we do few actually avail themselves of such options.

“The science in Project Hail Mary is all pretty firmly grounded. There’s some BS all the way down at the quantum level, where Astrophage cell membranes can keep neutrinos in… But outside of that, everything else just follows established physics and science.” - Andy Weir, author of Project Hail Mary

Unrealistic science fiction can be great, but folks should sometimes point out the the unrealism of particular stories, especially stories that are very popular, and widely said to be realistic, including by their authors.

Other have pointed to implausible physics and politics, but after reading two dozen reviews, I don’t find anyone else mentioning my three comments on Project Hail Mary:

Rare Event: In the story, a big dimming of our Sun and a dozen nearby stars happens over decades. This must be a very rare sort of event, or we’d have noticed this scenario before out there among the stars. It also can’t last that long or spread that far each time, before reverting to the usual star appearance.

Close Alien: Our hero meets an alien from a star roughly 20 lightyears from Earth, who is at a very similar level of tech development to us. For example, they haven’t yet discovered radiation or relativity. Say no more than a century different. In a 14Gyr old universe that level of time correlation seems crazy unlikely. Also, to have aliens that spatially close be typical, our universe must be chock full of civilizations. Which then must quite reliably die fast to produce our empty looking universe.

Similar Alien: They have different bodies and sensors, but once they manage to talk, our hero and alien get along better than would two random humans from human history. The alien’s culture is much like our hero’s culture, which is quite different from most other human cultures in history. This is worse than most historical fiction, which puts modern hero characters in old worlds.

Since 2023, it has been widely reported that LLMs seem quite good at coding, and that this seems their most promising application area. Global spending on software is ~$1-2T/yr, so potential big saving there soon might plausibly justify last year’s ~$0.5T/yr AI investment.

For many decades the demand for software has been large and elastic. Most firms have many software projects they’d like to do, which they don’t do mainly due to cost. And market leaders tend to be firms whose software investments went well. So if the total cost of software fell by a factor of two, total spending on software should more than double. That’s supply and demand. And AI getting a big cut of that increased spending soon might justify its investment.

Of course fielding useful software involves many tasks, including identifying opportunities, securing funding, overseeing projects, defining requirements, marketing, user support, and writing, testing, and maintaining code. Just making code writing much cheaper doesn’t obviously make total software cost much cheaper. Much depends on just how many of these other tasks can be made cheaper as well.

If AI is going to have a big impact, when should we expect to see it? Software projects typically take ~6-9 months from conception to delivery, though orgs can take 1-3 years to reorganize workflows, incentives, etc. to accommodate new techs. Legacy software may not be replaced for up to 3-10 years.

So it seems that one should expect to see substantial AI-driven changes to the scale of the software industry within roughly 3 years of a widespread consensus that AI makes it much cheaper. Which is about now if that consensus happened 3 years ago. Or in about 3 years if that happened one year ago.

The number of U.S. software workers increased by ~50% in the last decade, probably mostly due to falling costs. So we should expect an even faster growth in software spending if AI is in fact causing a big increase in the rate at which its costs fall.

And if we don’t see such a big increase in the next few years, that will suggests that AI does not actually cut software costs nearly as much as advocates hope. Which is of course the usual scenario for hyped new techs. And should lead to a crash.

Of course that’s the short run; we might still plausibly see a “general purpose tech” impact that takes several decades to play out, as we’ve seen previously for techs like steam, electricity, personal computers, and the internet.

Added 13Apr: After a long convo with Bram Cohen, here is my revised estimate. About as likely as not, AI will help the software industry grow in spending by a factor of 2-3 over the next decade, and gain 10-20% of software revenue, for $0.4-0.6T/yr revenue. AI will also gain as much revenue from all other industries combined, so total: ~$1T/yr.

But watching a documentary on early activist H.D. Thoreau brought home to me just how huge an ask this seems. Our modern world has come to deeply adore and revere changing its morals fast via youth movements, and a great many features of our modern world support this new pattern.

For example, youths are generally more risk-taking, emotionally expressive, eager to impress potential mates, less invested in prior arrangements, and better able to bond together into groups. Which attracts youths to the chance to skip the usual dues to rise fast in status as leaders of new tightly-bonded emotional youth movements.

Helping further, we legitimized fashions, seeing those who first adopt new popular changes as more virtuous. And we put kids together in high school and college, where they have more time for activism, bond into their own youth cultures, and are taught to see the world more abstractly and thus morality more simply and universally. Also, better communication tech has let them coordinate faster across wider distances.

Finally, the modern world has widely adopted the views (a) that morality is a whole separate realm where the usual adult knowledge and experience are less relevant, (b) that moral opinions should from come authentically from within, and (c) that youthful opinions on morals tend to be less corrupted by habit and self-interest.

All of this has created a perfect storm encouraging youth to repeatedly make and join new internal-feelings-driven moral crusades, movements maximally suspicious of opposing older adults with ties of interest and habits to the existing order.

Could we apply industry to more strongly to manage this process? For example, by paying big orgs to create, suppress, and influence such movements to achieve key metrics. Yes, big orgs do substantially influence youth movements today, but mostly from behind the scenes. And these are mostly not for-profit orgs, and our world is pretty hostile to for-profit orgs operating outside their usual scopes, especially in sacred areas like moral activism. Social media feed algorithms seem to be the main form of this now, but I doubt they could do that much more than they do now.

We should do our best to try, but damn does this look hard.

Hayek, however, seems fine with using reason to choose within big firms, and he admits that cultural evolution (a) has often induced simpler societies to prevent such capitalism, (b) has often induced governments to greatly hinder capitalism in their later civilization periods, and (c) seems a proximate cause of the recent rise of interest in socialism. So why not estimate that the levels of capitalism and reason use that we seem to be drifting toward are in fact the most adaptive? Why see all that as a mistake?

Hayek seems to actually rely here not on cultural evolution, but instead on his theoretical economic analysis, together with empirical correlations between capitalism and places and periods that have had especially large wealth and growth. Which allows him to conclude that allowing cultural evolution to push us far enough away from capitalism now would plausibly result in the fall of our civilization, causing many deaths and much suffering. Which would be bad more because suffering is bad, and less because cultural evolution would go awry.

Behind Hayek’s argument there, however, seems to be a judgment that our modern world looks especially vulnerable to appeals to deeply embedded ancient moral instincts, and to flattery about our abilities to reason. However, as he never says this explicitly, Hayek never offers arguments for why we should expect to be more vulnerable to such things now.

This is where I offer cultural drift analysis as a complement to Hayek’s story. At the level of cultural features that we can only vary effectively in large groups, over the last few centuries our civilization has drifted toward less variety, weaker selection pressures, and faster rates of change of culture and environments. All of which does plausibly make us more vulnerable to flattery and simplistic moral appeals undermining our commitments to morals supporting capitalism.

However, such analysis also predicts that these same forces make us vulnerable to many more fatal conceits, i.e., to decay in many other key features of our shared culture. Does Hayek also fear and warn against excess trust in reason and moral instincts there? Is it feasible for us to reason well enough to usefully overturn other non-capitalist morals that we have inherited from cultural evolution? Hayek said:

Rebellion against private property and the family was, in short, not restricted to socialists. … Limits of space as well as insufficient competence forbid me to deal in this book with the second of the traditional objects of atavistic reaction that I have just mentioned: the family. I ought however at least to mention that I believe that new factual knowledge has in some measure deprived traditional rules of sexual morality of some of their foundation, and that it seems likely that in this area substantial changes are bound to occur. (p.51) …

Nor do I dispute that reason may, although with caution and in humility, and in a piecemeal way, be directed to the examination, criticism and rejection of traditional institutions and moral principles. … I wish neither to deny reason the power to improve norms and institutions nor even to insist that it is incapable of recasting the whole of our moral system in the direction now commonly conceived as `social justice’. We can do so, however, only by probing every part of a system of morals. (p.8)

So Hayek is relatively open to rationality overturning traditional morals in one big area of life, and is in principle open in many other areas. So let me say this clearly: our usual styles of rational analysis deployed over the last few centuries seem to have been quite inadequate to the task of changing morals while preserving or enhancing their cultural adaptability. Maybe we could up our game, but that does look quite hard.

Nations seem similar. Yes, nations value some activities more, and engage in those more as a result. But nations often double-down on stuff after seeing themselves as win at it in ways that they personally respect, and expect others to respect. Nations continue to do that stuff lots in part to remind the world of how grateful it should be for their contribution.

For example, the US has seen itself as pioneering and greatly advancing democracy, free speech, medicine, higher education, basic research, legal due process, mass production, mass media, space exploration, entrepreneurship, the internet, and global military suppression of nazism, communism, and terrorism. This helps explain continued record US spending on medicine, education, military, and legal process.

Other nations act similarly. For example, Britain doubles down on law, parliaments, and anti-racism. France doubles down on liberties and fancy food. India doubles down on yoga and spirituality, Russia on war, sacrifice, and anti-decadence, and China on development.

If you want a nation to do more of X, maybe praise what they’ve already done on X.

Authenticity having your behaviors driven from within you, instead of letting others influence them. Follow your heart, you do you, go with your gut, that sort of thing. It is such a widely accepted norm that the authors who write books on it don’t actually argue for it much; they instead use it to argue for other stuff. My reading was a waste.

But, why exactly is authenticity such a good thing? Yes, there’s this quote about me, “Robin Hanson is more like himself than anybody else I know.” And, yes, my webpage has long said: “I’m not a joiner; I rebel against groups with ‘our beliefs’.” So as a matter of practice I seem to be authentic. Yet I still don’t see why it’s good, per se.

The modern world changes faster, and gives us more options, which puts a premium on agency; we can’t just ride along with our slowly changing peasant village anymore. But that means you need make choices, not that they need to come from within.

We’ve long taken controlling more as a sign of status, so others controlling you lowers your status. But what would this effect be stronger in the modern world?

Maybe in the modern world imitation and social pressures have become easier to see. In the old stable peasant village you acted like everyone else, but so did everyone, and you were not noticeably following any particular other visible models. However, in the modern world choices are more varied and contested, and so we can more easily see who in particular is pressuring or influencing who else in particular.

That wouldn’t necessarily be bad, except that looking too obviously “try hard”, like you are trying to choose actions to impress and please others, shows an unimpressive lack of confidence. Just as the most impressive dancers make their dancing look “effortless”, maybe the most impressive social displays are those that seem to come naturally, with little noticeable effort.

Cultural evolution says that most everything that comes from inside of you was stuff that went there before, from your prior cultural exposures. But seeing you trying to please and conform looks quite different to observers than your seeming to just do stuff from within, even though all of that stuff inside resulted from your prior efforts to please and conform, perhaps as a child. It is like the difference between a dancer who is visibly struggles to do her dance routine, and one for who the routine looks effortless, enjoyable, and even invented on the spot.