Dreams of Autarky
Selections from my 1999 essay "Dreams of Autarky":
[Here is] an important common bias on "our" side, i.e., among those who expect specific very large changes. … Futurists tend to expect an unrealistic degree of autarky, or independence, within future technological and social systems. The cells in our bodies are largely-autonomous devices and manufacturing plants, producing most of what they need internally. … Small tribes themselves were quite autonomous. … Most people are not very aware of, and so have not fully to terms with their new inter-dependence. For example, people are surprisingly willing to restrict trade between nations, not realizing how much their wealth depends on such trade. … Futurists commonly neglect this interdependence … they picture their future political and economic unit to be the largely self-sufficient small tribe of our evolutionary heritage. … [Here are] some examples. …
[Many] imagine space economies almost entirely self-sufficient in mass and energy. … It would be easier to create self-sufficient colonies under the sea, or in Antartica, yet there seems to be little prospect of or interest in doing so anytime soon. …
Eric Drexler … imagines manufacturing plants that are far more independent than in our familiar economy. … To achieve this we need not just … control of matter at the atomic level, but also the complete automation of the manufacturing process, all embodied in a single device… complete with quality control, waste management, and error recovery. This requires "artificial intelligence" far more advanced than we presently possess. …
Knowledge is [now] embodied in human-created software and hardware, and in human workers trained for specific tasks. … It has usually been cheaper to leave the CPU and communication intensive tasks to machines, and leave the tasks requiring general knowledge to people. Turing-test artificial intelligence instead imagines a future with many large human-created software modules … far more independent, i.e., less dependent on context, than existing human-created software. …
[Today] innovations and advances in each part of the world depending on advances made in all other parts of the world. … Visions of a local singularity, in contrast, imagine that sudden technological advances in one small group essentially allow that group to suddenly grow big enough to take over everything. … The key common assumption is that of a very powerful but autonomous area of technology. Overall progress in that area must depend only on advances in this area, advances that a small group of researchers can continue to produce at will. And great progress in this area alone must be sufficient to let a small group essentially take over the world. …
[Crpto creditial] dreams imagine that many of our relationships will be exclusively digital, and that we can keep these relations independent by separating our identity into relationship-specific identities. … It is hard to imagine potential employers not asking to know more about you, however. … Any small information leak can be enough to allow others to connect your different identities. …
[Consider also] complaints about the great specialization in modern academic and intellectual life. People complain that ordinary folks should know more science, so they can judge simple science arguments for themselves. … Many want policy debates to focus on intrinsic merits, rather than on appeals to authority. Many people wish students would study a wider range of subjects, and so be better able to see the big picture. And they wish researchers weren’t so penalized for working between disciplines, or for failing to cite every last paper someone might think is related somehow.
It seems to me plausible to attribute all of these dreams of autarky to people not yet coming fully to terms with our newly heightened interdependence. … We picture our ideal political unit and future home to be the largely self-sufficient small tribe of our evolutionary heritage. … I suspect that future software, manufacturing plants, and colonies will typically be much more dependent on everyone else than dreams of autonomy imagine. Yes, small isolated entities are getting more capable, but so are small non-isolated entities, and the later remain far more capable than the former. The riches that come from a worldwide division of labor have rightly seduced us away from many of our dreams of autarky. We may fantasize about dropping out of the rat race and living a life of ease on some tropical island. But very few of us ever do.
So academic specialists may dominate intellectual progress, and world culture may continue to overwhelm local variations. Private law and crypto-credentials may remain as marginalized as utopian communities have always been. Manufacturing plants may slowly get more efficient, precise, and automated without a sudden genie nanotech revolution. Nearby space may stay un-colonized until we can cheaply send lots of mass up there, while distant stars may remain uncolonized for a long long time. And software may slowly get smarter, and be collectively much smarter than people long before anyone bothers to make a single module that can pass a Turing test.
The relevance to my discussion with Eliezer should be obvious. My next post will speak more directly.
Eliezer, both computing and manufacturing are old enough now to be "traditional"; I expect each mode of operation is reasonably well adapted to current circumstances. Yes future circumstances will change, but do we really know in which direction? Manufacturing systems may well also now ship material over distances "for reason of centralizing information".
Yes, small isolated entities are getting more capable, but so are small non-isolated entities, and the later remain far more capable than the former.
I mentioned this recently on thhe thread two down from this one, but just in case it isn't sinking in, the main issue in this area is not with isolated entities - rather it is with folks like Google - who take from the rest of the world, but don't contribute everything they build back again - and so develop their own self-improving ecosystem that those outside the company have no access to. The only cost they pay involves not gaining in the short term by monetising their private tech (by sharing it) - and that cost can be swallowed gradually, a drop at a time.