The cognitive architecture of bias

You’ll often hear people, including people on this blog, talk about self-deception as a central factor in creating bias.  Work from cognitive neuroscience, however, shows that the "self-deception" description doesn’t really capture how deep the issue runs.  In this post, I’ll give an analogy that a number of us use these days to describe our understanding of the cognitive architecture of conscious speech.  The earliest I can remember using the analogy was in a paper I wrote in the mid/late 90s, and the model behind it drives some of my comments over at Chatty Apes.  Jon Haidt gives the analogy in an interview here.  Rob Kurzban has a superb new article on it here.  Below, I provide the analogy.

Imagine some bureaucratic organization with a number of different departments.  A sort of caricature of George W. Bush’s presidential administration works.  There are a number of strong-willed people in key White House positions (including Bush, Dick Cheney, and Karl Rove) with overlapping but somewhat different agendas that lay out simple priorities (Cut taxes for rich people!  Get Saddam!  Help out our fat-cat cronies in defense and oil industries!  Keep us and people like us in power!).  There are a number of different topic-specific departments (Defense, Commerce, Education, etc.) with dedicated staffs that work on detailed operations.  Some departments or staffs are focused on coordinating action with other institutions like Congress or foreign governments.  And there are communications offices tasked specifically with coordinating information flow to the public in a way that will best allow the deep, unpublicized preferences of the Administration to be translated into action.

Inside the Administration, there is a constant balancing act between what they really want to do and what is possible.  Some things they may want to do are limited by their ability to get people (Congress, foreign governments, the public) to go along with it.  Here, communications strategies become crucial.  Maybe they want to cut taxes specifically for rich people, but they understand that most people aren’t rich and will strongly disapprove of a bunch of rich-only tax cuts.  The solution may be to combine very large tax cuts for rich people with very small tax cuts for middle-class people in a terrifically confusing package, and then tell everyone, consistently and earnestly, that one’s proposal simply involves "across the board" tax cuts for everyone.  If most people don’t have the information necessary to learn the truth, and mostly they’re just relying on the Administration’s communications to figure out whether it’s a good idea for them, the Administration could achieve both goals of cutting rich people’s taxes and being well thought of by middle-class voters.

So, the Administration decides to cut rich people’s taxes, and it comes up with a communication strategy that is maximally bent on effecting the policies and has little direct relationship to the real motivations behind the policies.  Optimally, the Administration will appoint press agents to communicate the message, and, even more optimally, will in fact not explain to the press agents what the real motives are behind the favored actions.  It will just tell those press agents the popular version with the popular motives.  The best liars believe the lie.

If you’re in a position to ask an Administration press agent about the policy, that person will tell you, consistently and earnestly, that throughout the Administration the motives are the unobjectionable ones.  The President thinks the fair thing is to give all hard-working Americans a little across-the-board tax relief.

Press secretaries often get it wrong through no fault of their own even with less vague things than motives.  Consider the Valerie Plame CIA leak scandal.  Early on, Scott McClellan, Bush’s press secretary at the time, positively insisted that Karl Rove had nothing to do with it.  "The President knows that Karl Rove wasn’t involved."  But then it turned out, not so surprisingly, that Rove and others were more than a little involved.  A likely story here is that Rove lied to McClellan and his staff, and that McClellan was sent out with bad information to be the most aggressive spokesman he could be.

Now the very difficult point:  This analogy is meant to be about the human brain — it’s your brain I’m talking about.  Research from psychology and neuroscience shows that your brain has organizational characteristics similar to this caricature of the Bush Administration.   There are systems that are responsible for powerful, simple emotional reactions that serve to focus other systems to give fine-tuned attention to your brain’s priorities and preferred outcomes.  Most of the detailed systems don’t care much about why they have the jobs they are given; they just do the work of carrying them out in a highly distributed, bureaucratic way.  And you — the conscious, chatty you — are that dimwitted patsy, the misinformed press agent.   The conscious you is not the President and you’re certainly not Karl Rove.  You are the Scott McClellan for your bureaucratic brain.  You’re constantly being duped into believing public-friendly stories about yourself, because your entire job is to tell stories handed to you by ruthless, clever, unconscious communications systems.  You’re not the whole head; you’re just the talking head.

You can’t even trust your own introspection, because your press agent simply doesn’t have direct access to the Oval Office.  The best you can do, even with your own behavior, is to try to piece together hypotheses about the hidden motives at work based on what the person actually does, situated in the context of things you know are generally true of why people might want to do those things.

"Self-deception" doesn’t fully capture the depth of what’s going on here.  When you talk to someone, you’re not talking to an integrated "self" that may or may not be deceived — you’re talking to a press agency that doesn’t really make many important decisions, that is only loosely connected to the rest of the bureaucracy, and that has the core mission not of truth-seeking, but bias that serves the interests of its genes.

The thing I really worry about:  To the extent that our scientific accounts of people’s underlying motivations end up painting, as I think they will, an unflattering picture about human predictability and genetic selfishness, I think we’re likely to end up with a small group of scientists who acknowledge it, and a very large group of people (including most scientists) with press agencies that are strongly motivated to deny it and so drift farther away from paying attention to the better work in social science.  How’s that for optimism?