Sunday, January 10, 2010

"A no-shit Sherlock: Escalating Complexity and the Collapse of Elite Authority"

"Elementary, my dear Watson."

Armed and Dangerous has an essay, forwarded by John Robb of Global Guerrillas, that is an excellent analysis of David Brooks' recent moaning in the New York Times about the "Tea Party Teens." (All links are below in A&D's piece.)

I have been so busy with computer breakdown and catch-up that I haven't read several of A&D's recent perceptive pieces. A sample snippet is this observation on "Gramscian Damage":

"The most paranoid and xenophobic conservatives of the Cold War were, painful though this is to admit, the closest to the truth in estimating the magnitude and subtlety of Soviet subversion. Liberal anticommunists (like myself in the 1970s) thought we were being judicious and fair-minded when we dismissed half of the Right’s complaint as crude blather. We were wrong; the Rosenbergs and Alger Hiss really were guilty, the Hollywood Ten really were Stalinist tools, and all of Joseph McCarthy’s rants about 'Communists in the State Department' were essentially true. The Venona transcripts and other new material leave no room for reasonable doubt on this score."

I urge all of you who have time this Sunday afternoon to read and consider A&D's piece below. Follow all the links and read them as well. Some of the comments are particularly on-point and trenchant. Read and consider what "complex adaptive system" you can link into to preserve your life, your liberty and your property.


A no-shit Sherlock

Escalating Complexity and the Collapse of Elite Authority

In yesterday’s New York Times, David Brooks wrote perceptively about the burgeoning populist revolt against the “educated classes”. Brooks was promptly slapped around by various blogosphere essayists such as Will Collier, who noted that Brooks’s column reads like a weaselly apologia for the dismal failures of the “educated classes” in the last couple of decades.

Our “educated classes” cannot bring themselves to come to grips with the fact that fundamentalist Islam has proclaimed war on us. They have run the economy onto recessionary rocks with overly-clever financial speculation and ham-handed political interventions, and run up a government deficit of a magnitude that has never historically resulted in consequences less disastrous than hyperinflation. And I’m not taking conventional political sides when I say these things; Republicans have been scarcely less guilty than Democrats.

In the first month of a new decade, unemployment among young Americans has cracked 52% and we’re being officially urged to believe that an Islamic suicide bomber trained by Al-Qaeda in Yemen was an “isolated extremist”.

One shakes one’s head in disbelief. Is there anything our “educated classes” can’t fuck up, any reality they won’t deny? Will Collier fails, however to ask the next question: why did they fail?

The obvious and most tempting hypothesis for a libertarian student of history like myself is that the Gramscian damage caught up with them. And I think there’s something to that argument, especially when the President of the U.S. more beloved of those “educated classes” than any other in my lifetime routinely behaves exactly as though he’d been successfully conditioned to believe the hoariest old anti-American tropes in the Soviet propaganda arsenal. And is praised for this by his adoring fans!

I think there’s much more to it than that, though. When I look at the pattern of failures, I am reminded of something I learned from software engineering: planning fails when the complexity of the problem exceeds the capacity of the planners to reason about it. And the complexity of real-world planning problems almost never rises linearly; it tends to go up at least quadratically in the number of independent variables or problem elements.

I think the complexifying financial and political environment of the last few decades has simply outstripped the capacity of our “educated classes”, our cognitive elite, to cope with it. The “wizards” in our financial system couldn’t reason effectively about derivatives risk and oversimplified their way into meltdown; regulators failed to foresee the consequences of requiring a quota of mortgage loans to insolvent minority customers; and politico-military strategists weaned on the relative simplicity of confronting nation-state adversaries thrashed pitifully when required to game against fuzzy coalitions of state and non-state actors.

Charles Murray and Richard Herrnstein argued tellingly in their 1994 book The Bell Curve that 20th-century American society had become a remarkably effective machine for spotting the cognitively gifted of all socioeconomic and racial backgrounds and tracking them into careers that would maximize their output. They pointed out, though, that the “educated class” produced by this machine was in danger of becoming self-separated from the mass of the population. I agree with both arguments, and I think David Brooks and Will Collier are pointing us at the results.

In retrospect, I think race- and class-blind meritocracy bought us about 60 years (1945-2008) of tolerably good management by Western elites. The meritocracy developed as an adaptation to the escalating complexity of 20th-century life, but there was bound to be a point at which that adaptation would run out of steam. And I think we’ve reached it. The “educated classes” are adrift, lurching from blunder to blunder in a world that has out-complexified their ability to impose a unifying narrative on it, or even a small collection of rival but commensurable narratives. They’re in the exact position of old Soviet central planners, systemically locked into grinding out products nobody wants to buy.

My readers might well ask, at this point, “Great. Does this mean we’re screwed?” If a meritocracy drawn from the brightest, best-educated people in history can’t cope with our future, what do we do next?

The answer is, I think implied by three words: Adapt, decentralize, and harden. Levels of environmental complexity that defeat planning are readily handled by complex adaptive systems. A CAS doesn’t try to plan against the future; instead, the agents in it try lots of adaptive strategies and the successful ones propagate. This is true whether the CAS we’re speaking of is a human immune system, a free market, or an ecology.

Since we can no longer count on being able to plan, we must adapt. When planning doesn’t work, centralization of authority is at best useless and usually harmful. And we must harden: that is, we need to build robustness and the capacity to self-heal and self-defend at every level of the system. I think the rising popular sense of this accounts for the prepper phenomenon. Unlike old-school survivalists, the preppers aren’t gearing up for apocalypse; they’re hedging against the sort of relatively transient failures in the power grid, food distribution, and even civil order that we can expect during the lag time between planning failures and CAS responses.

CAS hardening of the financial system is, comparatively speaking, much easier. Almost trivial, actually. About all it requires is that we re-stigmatize the carrying of debt at more than a very small proportion of assets. By anybody. With that pressure, there would tend to be enough reserve at all levels of the financial system that it would avoid cascade failures in response to unpredictable shocks.

Cycling back to terrorism, the elite planner’s response to threats like underwear bombs is to build elaborate but increasingly brittle security systems in which airline passengers are involved only as victims. The CAS response would be to arm the passengers, concentrate on fielding bomb-sniffers so cheap that hundreds of thousands of civilians can carry one, and pay bounties on dead terrorists.

Yes, this circles back to my previous post about the militia obligation. I’m now arguing for this obligation to be seen as, actually, larger than arming for defense (although that’s a core, inescapable part of it). I’m arguing that we need to rediscover CAS behavior in politics and economics — not because financiers or bureaucrats are dangerous or evil, but because even with the best will in the world they can’t cope. The time when they could out-think and out-plan the challenges of the day operating as an elite has passed.

The people who will resist the end of the engineered society, the managed economy and the paternalist state are, of course, the “educated classes” themselves. Having become accustomed to functioning as the aristocracy of our time, they will believe in anything except their own obsolescence as rulers. It remains to be seen how much longer events will permit their delusions to continue.


rexxhead said...

Dead-bang on target. Congrats to A&D for a one-shot-kill.

" the economy onto recessionary rocks with ... ham-handed political interventions"

'ham-handed' is redundant. Any political intervention, carried out long enough, will do exactly the same.

"planning fails when the complexity of the problem exceeds the capacity of the planners to reason about it."

And because complexity always rises, eventually, all planning will fail; therefore only an unplanned system (e.g.: the free market or other CAS) can actually 'work' over the long run.

jon said...

The “educated classes” are adrift, lurching from blunder to blunder in a world that has out-complexified their ability to impose a unifying narrative on it, or even a small collection of rival but commensurable narratives. They’re in the exact position of old Soviet central planners, systemically locked into grinding out products nobody wants to buy.

actually, the market for defense contracting is currently super-massive. we have nowhere else to go because a massive war machine has eaten research and engineering alive -- except of course for the occasional token esoteric doctorate study some senator will wave around in favor of budget cuts in aid to state universities -- but we're doing just fine, because it pays well. a 2.5% raise during a depression?

no problem.

far from complex or even adaptive, a gigantic statist tumor barely capable of third generation warfare seems to have pervaded the concept of national defense, and been placed squarely in the blind spot of big-government GOP voters. what is even worse in my opinion is that the christian majority therein abandoned the bulk of the bible's economic lessons in exchange for it.

entrepreneurs fail when they make decisions like that. whole societies will, too.

jimmyjet said...

I have a more cynical view.

For years the people who rule society (call them what you will- aristocracy, Bilderbergers, nomenklatura, Wall Street- the actual term is irrelevant) have hired and trained politicians, journalists, Generals and professors to ensure that they stay in power- political and economic.

Divide and conquer has always worked.

Only in a democracy do you get the government you deserve.

Until "the people" figure it out and say "Enough" and are willing to accept the personal sacrifice that entails (the whole "lives fortunes and sacred honor" concept) we will continue to get the government we deserve.

Hoping for a peaceful resolution, but keeping my powder dry...

Anonymous said...

Actually the author's explanation of the situation is a perfect example of the real problem: a lack of understanding of the real issues resulting in an overly complex non-workable theory.

A good example of what really is going on can be found by looking at France under Napoleon: he was a solider and he revolutionized the army and society overall by instituting a meritocracy: you get promoted for producing good results. However, he wasn't a sailor and left the navy organization pretty much intact to his ultimate downfall. For the navy was left to run under the old elitist rules: its not what you know but who you know and blow. French naval officer appointments were based on political contacts or nepotism; they were a bunch of incompetent unproven egotistical fops and fools and therefore the French navy, regardless of how many ships they had or the armament they carried, was never a match for the English navy.

Modern society's problems are due to the control being exerted by a corrupt power elite which strives to eliminate opposition by dumbing down the citizenry and making them controllable mindless egotistical fools: i.e. spoiled me-too sell outs who are put into positions of authority not because of merit but because they are obedient dupes. The so-called overly complex planning scenarios are the fantasy productions of these idiots.

People certainly do exist who are capable of developing very complex but workable organizational and development schemes - go back and look at the Liberty Ship production program, for a great example go read up on the man who built the Panama Canal, or take a look at what a few SF teams accomplished in northern Iraq and Afghanistan. But these projects were done by those who were intelligent, skilled, and responsible. They weren't a bunch of pampered idiot freaks. Pragmatism and meritocracy have always been the strength of our Republic, there are plenty of those still in the country who exemplify this but they are not in control, the freaks are, and until this changes things will just get worse.

De Oppresso Liber


III more than them said...

anon said......
"Pragmatism and meritocracy have always been the strength of our Republic, there are plenty of those still in the country who exemplify this but they are not in control, the freaks are, and until this changes things will just get worse."

Bravo!! Bravo!! I believe this sums up a great deal of what the country faces, and has faced. A strong moral code for work, business and life has a difficult time facing the power-hungry evil that besets it. Its answer, is something that the same moral people do their best to avoid - CONFLICT. Yet, it is the moral, seeking a good life, one filled with Liberty and opportunity (not equality of outcome), that will have to rise to the call to eject the evil, by armed force if necessary. "We wish to live in peace and liberty. But, when those are no longer possible, we will consider dying to restore them. What say you?"

straightarrow said...

Horse apples! He used a Hell of a lot of words to entirely miss the point.

We are where we are because the people manning the steering mechanisms intentionally ignored things they knew to be true because observance of things irreducible that contradicted their desires contradicted their desires. It ain't any more difficult than that.

They didn't want it to be true, so it wasn't. When the failures hit, it was the fault of everyone except those who caused it. Remember? They still have the steering mechanism.

It wasn't because the complexities overwhelmed. Hell! Every 8th grade dropout could have and some did predict the outcome.

If someone wants to be gracious and allow the comfort of falsely saying our present situation was practically unavoidable because the complexities outran the ability to assess and adjust, ok. But it isn't true.

I favor telling the truth, removing these people from the equation, burning all documentation of the bona fides,and allowing them to only work picking up litter on the highway.

Anonymous said...

Chess has complexity.

We have something that's evolving into Mussolini-style corporatocratic fascism.

I don't think complexity is our primary impediment.


Sean said...

Blah, blah, blah, the something this, and that. Bad, good, decide.

Anonymous said...

Even more cynical?
I believe we always get the government we collectively deserve.
Not until we have sufficient numbers taking responsibility for themselves, will be deserve a decent government.