Once upon a time I liked to derive lessons from stories; sometimes I still do. A recurring lesson is the impact of minor mistakes and random circumstances on the character’s fate. If only they stayed quiet, or paid more attention, or used a newer car, or avoided traffic, or were less trusting… Mistakes that were easy to avoid with just a little more knowledge or composure or preparation.
Of course, the writer needs to end the story and villain’s mistakes allow difficult odds to be overcome in an exciting way. But there are deeper reasons for errors: individual pride, complacency of experience, overconfidence of inexperience, effects of stress on decision-making, trade-offs inherent to skill-sets.
We focus on mistakes as failures to prepare at the expense of a broader lesson: “The best laid plans of mice and men often go awry.” Leaders respond with deeper plans, contingency plans, redundancy, more data, explicit procedures, education… They develop meta-plans to make plans responsive to the dynamics of execution, they build ever-more complete ways to evaluate results and feed them back to decision makers at ever-faster rates.
But more complex plans have more ways to fail, redundancy breeds complacency, contingency plans are under-tested, procedures reduce individual motivation and attention to detail, credentials give a false sense of confidence… And problems again arise for reasons so banal they would be funny if they weren’t so painful.
The Madness of Doing
Of course, plans are valuable, better plans confer an advantage, and dynamic plans help manage uncertainty. A master strategist may be unbeatable by a sufficiently poor one. But still: “no plan survives first contact with the enemy.”
In the military – where this famous quotation originated – nature can spoil the best plans with weather and disease. Officers can ruin them with miscommunication, personal grudges, too much or too little initiative in interpretation of directives. Or with an ill-timed bout of diarrhea, a bad omen, a fight with a spouse or a friend. Or by tripping at the wrong time, being distracted by an itch or frazzled by an exploding shell. And then the soldiers must follow officer’s orders even in situations that seem hopeless, even after witnessing the demise of others, after seeing errors sure to foil the plan, after getting hurt, after tripping at the wrong time. Success depends on their ability to maintain their composure, retain their faith, and perform to their abilities in the madness that is battle.
Or more broadly, in the madness that is doing – for it isn’t specific to the military. It’s difficult to believe how little can be assumed about quality and success of execution without experiencing it. This alone accounts for much of the disconnect between theoreticians and practitioners, young and old, amateurs and professionals.
Employees show up late, drunk, or not at all without warning. Customers ignore even the most conspicuous text. Stores can’t prevent employee theft. Drivers don’t change the oil or check tire pressure. People lash out with no warning or explanation. Doctors and nurses forget to wash their hands. And this is after intelligent, highly-trained strategists have tried to solve the problem. This list can be expanded until one becomes amazed that our complex civilization ever functions.
When it comes to execution, nothing is simple or obvious enough to be taken for granted. Every step, however trivial, can cause errors. Every assumption can prove wrong. Every reliance on common sense or domain knowledge or experience or loyalty can backfire – as can failure to rely on these things. Execution is hard, but it’s hard for banal reasons. Mistakes seem trivial and preventable. Explanations of complexity seem like excuses.
Idealistic strategists believe that execution will follow smoothly from well-made plans with enough preparation and proper systems – or at least smoothly enough to make strategy the limiting factor to success. And we want to believe them.
We blame failures on anything and everything we can think of – from the conspiracy of the powerful to the immorality of the lazy – with even the most improbable explanations being easier to accept than the banal reality that people cannot be relied on to follow the simplest of instructions. Or that those we want to help can disagree with, even sabotage our efforts. Or that the product of our best intentions may help promote the ends we despise.
But a little introspection should reveal that we routinely fail at trivial tasks, even on the tiny scale of our lives. We set our alarms wrong, forget appointments, drive drunk, promise that we’ll go to the gym or stop smoking – tomorrow. We get into fights with strangers, roommates, even friends and family that we aren’t able to resolve: we just move apart or agree to disagree or hold grudges. We ignore instructions and act on assumptions. We get so overwhelmed by forms and choices that we give up on simple tasks, give up or pay to make them go away. We have conversations where we can’t get our point across regardless of how obvious it is or how hard we try; eventually we just throw up our hands in disgust – or perhaps post a recording to show our incredulity.
These difficulties are ever-present, but there is always an easy scapegoat: that person is unreasonable, that company is selfish, that clerk is stupid, we had a long day and deserve a break. We move through life soaked by flaws, trade-offs, excuses, and misunderstandings, but we don’t connect these everyday imperfections with the challenge of turning plans into results. We get out of traffic jams caused by irrational habits, walk into meetings where we can’t pry people from their smartphones, much less their thoughts, and – without a hint of irony – present plans that, with everyone’s support, will fix our group, nation, world.
We somehow convince ourselves that problems are a result of insufficient understanding that explanations can correct, or local imperfections that can be escaped elsewhere, or flawed systems that can be fixed at higher levels.
But at all times and in all places explanations and systems have to deal with similarly flawed people and organizations: the ones who are unreasonable, foolish, selfish, backwards, unorganized, confused, and overwhelmed. Large systems just have to deal with more of them, with fewer options to run away or make excuses.
An overwhelming portion of our institutions exist because of banal problems and limitations. These components don’t directly serve the institution’s mission which makes the entire organization appear inefficient or old fashioned or unnecessary or confining or even immoral. Policies seem incompetent, governments corrupt, companies undeserving of their profits, markets unnecessary, leaders evil.
But on a daily basis we sacrifice correctness for convenience, make mistakes, create new limitations and deal with old ones. Our institutions are built for us, by us, and with us. We are their inputs, outputs, gears, and building blocks. We are their customers, partners, employees, regulators, and competitors. We come in all varieties with our own physical, intellectual, and emotional limits, with our own lives that drain those limits. Our limitations permeate our institutions just like they permeate our own lives. They can’t escape them until we do, and we’d rather wait until they make improvement trivial or limitations irrelevant. As Walt Kelly said, “we have met the enemy and he is us.”
The Achievement of Goals
To just make progress towards established goals is an accomplishment. Most organizations, and most people, fail long before they reach the limits imposed by their plans and capabilities. They simply get overwhelmed by all the steps, inefficiencies, limitations, and mistakes: they give up. They excuse their difficulties as exceptional, even when they are ordinary. Successful people and organizations don’t so much overcome banal realities as keep working and making trade-offs until gains become large enough to overshadow the setbacks. Or at least appear to be large enough: low probability, high-cost mistakes love to hide in trade-offs.
It isn’t that there aren’t better and worse ways to do things or that plans don’t influence success or that mistakes cannot be reduced in quantity, magnitude, and impact. Our hopeful intuitions have much merit: we can improve and we can avoid banal problems. We just can’t all improve in every way and we can’t avoid all the problems. This, hopefully obvious, caveat makes all the difference to the proper scope of action and expectations of success.
There are four broad sources of mistakes: (1) our own limitations, (2) misunderstandings, (3) limitations of others, and (4) randomness that stresses the other three. Innovative organizations derive their competitive advantage significantly from their superior management of these factors.
They tend to be led by people who are clear about their goals and serious about self-improvement. They select people with fewer limitations that affect execution. They educate them and give them the tools they need. They automate, systematize, and backup relentlessly to reduce common sources of errors and to incorporate the lessons of past mistakes.
Capable people with a clear mission have fewer misunderstandings. Effective organizations reduce them further with agreement on core values, expertise in communication, well-thought-out plans, easy feedback mechanisms, proper incentives, and general goodwill.
They limit the impact of others with interface procedures, with higher demands on partners, with redundancy, with systems that verify the work of outsiders and limit the impact of their mistakes.
With fewer limitations and misunderstandings, with more clarity and goodwill, with built-in learning mechanisms, such organizations face less stress and are much better equipped to deal with uncertainties thrown by fortune. They improve still further with experienced people who’ve proven themselves adapt at the dynamic nature of execution.
While mistakes occur, they are smaller in number and impact. They tend to be learned from, to make such organizations stronger. These organizations become more effective than their competitors and make impressive progress towards their goals, progress that often goes substantially according to plan.
Variations of this approach are at the root of many successes. We see them as evidence of superiority of specific values, skills, and systems – and as proof of efficacy of plans. We want to broadly promote these values and skills, apply the system on a larger scale, or expand the approach to other domains. The founders are proud to be recognized and provide us with explanations and blueprints. But our efforts fail.
The Limits of Scalability
The approach doesn’t seem to generalize past specifics like the vision of founders, skills of employees, support of early customers, freshness of systems, and limited levels of external pressure. It didn’t eliminate mistakes, misunderstandings, and compromises as much as reduced freedom and scope until the differences that drive them became small enough to be manageable.
Founders selected employees who fit a specific set of skills, values, and visions. Whether consciously or not, the organization also selected customers who fit its solution and were tolerant of the specific approach of bringing it about. The novelty of the solution allowed the organization to limit or control its interactions with others. And so on.
While plans and skills were essential, so was the limited scale. The organization effectively built a mini kind-of-perfect world – an impressive and useful achievement, but not one that can be generalized.
What’s worse, not only do external attempts at application fail, the successes that inspired them also do not last. Our shinning stars burn out, retreat, or become stalwarts, reminiscent of the incumbents they replaced.
It turns out that banal problems were suppressed, not conquered. They continued to hide behind every choice, to expand their sphere of influence with every new employee, feature, customer, and interaction. However small the probability of error, with enough time mistakes are certain: B players get hired, passions flair, sloppy work gets done. The organization descends down the slippery slope.
The slow build-up of imperfections and the growth of the mistake surface area can be managed for a long time, at least in theory. What accelerates disintegration beyond control are scalability limits: external forces and constraints that increase in influence as the organization grows.
Most obviously, there are only so many people, so much time, so much space, and so much money. The more you require from employees, partners, and customers, the fewer of them there are and the more expensive and time-consuming they are to reach. Once you get large enough – or decide to move cheaper or faster – you must loosen the reigns.
Less obviously, as you grow you increasingly have to deal with people who use your solution in unintended ways, regulations that limit your freedom, groups that oppose your means or goals, competitive pressures, fraud, and those who treat your solution as a right. These challenges are primarily a function of the percentage of the world you interact with. You have little control over them, and they conflict not only with your axioms, plans, and systems, but also among themselves.
Finally, the specifics of your approach depend on the scale you work at. What works between trusted friends fails between strangers. What works with early adopters fails with the general population. What works with people of similar education, or social status, or nationality fails as you become more inclusive. Processes and cultures of small organizations fundamentally cannot be the same as those of large organizations. You are forced to adjust your approach and to do it under pressure and constraints inherent to a living organization.
The limits of scalability force trade-offs which accelerate the already growing pressure of banal issues. An expanding organization has to deal with an ever-larger percentage of the universe, with an ever-larger percentage of flaws, opinions, pressures, and constraints. It has to compromise, patch, and change even if these actions undermine its own foundations.
Many effective actors disintegrate at this stage. They see their success as a function of absolute truth rather than of limited scope and either search for grander truth or rally against differences that prevent their truth from being accepted – both in vain.
Those who successfully navigate this transition gain expertise in management of pragmatic problems of scale and get to continue to grow and exert influence, but at a heavy price: their foundations have been irreversibly weakened. They are now exposed to attacks from new upstarts based on tiny perfect worlds, idealistic founders, and newest technologies. The pressure of past compromises pushes from the inside, the indifferent world bashes from the outside, and their former ally – creative-destruction – pounds on the cracks in their foundation.
The Force of Change
We feel surprised, disappointed, even deceived by these adjustments, whether with dedicated individuals, opinionated startups, idealistic political parties, successful companies, or impactful non-profits. Critics point out all the inconsistencies as evidence of incompetence, ill intent, and immorality. We cast blame and put our hopes in the next rising star.
But the weight of banality and the pressures of scalability assure that at no time, save perhaps for the very early stages, can all the actions be consistent with all the visions and strategies. Fast change, imperfect people, incomplete information, and conflicting demands lead to ad hoc, imperfect systems regardless of intent. Systems which then themselves place sub-optimal demands on other people and organizations.
But while this cycle – by which idealistic upstarts out to change the world turn into jaded incumbents that impede progress – isn’t preventable, action is not futile either. Change occurs, and occurs substantially because of attempts at improvement: truths are proven, possibilities are made real, standards are raised. The competitive landscape adjusts to this new status quo, which – like those before it – seems as luxurious to the generation that created it as it seems inadequate to the generation that is born into it.
These efforts are always open to criticism because it isn’t actually possible to act without such exposure. Critics see easily avoidable imperfections and talk as if all actions could have been perfect, when the banal nature of errors assures a near infinite number of possible mistakes, while compounding effects give the most trivial of errors the potential to become catastrophic. Constraints and limited information force even the most prepared actors to make probabilistic decisions which permit undesirable consequences and can be interpreted as mistakes, should the potential costs turn real. The most astute critics, who recognize the necessity of trade-offs, can always challenge the models used to make decisions: there is, after all, no shortage of theoretically coherent alternatives.
Change doesn’t follow from well-laid plans and this scares us. We wonder whether it’s progressive, stable, or worth the costs, both obvious and hidden. But we are no more able to prevent attempts to build better worlds than we are able to plan them. They are the driving force of change, however different the manifestation of this force is from the hopes of those who unleash it.