We like to think that our lives are ordered, predictable and subject to a great deal of control. The past is finite; we see only one outcome. We attach causality and narrative to it so that it makes sense. We roll our ability to make sense of the past over into the future, which is infinite; there are many outcomes, as yet unknown and unknowable. Randomness, chance, and luck influence us far more than we realize. Certainty is an illusion. Uncertainty is everywhere.
Let’s explores three fundamental truths about uncertainty (click the links to jump to a section):
- The world is inherently uncertain
- The world is getting more uncertain
- Humans crave certainty (and deplore uncertainty)
The world is inherently uncertain
Quantum mechanics and general theory tell us a lot about how the world works. Quantum mechanics deals with phenomena at the extremely small, particular, scale. General theory describes the force of gravity and the large scale structure of the universe.
The Uncertainty Principle: what quantum mechanics tells us about uncertainty
We used to think the world was entirely predictable:
The success of scientific theories, particularly Newton’s theory of gravity, led the French scientists the Marquis de Laplace at the beginning of the 19th century to argue that the universe was completely deterministic. Laplace suggested that there should be a set of scientific laws that would allow us to predict everything that would happen in the universe, if only we knew the complete state of the universe at one time. For example, if we knew the positions and speeds of the sun and the planets at one time, then we could use Newton’s laws to calculate the state of the solar system at any other time. The doctrine of scientific determinism was strongly resisted by many people, who felt that it infringed Gods freedom to intervene in the world, but it remained the standard assumption of science until the early years of this century.A Brief History of Time: From the Big Bang To Black Holes (Amazon UK, US)
In 1926, a German scientist called Werner Heisenberg used quantum mechanics to blow up Laplace’s theory. Investigating the smallest particles, he found that:
In order to predict the future position and velocity of a particle, one has to be able to measure its present position and velocity accurately. The obvious way to do this is to shine light on the particle. Some of the waves of light will be scattered by the particle and this will indicate its position. However, one will not be able to determine the position of the particle more accurately than the distance between the wave crests of light, so one needs to use light of a short wavelength in order to measure the position of the particle precisely. Now, by Planck’s quantum hypothesis, one cannot use an arbitrarily small amount of light; one has to use at least one quantum. This quantum will disturb the particle and change its velocity in a way that cannot be predicted. Moreover the more accurately one measures the position, the shorter the wavelength of the light that one needs and hence the higher the energy of a single quantum. So the velocity of the particle will be disturbed by a larger amount . In other words, the more accurately you try to measure the position of the particle, the less accurately you can measure its speed, and vice a versa.
Heisenberg showed that the uncertainty in the position of the particle times the uncertainty in its velocity times the mass of the particle can never be smaller than a certain quantity, which is known as Planck’s constant. Moreover, this limit does not depend on the way in which one tries to measure the position or velocity of the particle or on the type of particle: Heisenberg’s uncertainty principle is a fundamental inescapable property of the world.A Brief History of Time: From the Big Bang To Black Holes (Amazon UK, US)
The Three Body Problem: what General Theory tells us about uncertainty
Ben Hunt is the creator Epsilon Theory, a newsletter and website that examines financial markets through the lenses of game theory and history. He explains the Three Body Problem:
Imagine three massive objects in space … stars, planets, something like that. They’re in the same system, meaning that they can’t entirely escape each other’s gravitational pull. You know the position, mass, speed, and direction of travel for each of the objects. You know how gravity works, so you know precisely how each object is acting on the other two objects. Now predict for me, using a formula, where the objects will be at some point in the future.
Answer: you can’t. In 1887, Henri Poincaré proved that the motion of the three objects, with the exception of a few special starting cases, is non-repeating. This is a chaotic system, meaning that the historical pattern of object positions has ZERO predictive power in figuring out where these objects will be in the future. There is no algorithm that a human can possibly discover to solve this problem. It does not exist… What Poincaré proved is that there is no formula where you can plug in the initial information and get the right answer for where any of the objects will be at any future point in time. No human can predict the future of this system.
Connecting the dots of the Universe
Scientists have been searching for a Theory of Everything, or Grand Unified Theory, for how the universe works for decades, to no avail. Even if they were to discover one, Hawkings maintains that it would still not mean that we would be able to predict events in general, for two reasons.
The first is the limitation that the uncertainty principle of quantum mechanics sets on our powers of prediction. There is nothing we can do to get around that. In practice, however, this first limitation is less restrictive than the second one. It arises from the fact that we could not solve the equations of the theory exactly, except in very simple situations. (We cannot even solve exactly for the motion of three bodies in Newton’s theory of gravity, and the difficulty increases with the number of bodies and the complexity of the theory).
If we can’t predict anything at the quantum level, and we can’t predict anything at the general (theory) level, what would suggest that we can predict with any great accuracy what we’ll be doing in three days or three months time? Or, what interest rates will be next year? Whether the S&P 500 will continue to climb or crash? The price of Bitcoin or Ethereum in one year from now? As sure as the sun rises in the East and sets in the West, because the earth revolves around it, the world is an inherently uncertain place.
Next time you find yourself craving certainty, it’s worth reminding yourself that uncertainty is an inherent part of the natural order.
The world is getting more uncertain
We know from chaos theory that even if you had a perfect model of the world, you’d need infinite precision in order to predict future events. With sociopolitical or economic phenomena, we don’t have anything like that. And things are getting worse, not better, because the growing complexity of the world dwarfs any improvement in sophistication or computational power.“Always expect the unexpected”, an interview with Nassim Nicholas Taleb in Wired
Uncertainty is a function of the complex adaptive system in which we live: “a system in which a perfect understanding of the individual parts does not automatically convey a perfect understanding of the whole system’s behavior”. In complex adaptive systems, the whole is more complex than its parts, and more complicated and meaningful than the aggregate of its parts.
The difference between Complicated and Complex
Understanding the important but subtle difference between something that is complicated and something that is complex is important. General Stanley McChrystal, commander of the Joint Special Operations Task Force in Iraq in 2003 explains:
Being complex is different from being complicated. Things that are complicated may have many parts, but those parts are joined, one to the next, in relatively simple ways: one cog turns, causing the next one to turn as well, and so on, the workings of a complicated device like an internal combustion engine might be confusing, but they ultimately can be broken down into a series of neat and tidy deterministic relationships; by the end, you will be able to predict with relative certainty what will happen when one part of the device is activated or altered.
Complexity, on the other hand, occurs when the number of interactions between components increases dramatically – the interdependencies that allow viruses and bank runs to spread; this is where things quickly become unpredictable. Think of the ‘break’ in a pool game – the first forceful strike of the coloured balls with the white cue ball. Although there are only sixteen balls on the table and the physics is that of simple mechanics, it is almost impossible to predict where everything will end up. In a perfect world, with an impossibly level table, balls that were identical down to the micron, and a player who could strike with the precision of one millionth of a degree, a computer could foresee where the balls would slow to a halt. However, introduce even the slightest deviation in the trajectory of a single ball, and quickly all the balls that it touches, and all the balls that they touch, will diverge. The density of interactions means that even a relatively small number of elements can quickly defy prediction.Team of Teams: New Rules for Engagement in a Complex World (Amazon UK, US)
Interactions and interdependencies are key. Complex systems are entangled systems, made up of an ever increasing number of nodes and connections within the system – population increases, road, bridges, rail, sea and air routes, and of course, the internet, cellular and satellite networks. As we increase the number of nodes and connections in system, we increase the number of interactions and interdependencies. In turn, this increases complexity and an increasing lack of certainty. McChrystal reiterates the point:
In a complex system with dense interconnections one would need an impossible resolution of data in order to make reliable medium to long run predictions. There are causes for the events in a complex system, but there are so many causes and so many events linked to one another through so many direct and indirect paths that the outcome is practically unpredictable, even if it is theoretically deterministic.
This is best demonstrated by Metcalfe’s Law, which states that a network’s value is proportional to the square of the number of nodes in the network. Because every new node in the network connects with every pre-existing node, as you gain nodes, you non-linearly increase the number of connections that each node has with the other.
Everthing’s obvious, in hindsight
Importantly, in complexity, because of the ambiguous relationships between the connections in the system and cause and effect, we can only determine why something has happened after the event. As Dave Snowden, creator of the Cynefin framework, explains:
In a complicated context, at least one right answer exists. In a complex context, however, right answers can’t be ferreted out. It’s like the difference between, say, a Ferrari and the Brazilian rainforest. Ferraris are complicated machines, but an expert mechanic can take one apart and reassemble it without changing a thing. The car is static, and the whole is the sum of its parts. The rainforest, on the other hand, is in constant flux—a species becomes extinct, weather patterns change, an agricultural project reroutes a water source—and the whole is far more than the sum of its parts. This is the realm of “unknown unknowns,” and it is the domain to which much of contemporary business has shifted.A Leader’s Framework for Decision Making, Harvard Business Review
Humans (and media commentators especially) are excellent at attempting to reduce apparent uncertainty by attaching post hoc narratives to events (something Nassim Taleb calls the narrative fallacy). But the reality is that as we increase the number of nodes and connections into our global systems, we are increasing its complexity, its non-predictablity, and hence its uncertainty.
Humans crave certainty (and deplore uncertainty)
We don’t like uncertainty or the unknown. We need to categorize, classify, organise and structure the world. Categorising ideas and objects helps us to recognise, differentiate and understand. It simplifies life. To understand and control our environment helps us to deal with the future. We want to know how and why things happen and what is going to happen in the future.Seeking Wisdom: From Darwin to Munger (Amazon UK, US)
Evolution has seen to it that humans crave certainty and deplore uncertainty. Our ancestor’s ability to predict the whereabouts of local tigers (so as to avoid them) or wooly mammoths (so as to to be able to hunt, kill and eat them) significantly increased their chances of survival.
Whilst ‘knowing’ what the future holds feels reassuring, uncertainty has the opposite effect. At a neurobiological level, our limbic brain translates uncertainty into a strong threat or alert response, at the expense of our neocortex, the thinking part of our brain. This is our Fight, Flight, Freeze response and it has an awful lot answer for! When triggered our feelings range from general uneasiness to outright anxiety and panic. Our ability to focus and make good decisions is impaired. We literally embody uncertainty.
This ancient inner limbic part of our brain served us our ancestors well. In their more simple world, it kept them alive by always looking out for threats and responding to them quickly. Even if they misinterpreted a threat, it was better to be wrong and stay alive than risk it and not get a second chance. But in our modern, complex world, these evolutionary habits can be problematic. Whilst we still need to be able to respond to genuine threats, sub-consciously our brain is often processing and responding to more modern day challenges in the same way as our ancestor might have done hundred’s of thousands of years ago when the threat was far more significant.
Our wiring presents a paradox. Designed by evolution to protect us from harm in uncertain situations, it is now often over-triggered as we try to navigate a world that, as we’ve discovered, is inherently uncertain. What gives? According to Jennifer Garvey Berger:
Because there’s no way of knowing what’s next, we are always walking forward with our hands out in the dark, waiting to bump into things. And because things are changing, we have lost much of the ability to predict what will happen next from what has happened before, to pull out the memories from other dark rooms we have bumped through in the past. Complexity is about getting our heads around what is possible (because anything could happen) rather than what is probably going to happen (which is determined from what has happened before).
This shift – from trying to get your head around what is most likely, to trying to get your head around what is in the field of possibilities – is much bigger than it sounds . As research has shown in study after study, at brains just don’t like this. A general pattern is to prune and simplify. We need to work at it if we are going to create new patterns of behaviour for thinking and acting in this world.Simple Habits for Complex Times: Powerful Practices for Leaders (Amazon UK, US)
Uncertainty is not comfortable, but it is inherent, and increasingly so. To navigate this uncertain and complex world, we need to develop our ability to ‘let go’ of certainty and ‘hold’ uncertainty in ways that don’t over-ride our senses but rather allow us to turn it into an opportunity to deal with it in more thoughtful and constructive ways.
If you enjoyed this, you might like:
Slate Star Codex’s Book Review: Surfing Uncertainty summarises the Predictive Processing model of how the brain works: “The key insight: the brain is a multi-layer prediction machine. All neural processing consists of two streams: a bottom-up stream of sense data, and a top-down stream of predictions. These streams interface at each level of processing, comparing themselves to each other and adjusting themselves as necessary”.
How Vertical Development helps new leaders truly transform explores the difference between horizontal development, which focuses on what you know, and Vertical Development, which concerns how you think.
Want to be a better tech leader? It’s complex. explores the difference between complicated and complex, sensemaking tools for building and leading businesses in a complex world, and the necessary shift from informational to transformational learning as your business and leadership responsibilities grow.
A recording of my livestreamed YouTube conversation, discussing the difference between Complicated and Complex.
When it comes to predicting the future “the average expert is as accurate as a dart throwing chimpanzee”. But there are some ways that we can improve. Read my summary of Superforecasting: The Art & Science of Prediction, by Philip Tetlock.