October 8, 2019 ☼ Notes
These are my notes from Jim Rutt’s excellent podcast with David Krakauer. Errors and omissions are my own. All credit goes entirely to David and Jim.
David Krakauer is the President of the Santa Fe Institute, where he studies the evolution of intelligence and stupidity in the universe. Jim Rutt is the host of the Jim Rutt Show and Chairman of the Santa Fe Institute.
The Santa Fe Institute was founded in the mid-80s to apply the mathematical methods developed in theoretical physics and computer science to solve problems in the “complex” social and biological sciences. Complexity science is the domain of adaptive phenomena. The components of a complex system adapt to their environment over time. Examples include neurons in a brain, birds in a flock, and traders in a stock market.
Reductionism is possibly the most useful metaphor in science. It is based on the idea of linearity: an object is the sum of its parts. If you can understand the parts, you can understand the object completely. Reductionism, begging with Newton, has allowed us to understand systems including the planets orbiting the sun, to the interior of an atom.
Reductionism cannot get a handle on collective behaviour. Consider water — you can work out lots from a single H2O molecule, for example, that it is a polar solvent likely to form hydrogen bonds, but it is impossible to understand the phases of water without collections of H2O molecules.
Emergence is the flip side of reduction. It explains how components come together and exhibit collective properties, which cannot be predicted by only looking at the components themselves. The whole is more than the sum of its parts.
Jim’s metaphor: “Reductionism is the study of the dancers while complexity is the study of the dance.”
Some fields have complete, reductionist theories of emergence, e.g thermodynamics and superconductivity. We can go all the way up the explanatory chain from the atomic level to the macro level and back down again.
In David’s interpretation (as distinct from Dave Snowden’s), every part of a complicated system obeys the classical laws of physics. The elements are all stationary, not adapting or evolving in time. The moon doesn’t get better at orbiting the sun because its been doing it for 4 billion years.
Complex systems are marked by extensivity. As the system gets larger, the mathematical description of the system must grow too. This is due to the adaptation of the constituents of the system. Each adaptive unit of a given size requires a certain description length. So if you add more units, the description size of your system must also increase.
Complexity is related to deterministic chaos. A chaotic system can be generated by very simple equations (for example, a double pendulum). The output of a chaotic system (a record of the states of the pendulum as time goes on) is pseudorandom, so requires a long description length.
Consider the following example: I decide I want to take a sip from my coffee cup. That thought, a mental construct, is translated into the activity of billions of neurons as I reach for my cup and take a sip. This seems to imply that consciousness, a high-level process, dictates lower level processes. But the initial idea of wanting to take a sip of coffee is already encoded in billions of neurons across your brain. Humans only have access to a low dimensional representation of this, which we call the thought.
Top down causality is the idea that coarse-grained, higher level phenomena (our thoughts) can influence bottom-up phenomena (individual neurons). But this is mistaken because any coarse-grained phenomenon is only a representation of a bottom-up phenomenon (our thoughts are the only representation of interactions between billions of neurons we have access to). Our intuitions mislead us.
“Most of us are amateur Sherlock Holmes-like thinkers”. We try and tease causes out of effects. We look for patterns. We look for simple causality, which is useful when it exists but is very often a mirage.
Complex causality is fundamentally different. Imagine you want to understand a matrix of components which are all highly interrelated. You need to integrate over all components across space and time. This is multi-causal. There is no one component which causes anything else. It is the relationship between components which is the cause, and it is impossible to tease them apart and make predictions based on a change in any particular component.
Humans, with our love for patterns and order, crave simplicity. But when it comes to understanding cancer, or the state of the economy, there isn’t a simple solution. These domains are inherently complex.
Machine learning has become so popular because it can take into account all the degrees of freedom of high-dimensional systems.
“There is this intriguing allure of the 1-dimensional, monocausal theory everywhere. It’s in the economy, where they call it ‘price’, and its in psychology, where they call it ‘IQ’.” There is a human urge to build theories around the simplest possible explanation. This works well for celestial mechanics, but often it doesn’t give you purchase on all the relevant variables in the system. This is “cargo-cult science”, a theory that seems scientific, but has no predictive power.
The entropic view of time: there are more ways to be wrong than be right. There are more ways of breaking an egg than making an egg. Time is irreversible because things decay, and order crumbles.
The Darwinian view of time: there is creation of order in open systems by natural selection or learning, which occurs sequentially over generations. Darwinian selection leads to an increase of order with time.
Combining the two pictures gives you “complex time”. This tells you how long complex entities can fight off entropy. It gives you new temporal phenomena like the “lifespan” of an organism, or a city.
Complexity balances the entropic systems which lead to disorder, and the selection processes which lead to order. The tension between these two gives the size of order, and the lifespan of ordered entities.
In very competitive systems (those with a tough selection mechanism), where both parties are nearly evenly matched, the distribution of lifespan is expected to be random. If the lifespan of companies are randomly distributed, it is strong evidence for efficient markets. If not, it is evidence for monopoly power. In general, the lifespan of companies and organisms follow an exponential distribution.
Cities, unlike organisms, do not follow an exponential distribution in lifespan.This suggests they are not competing against one another like organisms are competing for food and companies are competing for market share (amongst other things). Jim suggests cities have a fundamental monopoly on space. Other cities cannot steal it, which leads to their persistence in time.
If you give a cell a superabundance of resources, its ordering processes will beat its disordering processes. The cell has more than enough resources to correct copying errors, so it becomes immortal — it “beats” entropy.
Lots of empirical prediction, people understand the “what”, without any fundamental understanding of the “how” or “why”.
We can’t model humans like billiard balls — there is richness in social sciences which no existing theories can explain. As a result we have seen a massive uptick in statistical models, which make empirically useful predictions but are completely opaque to human reason. You cannot ask them how they made a particular decision, or to clarify their assumptions.
Growing out of game theory and non-linear dynamics is a third way between Newtonian determinism and the statistical approach — describe phenomena at an aggregate level (e.g. the organism, the firm, the city). This allows understanding at the collective level, although it forsakes microscopic prediction.
There is a schism between those who are content to be told what the world will do (machine learning / statistical approach), and those who want to understand what the world does (theoretical physics / complexity science approach).
An AGI would be able to make predictions based on bottom-up elements, but also have understanding of the macroscopic features of a system. An AGI which understood human consciousness would be able to predict human actions based on knowledge about indvidual neurons, and would also understand those actions in terms of higher-level concepts like thoughts.
David believes there is a fundamental tradeoff. You can either take the coarse grained path, and forfeit the microscopic degrees of freedom in favour of high-level understanding, or you go follow the machine-learning approach, and get accurate predictions from all the degrees of freedom without understanding how they interrelate.
David claims there will be an uncertainty principle in complexity science akin to Heisenberg’s uncertainty principle in physics. The uncertainty principle of prediction: if you can do high-resolution prediction, you cannot explain it, because you need complex causality which cannot be understood.
He studies machines that manifest intelligence - these are varied and include brains, polities, societies, genomes and more.
Intelligence should be differentiated from ignorance and stupidity. Intelligence is making hard problems easy. It is establishing a mechanism or rule-system which allows you to efficiently arrive at a correct outcome. Like knowing the “god-algorithm” which allows you to solve a Rubik’s cube in 24 moves or fewer. Ignorance is simply insufficient data to reach a conclusion. Stupidity is the application of rules which do not guarantee a correct answer, in infinite time. Stupid performance is worse than random.
David classifies rule-systems throughout the history of human culture. He asks questions like “How well does a cell follow a gradient? Is it truly optimal?”; “How well does an economy clear a market?”.
David believes our anthropomorphism has driven us to put intelligence into a tiny box, IQ, which misses many other complex factors. David believes intelligence is multi-causal and a single clear explanation is wishful thinking.
He was trying to answer the following question: If you are in a nonequilibrium system, where the second law is operating, how do you maintain sophisticated states of order?
One problem in complex systems is conflict - when agents have imperfectly aligned strategic incentives. Conflict occurs in all complex systems because there is no global common knowledge. Each agent has its own window into the world, which leads to misaligned strategic objectives (even if agents don’t intend to disagree).
Policing is a robustness mechanism which preserves a system of order. With true impartial policing (no bias), you can maintain complex states of order with very few police. A hallmark of a well-ordered society is one where the number of police is small relative to the size of the society. Consider modern Japan vs. East Germany in the 1960s.
In decentralised policing (where policing is not controlled by a central authority like government) you also need “second order” police, who will punish the non-punishers. There has to be a cost to not-policing (this is a free-rider problem).
David tried to unpick the mechanisms that culture has for propagating good (and bad) ideas into the future. He wanted to know how a structured set of codes (like a legal constitution) are transported into the future, and how do these codes evolve over time.
The idea of a meme was conceived in analogy with a gene. David’s work showed there are units of meaning which are propagated in time, but they are far more fluid than genes. For this reason, the idea of a meme is less valuable than a gene, because memes are less robust.
Many of society’s ills are due to our collective desire to simplify unnecessarily. The economy and society are complex systems, and any policy taken has tradeoffs associated with it. Keeping all of these in mind is exhausting and humans are prone to massive simplification as a result. The gateway drug to complexity is thinking in terms of probabilities. Thinking about political issues not as binary left and right divides but as multi-dimensional optimisation problems is a first step to applying complexity to real life.
One of the reasons complexity science is interesting is that it justifies a system of folkloric belief. Folklore and old wives’ tales evolved with society, and they have more knowledge about human nature embedded within them than we might expect.
David is pessimistic about schools but optimistic about the world of gaming. School is becoming more reductive: teachers are consistently reducing the scope of lessons to focus narrowly on standardised tests. In contrast, game designers are putting highly rich, complex phenomena into their games. Young kids probably learn more playing Minecraft today than in the classroom.
Work at the Santa Fe Institute affects the world in several ways:
David recently released a book of essays called Worlds Hidden in Plain Sight which I highly recommend.