Fast Takeoff in Biological Intelligence

[Speculative and not my area of expertise; probably wrong.]

One of the possible risks of artificial intelligence is the idea of “fast” (exponential) takeoff – that once an AI becomes even just a tiny bit smarter than humans, it will be able to recursively self-improve along an exponential curve and we’ll never be able to catch up with it, making it effectively a god in comparison to us poor humans. While human intelligence is improving over time (via natural selection and perhaps whatever causes the Flynn effect) it does so much, much more slowly and in a way that doesn’t seem to be accelerating exponentially.

But maybe gene editing changes that.

Gene editing seems about as close as a biological organism can get to recursively editing its own source code, and with recent advances (CRISPR, etc) we are plausibly much closer to functional genetic manipulation than we are to human-level AI. If this is true, humans could reach fast takeoff in our own biological intelligence well before we build an AI capable of the same thing. In this world we’re probably safe from existential AI risk; if we’re both on the same curve, it only matters who gets started first.

There are a bunch of obvious objections and weaknesses in this analogy which are worth talking through at a high level:

  • The difference between hardware and software seems relevant here. Gene editing seems more like a hardware-level capability, whereas most arguments about fast takeoff in AI talk about recursive improvement of software. It seems easy for a strong AI to recompile itself with a better algorithm, where-as it seems plausibly more difficulty for it to design and then manufacture better hardware.

    This seems like a reasonable objection, though I do have two counterpoints. The first is that, in humans at least, intelligence seems pretty closely linked to hardware. Software also seems important, but hardware puts strong upper bounds on what is possible. The second counterpoint is that our inability to effectively edit our software source code is, in some sense, a hardware problem; if we could genetically build a better human, capable of more direct meta-cognitive editing… I don’t even know what that would look like.
  • Another consideration is generation length. Even talking about hardware replacement, a recursively improving AI should be able to build a new generation on the order of weeks or months. Humans take a minimum of twelve years, and in practice quite a bit more than that most of the time. Even if we end up on the curve first, the different constant factor may dominate.
  • We don’t really understand how our own brains work. Even if we’re quite close to functional genetic editing, maybe we’re still quite far from being able to use it effectively for intelligence optimization. The AI could still effectively get there first.
  • Moloch. In a world where we do successfully reach an exponential take-off curve in our own intelligence long before AI does, Moloch could devour us all. There’s no guarantee that the editing required to make us super-intelligent wouldn’t also change or destroy our values in some fashion. We could end up with exactly the same paperclip-maximizing disaster, just executed by a biological agent with human lineage instead of by a silicon-based computer.

Given all these objections I think it’s fairly unlikely that we reach a useful biological intelligence take-off anytime soon. However if we actually are close, then the most effective spending on AI safety may not be on AI research at all – it could be on genetics and neuroscience.

Milk as a Metaphor for Existential Risk

[I don’t believe this nearly as strongly as I argue for it, but I started to pull on the thread and wanted to see how far I could take it]

The majority of milk sold in North America is advertised as both “homogenized” and “filtered“. This is actually a metaphor created by the dairy industry to spread awareness of existential risk.

There has been a lot of chatter over the last few years on the topic of political polarization, and how the western political system is becoming more fragile as opinions drift farther apart and people become more content to simply demonize their enemies. A lot of causes have been thrown around to explain the situation, including millennials, boomers, free trade, protectionism, liberals, conservatives, economic inequality, and the internet… There’s a smorgasbord to choose from. I’ve come to believe that the primary root cause is, in fact, the internet, but the corollary to this is far more frightening than simple cultural collapse. Like milk, humanity’s current trend toward homogenization will eventually result in our filtration.

The Law of Cultural Proximity

Currently, different human cultures have different behavioural norms around all sorts of things. These norms cover all kinds of personal and interpersonal conduct, and extend into different legal systems in countries around the globe. In politics, this is often talked about in the form of the Overton window, which is the set of political positions that are sufficiently “mainstream” in a given culture to be considered electable. Unsurprisingly, different cultures have different Overton windows. For example, Norway and the United States have Overton windows that tend to overlap on some policies (the punishment of theft) but not on others (social welfare).

Shared norms and a stable, well-defined Overton window are important for the stable functioning of society, since they provide the implicit contract and social fabric on which everything else operates. But what exactly is the scope of a “society” for which that is true? We just talked about the differences between Norway and the U.S., but in a fairly real sense, Norway and the U.S. share “western culture” when placed in comparison with Iran, China, or North Korea. In the other direction, there are distinct cultures with different norms around things like gun control, entirely within the U.S. Like all categorizations, the lines are blurry at times.

The key factor in drawing cultural lines is interactional proximity. This is easiest to see in a historical setting because it becomes effectively identical to geographic proximity. Two neolithic tribes on opposite ends of a continent are clearly and unambiguously distinct, where-as two tribes that inhabit opposite banks of a local river are much more closely linked in every aspect: geographically, economically, and of course culturally. Because the two local tribes interact so much on a regular basis, it is functionally necessary that they share the same cultural norms in broad strokes. There is still room for minor differences, but if one tribe believes in ritual murder and the other does not, that’s a short path to disagreement and conflict.

Of course, neolithic tribes sometimes migrated, and so you could very well end up with an actual case of two tribes coming into close contact while holding very different cultural norms. This would invariably result in conflict until one of the tribes either migrated far enough away that contact became infrequent, became absorbed into the culture of the other tribe, or was wiped out entirely. You can invent additional scenarios with different tribes and different cultures in different geographies and economic situations, but the general rule that pops out of this is as follows: in the long run, the similarity between two cultures is proportional to the frequency with which they interact.

The Great Connecting

Hopefully the law of cultural proximity is fairly self-evident in the simplified world of neolithic tribes. But now consider how it applies to the rise of trade, and technology over the last several millennia. The neolithic world was simple because interactions between cultures were heavily mediated by simple geographic proximity, but the advent of long-distance trade started to wear away at that principle. Traders would travel to distant lands, and wouldn’t just carry goods back and forth; they would carry snippets of culture too. Suddenly cultures separated by great distances could interact more directly, even if only infrequently. Innovations in transportation (roads, ship design, etc) made travel easier and further increased the level of interaction.

This gradual connecting of the world led to a substantial number of conflicts between distant cultures that wouldn’t have even know about each other in a previous age. The victors of these conflicts formed empires, developed new technologies, and expanded their reach even farther afield.

Now fast-forward to modern day and take note of the technical innovations of the last two centuries: the telegraph, the airplane, the radio, the television, the internet. While the prior millennia had seen a gradual connecting of the world’s cultures, the last two hundred years have seen a massive step change: the great connecting. On my computer today, I could easily interact with people from thirty different countries around the globe. Past technologies metaphorically shrank the physical distance between cultures; the internet eliminates that distance entirely.

But now remember the law of cultural proximity: the similarity between two cultures is proportional to the frequency with which they interact. This law still holds, over the long run. However the internet is new, and the long run is long. We are currently living in a world where wildly different cultures are interacting on an incredibly regular basis via the internet. Unsurprisingly, this has led to a lot of cultural conflict. One might even call it cultural war.

Existential Risk

In modern times, the “culture war” has come to refer to the conflict between the left/liberal/urban and right/conservative/rural in North American politics. But this is just the most locally obvious example of different cultures with different norms being forced into regular interaction through the combination of technology and the economic realities that technology creates. The current tensions between the U.S. and China around trade and intellectual property are another aspect of the same beast. So are the tensions within Europe around immigration, and within Britain around Brexit. So was the Arab Spring. The world is being squished together into a cultural dimension that really only has room for one set of norms. All wars are culture wars.

So far, this doesn’t seem obviously bad. It’s weird, maybe, to think of a world with a single unified culture (unless you’re used to sci-fi stories where the unit of “culture” is in fact the planet or even the solar system – the law of cultural proximity strikes again!) but it doesn’t seem actively harmful as long as we can reach that unified state without undue armed conflict. But if we reframe the problem in biological and evolutionary terms then it becomes much more alarming. Species with no genetic diversity can’t adapt to changing conditions, and tend to go extinct. Species with no cultural diversity…

Granted, the simplest story of “conditions change, our one global culture is not a fit, game over humanity” does seem overly pessimistic. Unlike genetics, culture can change incredibly rapidly, and the internet does have an advantage in that it can propagate new memes quite quickly. However, there are other issues. A single global culture only works as long as that culture is suitable for all the geographic and economic realities in which people are living. If the internet forces us into a unified global culture, but the resulting culture is only adaptive for people living far from the equator… at best that creates a permanent underclass. At worst it results in humanity abandoning large swaths of the planet, which again looks a lot like putting all our eggs in one basket.

Now that I’ve gotten this far, I do conclude that the existential risk angle was maybe a bit overblown, but I am still suspicious that our eventual cultural homogeneity is going to cause us a lot more problems than we suspect. I don’t know how to stop it, but if there were a way to maintain cultural diversity within a realm of instant worldwide communication, that seems like a goal worth pursuing.


Bonus: I struggled to come up with a way to work yogurt (it’s just milk with extra “culture”!) into the metaphor joke, but couldn’t. Five internet points to anybody who figures out how to make that one work.