Strength versus Intelligence

The Terminator is a creature from our primordial nightmares: tall, strong, aggressive, and nearly indestructible. We’re strongly primed to fear such a being—it resembles the lions, tigers, and bears that our ancestors so feared when they wandered alone on the savanna and tundra.

But shift perspective for a moment and imagine yourself as the bear. If you could sit down and talk with other bears and trade stories, you might try to frighten each other by talking of the terrifying hairless apes. These monsters are somehow capable of coordinating in huge groups: whenever one is attacked, others spring immediately to its defense, appearing from all sides, from over distant hills and down from the sky itself. They form larger and larger tribes that don’t immediately disintegrate under pressure from individuals. These “humans” work in mysterious sync with each other and seem to see into your future: just as you run through a canyon to escape a group of them, there is another group waiting for you at the other end. They have great power over the ground and the trees themselves: pits and rockslides and other traps mysteriously appear around them. And, most terrifyingly, the wise old bears murmur that it’s all getting worse: humans are getting more and more powerful as time goes on, conjuring deadly blasts from sticks and moving around ever more swiftly in noisy “cars.” There was a time, the old bears recall—from their grandparents’ memories of their grandparents’ tales, down through the generations—when humans could not do these things. And yet now they can. Who knows, they say with a shudder, what further feats of power humans will one day be able to achieve?

As a species, we humans haven’t achieved success through our natural armor plating, our claws, our razor-sharp teeth, or our poison-filled stingers. Though we have reasonably efficient bodies, it’s our brains that have made the difference. It’s through our social, cultural, and technological intelligence that we have raised ourselves to our current position.

No other species of large mammal comes close to having seven billion members. Few species are so immune to natural predators that the main risk to their survival comes from themselves. No other species has landed on the moon and created long-term habitats in space. Since our intelligence has achieved so much, it should be obvious we should not fear the robot, which is nothing but an armed and armored bear. Instead, we should fear entities that are capable of beating us at our own game. It is the “intelligence” part of “artificial intelligence” that we have to fear. If machines can outthink us and outcompete us in the fields of human domination—economics, politics, science, propaganda—then we have a serious problem.

But is it realistic that this could happen? Is an intelligent machine even possible? We know our grandparents would have found our current technology unbelievable, but it’s still quite a stretch to imagine human-level intelligence encased in a machine.

This short book will argue that human-level AIs—I’ll just call them “AIs” from now on—are plausible, that they could become extremely powerful, that we need to solve many problems in ethics and mathematics in order to program them safely, and that our current expertise is far from adequate for the task.

But first, let’s look at intelligence itself.