Book Review: Superintelligence by Nick Bostrom
Nick Bostrom presents a broad and interesting review of the possible futures of superintelligent AI. The book provides an easy to approach summary of threats and potential solutions. Still, the book left me with more of an introduction to vocabulary than a practical approach on what to do about it. As someone working in AI, I felt it was a little too academic in its approach but serves as an excellent general reader overview of the space.
The underlying premise of superintelligence is sound and almost inevitable. Assuming, as we should, that systems and humans continue to become more intelligent over time and that we can create machines to simulate this process, it stands to reason that at some point, we will make a machine that exhibits human-level intelligence. From that moment on, the machine will continue to learn and evolve. Suppose we also assume that learning and evolution can occur faster within a machine environment. In that case, it is a natural conclusion that once a machine is on par with human intelligence, it surpass us with an ever-growing gap between our intelligence and that of AI. Much of the book explores the potential consequences of creating such an AI and how we may limit its existential threat to humanity.
Discussions on the various methods of control and limitations we can apply on a superintelligent AI provide food for thought and an interesting way to look at competition and power within today's human environments. The book focuses mainly on the emergence of a singleton, a single all-powerful AI with no reasonable competitors. This singleton emerges under a wide range of initial conditions based on the idea that an AI with a small head start will grow non linearly in intelligence and quickly become a single powerful force. I have some reservations about this idea. There are genuine hardware restrictions, evident even today, that likely limit the upper ceiling of AIs growth.
Simple things like limitations to available RAM limit even today's AI. While it is reasonable to assume we will continue to produce RAM and other compute resources, these resource constraints lead me to believe that a multi-player scenario is more likely. One in which some several highly capable AIs compete for resources. How we teach them to interact with us and each other is a topic worthy of more exploration.
On a related front, I think in today's political climate, and with most countries driven by capitalist competition, we are even more likely to see a multi-player scenario. This scenario poses its own set of dangers. Any of these groups working on an AI with even a long term goal of superintelligence will be aware of constraints on today's resources. Knowing the power such an AI could give these groups the threat or weaponisation or more subtle AI manipulation is very real. Given that technology, AI or otherwise can be developed and released much faster than new social norms. There is a severe risk of danger in an unregulated AI free-for-all.
I am sure that at a national security level, AI is fast becoming a central discussion point. In a world where AI in some form controls the media we consume, the directions we take to work, and many other facets of day to day life, there is a chance to war on new frontiers. In days gone by, supplying illicit and illegal drugs to a population could be used as a tactic of war. Today, perhaps addiction has become digital, controlling the ideas of the population. While it feels somewhat like a conspiracy theory, it is all very possible. I am optimistic that we will develop new AI aware social norms that counter some of this potential manipulation, intentional or otherwise, but this will take time.
Overall while I enjoyed the book and would recommend it to others, I think there is a need for more discourse on the closer and perhaps just as dangerous threats. I liken it to reading about how the sun will one day engulf the earth whilst ignoring the genuine and present dangers of climate change. All that being said, if this book opens the door to more AI research and brings forward the development of a new AI aware social norm, then it has done its job.