Little by little, then all at once

When I was a kid, I played a lot of Age of Empires. My favourite moment was gathering enough resources to move to the next 'age'. When I did, suddenly, dozens of new upgrades were available that transformed my city and its citizens. Within a few clicks, I went from equal to my opponents to miles in front -- at least until they hit the same milestone.

Most major innovation, especially in AI, feels like entering a new age.

In the real world, timescales are a little longer, and there are rarely buttons we can hit to bring new prosperity to everyone. Nonetheless, with enough perspective, these changes look like immediate upwards steps. At least from the outside. From the inside, things are quick once they arrive, but getting there happens little by little.

Working in AI, there are a lot of things that are described as happening all at once. New models like AlphaFold and GPT-3 (despite the name) are pegged as overnight breakthroughs. For those outside any field, things appear out of nowhere. As AI becomes a bigger part of all our lives, it pays to learn enough to know how things look from the inside. To know what's coming and be ready for it when it does.

From the inside, these overnight shifts took years and dozens of people to come into existence. Much like in games, we gather the pieces, do the work and prepare to hit that milestone. That work can be draining, filled with setbacks and often feels painfully slow. If you asked the team at OpenAI about GPT-3 they would probably talk about how many false starts they had along the way. Yet one day, there is a change.

It's hard to say when the change happens. Which line of code finished the model. When we knew it was done. But in the fullness of time, those moments will blur together. The world will be marked by a fuzzy moment, before which something didn't exist and after which it did. Once it does, doors we never saw before are open and all at once, things change.

That isn't to say that change lasts forever. Much like in Age of Empires, there's another age waiting to be unlocked. We'll see new protein folding models, new language models and many new innovations in AI. As we unlock new capabilities we're quick to normalise them and push ourselves further. Even now, a lab somewhere is hacking away at the next overnight step change. Chipping away at the code, gathering data and training models.

Little by little, putting in the hours. Until suddenly, a new age, all at once.

I wrote the article above on Monday this week. To many, today's release of DALL-E 2 probably looked like the start of a new age. I hope for many, the tool allows the creation of new art, expressing new ideas and opens us to a world of new creative content.