More on agency
A 4 year old can explain how the digestive system works, operate a TV, play cards, put together breakfast, and do addition and subtraction. What they cannot do, however, is put the washing on at 11am tomorrow.
In many ways, tomorrow does not yet exist for a 4 year old, at least not in the same way that it might exist for an 8 year old, or an adult.
The other day, my wife and I asked our son whose birthday was coming up next.
“Daddas.”
“And how old will he be turning?”
“27.”
(We haven’t talked about it for a few days, and as far as I recall, he only knows that I am currently 26.)
“Okay, but when is Daddy’s birthday?” (We had been trying all morning to explain to him that it was later in the week).
“It’s tomorrow.”
This is a surprising thing about 4 year olds. No matter how hard you try, they cannot seem to grasp timeframes beyond the current day. Everything in the future — from a birthday, to going back to kindergarten, to Christmas — gets lumped into the ‘tomorrow’ basket. And if you go back a couple years, to the age of 2, good luck trying to explain to a child that they only have to wait another hour. They’ll each pick their own designation, but everything in the future becomes “10 more minutes” away.
One way to think about the mind of a 4 year old is that they have a rather eidetic world-model, but that it is atemporal. That is, they hold a single time-slice of the world in their mind, and they can label nearly every object in it, but they struggle to imagine how these things evolve over time. More accurately, they can label temporal events, like the dinosaurs having ‘gone extinct a long time ago’, but they do not tacitly understand how the world operates over timeframes over a few hours or day. As a result, they cannot plan coherently on those timeframes.
The bearing this has on AI is the following. The magic researchers often focus on is the neocortex, because it is this that seems to physiologically differentiate us from other animals, including other primates. If we can replicate the primary function of the neocortex, then we will have AGI.
But if the neocortex is understood foremost as a predictive world-model, then in many ways we already have this in the transformer architecture. Like my 4 year old, GPT can memorise a static model of the world: a picture or map of the world, with its many parts labelled. It even has a picture of language: it can learn any new word fairly quickly via instruction, figuring out how it fits functionally around all the other words, sentences, paragraphs, and books.
A hypothesis of mine is that evolution never had a particularly hard time in designing the neocortex. After all, we haven’t had that hard of a time coming up with the transformer architecture. A multi-modal, generally predictive algorithm necessarily leverages simplicity and uniformity as a core design principle.
So what was the bottleneck then? In the case of my son, he is not limited when it comes to memorising the general shape of the world, as an atemporal picture, or as a language. What is taking a long time to learn, and will continue to grow gradually throughout his childhood, and then onto adulthood, is how to apply this knowledge coherently over long-time horizons.
The magic does not seem to be in the neocortex alone, but in how it interacts with the limbic system. Scaling up the neocortex seems fairly easy from first principles: what seems hard is getting this predictive machine to do anything coherent, which requires a very particular arrangement with the steering committee (the limbic system).
This measure of intelligence is often called ‘t-AGI’. It asks of a system, what set of tasks can this system do, and how long would it normally take a human to do those same tasks? However long it takes a human defines that system’s t-AGI.
So while all humans have within them the capacity for AGI, throughout development, a human steadily grows its t-AGI. My 1 year old can do tasks coherently up to about 5 minutes. My 4 year old, tasks up to about an hour. 8 year olds seem to be able to do tasks coherently over the course of a day. A 13 year old can begin to organise their week. And an 18 year old starts to branch out to months. Later on in life, individuals often start to coherently organise over years and decades.
t-AGI does seem to be the metric of relevance when it comes to the economic impact, or ‘power’ of artificial systems. It doesn’t matter what arbitrary task your AI system is good at, unless that system can act coherently in the world on long-time horizons. It would not be surprising in the least if we had an AI that could solve a Millennium Prize problem by 2030, but which still struggled to operate as a fully-functional accountant.
Perhaps there is some prodigious eight year old out there currently gaining some traction on such a problem; there are no prodigious accountants.
It’s worth considering why this might be.
As you expand task-time, the number of variables to control for/states to predict for expands. Seemingly it expands in a greater-than-linear fashion: cooking eggs on toast, end-to-end, is more complicated than just getting out the ingredients, or just getting the cooked eggs onto the toast (as you might break the overall cooking process down for a toddler). When doing the task end-to-end, you add the dimension of simultaneously managing multiple time-sensitive processes. There’s a lot more ways that things could go wrong, and the second things start going wrong, the whole thing can unravel fast.
So you need more and more compute to control for the various things that may be changing in the world in and around your task, as task-length increases.
But this is the first-pass explanation. This explanation is task-specific, and I believe where a focus on performance on individual tasks breaks down as a indication of AGI (i.e. the current paradigm of benchmarking on individual tasks cannot measure for t-AGI). The issue an agent has, as task-length increases, is that to complete the task, increasingly more things, which often include other tasks, intervene on task-completion. To get anything complex done in the world, an agent has to learn how to dynamically manage its own time, such that it can robustly get tasks done despite the constant intervention of the world (and oftentimes the intervention of other agents, which is highly complex). This is very compute intensive, and unfortunately there’s not a whole lot of data to go on, unless you wait a very long time — at which point it becomes clear why t-AGI increases gradually with age.
A 4 year old might have a specific kind of castle in mind that they want to build. Each day, they set about building their castle, but things intervene. Their parents call them in for dinner; when they get back, they find that their little brother has destroyed the whole thing, as younger kids tend to do.
A 5 year old might have a specific kind of castle in mind that they want to build. They think for a moment. “Mum? Do I have time to build a castle?”, after which they take the building material to their room, where they can close the door to ensure that their little brother will not be able get in if they have to suddenly leave their construction.
It’s in that thinking for a moment that the magic happens. It’s no longer just the idea of the castle and running ahead to build it. It’s a simulation of the many things that may happen between the start and the end of the task, and the simulation of all the different things that could be done to reduce the chance that things could go wrong in between.
And that’s for a ~20 minute activity. Now try building a factory, or a business empire, or a political party. These are far more compute and data intensive problems — quite obviously — than getting a perfect SAT, or devising a new proof in theoretical mathematics (not to say that either is more worthwhile, but that, like a 2700 ELO in chess, it’s the case that it’s not that compute intensive, but rather very intimidating for humans to do, given that we are optimised for general-purpose agency in the world).
The point here is that there is no algorithm that you could place in the human brain — let alone within the mind of an artificial system say 2 orders of magnitude larger in compute — that could solve for this problem particularly faster than the 4 year old. That’s because compute or algorithms are not the major limiting factor here: it’s the real-world, long-time horizon data.
RL + transformers does encapsulate what is required for AGI, but I get the sense that this is a far more challenging problem than independently devising RL or transformers. Again, the magic does not seem to just be in the neocortex, but in getting it to do anything useful in conjunction with the limbic system.
From another angle: which took evolution longer to produce?