Intelligence Against Instinct: Why Prediction Is Not Enough
AI research has settled, with increasing confidence, on a definition of intelligence: prediction. To be intelligent is to anticipate what comes next, to select the correct response, and to convert that response into an advantage.
But prediction, however powerful, is only the surface of intelligence. It describes what happens when the future is already assumed to be continuous with the present.
Intelligence begins elsewhere—at the moment that continuity breaks.
What is intelligence?
Modern AI research has defined intelligence as fundamentally prediction. Blaise Agüera y Arcas: ‘intelligence is the ability to model, predict, and influence one’s future.’ Joseph Chen: ‘the ability to predict the future accurately and to benefit from that prediction.’
Such definitions are driven by engineering objectives: can we build computational machines that accurately predict the future? The accuracy is measured by testing: can the system get the right answer when asked a question? Intelligence starts to look like a grade school curriculum, complete with lessons, text books, exercises, quizzes, and test scores.
This is far too narrow a definition. It offers a universal ontological answer to an engineering problem.
This raises a series of questions:
What happens when our traditionally philosophical questions become driven by engineering objectives?
What happens when engineers seek a universal definition of intelligence as a system design objective?
What is the status of the historical record when it is engineered into probability vectors?
What is the significance of calling this vectorizing ‘intelligence’?
The result is a narrowing of intelligence to instinct—an automated response to an external stimulus. I’ll unpack this in what follows.
Instinct and Intelligence
A better definition of intelligence: the adaptive and expansive capacity to make the future less like fate and more like an open field of possibilities. A shorter version: the adaptive and expansive capacity to turn necessity into options.
We can find this throughout our written history. In De Ira, Seneca writes:
The cause of anger is a belief that one has been wronged, to which one ought not lightly give credence. One shouldn’t immediately assent even to what is clear and obvious, for some things are false that look like the truth.One must always take one’s time: the passage of time makes the truth plain. (De Ira, 2.22.2)
Anger is a habituated instinct—an automated response to a stimulus. Insofar as it contains information— ‘a belief that one has been wronged’ —that information can be deciphered and redirected. When that happens, we witness the birth of intelligence out of instinct. What appeared to be necessity actually turns out to be contingency— ‘for some things are false that look like the truth’ —so long as we are able to intervene in the otherwise automated flow of time.
Anger either eliminates options or it doesn’t see them in the first place. It wants to rush the sequence as fast as possible. If it is interrupted, it will naturally seek to get back on track. But if that interruption is prolonged, contingency becomes genuine possibility— ‘One must always take one’s time: the passage of time makes the truth plain’.
Intelligence emerges from instinct when the interruption that creates contingency is no longer seen as an obstacle but as opportunity.
Henri Bergson differentiates instinct from intelligence in exactly this way. He saw them as mingled tendencies in elán vital that took time to bifurcate:
Without going deeply into a matter we have dealt with elsewhere, let us simply say that intelligence and instinct are forms of consciousness which must have interpenetrated each other in their rudimentary state and become dissociated as they grew. (‘The Two Sources of Morality and Religion’ in Key Writings, 375)
Instinct is ‘riveted’ to its task while intelligence emerges when the interruption of instinct lasts ‘long enough to seek for reasons’.
We see this bifurcation of tendencies in the passage I quoted from Seneca. Anger moves by instinct—it is riveted to its task and is a form of fate and necessity. But we can cultivate the capacity to interrupt ourselves long enough to create options.
This can look like back-propagation:
When the light has been removed and my wife has fallen silent, aware of this habit that is now mine, I examine my entire day and go back over what I’ve done and said, hiding nothing from myself, passing nothing by. For why should I fear any consequence from my mistakes, when I am able to say, ‘See that you don’t do it again, but now I forgive you’. (De Ira, 3.36.3)
Our instincts can be read and reviewed because they are full of habituated information. They know what they are doing even if our consciousness is simply along for the ride. Anger unfolds its instinct for vengeance just as a gene unfolds its proteins. Both knowwhat they are doing implicitly, but it will take an effort of intelligence to make the implicit explicit.
From Bergson’s perspective, the mingled tendencies begin to bifurcate, and intelligence is the creative energy that does the work.
Language and Intelligence
Let’s return to these questions: What is the status of the historical record when it is engineered into probability vectors? And what is the significance of calling this ‘intelligence’?
Language is the mechanism by which instinct becomes intelligence. Seneca is not just describing the way in which we can intervene in necessity; he is teaching us how to do so. This is a crucial point: De Ira is both a chronicle and an artifact in the history of intelligence. If language is how intelligence grows, then our literary history is both chronicle and artifact of intelligence emerging from instinct.
We are not just reading the historical record. We are witnessing the birth of intelligence, and insofar as we can learn from Seneca’s explanation of anger, we can expand our own intelligence. Seneca’s text is simultaneously historical and contemporary.
We should not be surprised to see the bifurcation happening within these texts.
Genesis: Eve, prompted by the serpent, eats of The Tree of the Knowledge of Good and Evil. Both she and Adam gain consciousness as the experience of shame—they can recognize and reflect on their nakedness. God expels them, and history begins. Without that bifurcation, there is no story to tell; history cannot begin. Eden is pure stasis, pure instinct, pure necessity. It is Lucretius laminar flow of atoms prior to a clinamen.
The Iliad: Achilles has withdrawn from battle because he has been insulted by Agamemnon. His mother, the goddess Thetis, tells him that he has two fates of which he must choose:
For my mother Thetis the goddess of the silver feet tells me
I carry two sorts of destiny toward the day of my death. Either,
if I stay here and fight beside the city of the Trojans,
my return home is gone, but my glory shall be everlasting;
but if I return home to the beloved land of my fathers,
the excellence of my glory is gone, but there will be a long life
left for me, and my end in death will not come to me quickly. (IX 410-16)
The epic genre will foreclose his choice; he must return to battle. Yet, in this moment we see the suspension of a single-threaded fate. Necessity momentarily bifurcates, but to quote Bergson describing the momentary interruption of the bee at her work:
In the last of these moments, when instinct regaining mastery would drag her back by sheer force to her task, intelligence at the point of relapsing into instinct would say, as its parting word, ‘You must because you must’. (373-4)
We could stack up examples of figures introducing contingency into fate—from Babylonian astronomers to Socrates to Hamlet to Cervantes to Montaigne to Nietzsche. The question before us is this: when we vectorize our words and re-assemble them based on probabilities responding to prompts, have we designed intelligence or instinct?
I argue the latter.
Attention is all you need
A Transformer, as currently engineered, only serves instinct, though it does have the capacity to interrupt the algorithmic flow through the highest probability scores. It can suspend this movement, but this does not mean that it bifurcates from instinct as intelligence. It is riveted to its task; it must because it must.
To be intelligence, a Transformer would need a capacity to evaluate its outputs against goals and objectives as it is producing. To use Agüera y Arcas’ definition of intelligence, it would need to adjust its predictions and its modeling based on how it is ‘influencing one’s future’. In Chen’s terms, it would need to have a dynamic sense of ‘gainability’ and adjust its orientation as it works.
It would need the capacity to change goals because it notices something different in the situation—i.e., it would need discernment and self-criticism in addition to the ability to attend to the entirety of tokens in a context window.
All of this requires a broader understanding of intelligence. It can’t be understood essentially as a programmable function. It must be read through its effects, which is why I insist on a definition that attends to its effects on fate and necessity. Our history contains the long trail of this adaptive and expansive capacity to make the future less like fate and more like an open field of possibilities—creativity, contemplation, discernment, humility, conjecture, criticism, storytelling, speculation.
I argue that all of these are inside intelligence, not outside. We should not relegate them to mere ‘sentiment’ in conflict with ‘rationality.’ This is Modernity’s bifurcation.