THE VOICE IN THE MACHINE

On a word everyone uses and no one can define

For the past couple of years, whenever I give a talk about AI, someone in the audience inevitably asks about AGI, Artificial General Intelligence, and every time I feel the same discomfort. I have spent an hour or more speaking carefully about systems I understand, what they do, where they fail, how we got here, and then I am handed a word nobody in the room could define and asked to say when it will arrive. My answer is generally dismissive, something about the term being mostly a creation of the press, which I am not sure I believe even as I say it. After that, I can see the skepticism settle over the room, along with a doubt about whether I know what I am talking about. That is why I decided to develop and write down my own point of view on AGI, long before Sam Altman told TIME in August 2026 that OpenAI expected to have an internal system he would call AGI before the end of the year. Astra was released on September 3, and OpenAI’s president Greg Brockman closed the launch briefing by welcoming us to the AGI era. Jensen Huang went further a few days later. Those announcements gave me a reminder of how much work that undefined word is being asked to do, and this writing is my attempt to understand it better.

A note on timing. Everything below was written in the first week of September 2026, and the people I quote have a habit of announcing things by the hour. If some of it has been overtaken by events between my writing and your reading, that is not a flaw in the argument. It is, I am afraid, the argument.

Everyone talks about AGI

When did we all start talking about AGI as though we knew what we meant? It has become one of those phrases that circulates faster than its own definition, spoken by chief executives, newspaper columnists, and dinner guests, always with the same unspoken confidence, as though it were an agreed upon destination and the only real question was how long will it take to get there. I have spent my working life among people who build things and among people who write about the people who build things, and, alas, among people who write about people who write about people who build things. In this process I became suspicious of any word that everyone is sure of and no one can define exactly, or too many people define in different ways; in my experience a word like that has little to do with description and a great deal to do with persuasion.

I want to say plainly that I have come to think that AGI is mostly a red herring. I do not mean that its pursuit is fraudulent or that nothing is happening; a great deal is happening, more than what we asked for, and some of it is genuinely useful and even beautiful. I mean something narrower and, I think, more unsettling: that the term itself points us away from the questions worth asking, that it gathers our attention around an imagined destination while the actual road, with all its ordinary and difficult turns, runs somewhere else entirely. A red herring is not a lie. It is a distraction that smells convincing, and the more convincing it smells the further it can lead you from the trail.

To follow the trail rather than the smell, I found it useful to break my unease into four questions. First, do we even agree on what intelligence is, before we reach for the word general? Second, when people say AGI, do any two of them mean the same thing? Third, setting the definitions aside, do we actually need the thing at all, and if so for what? And fourth, the question that seems to me the most revealing and the least often asked, what would a world with AGI actually look like?

I will lean on three people as I go, and I have chosen them deliberately because they disagree with one another. Julian Togelius, a computer scientist who works on artificial intelligence and games, has written a short and careful book that takes the idea of general intelligence seriously enough to ask, with real rigor, what it would even consist of. Rodney Brooks, who has probably built more robots than anyone else, has spent years questioning the timelines and the hype from the direction of the physical world, where motors, mass and reliability create challenges that are of a different kind than those present in the software world. And Blaise Agüera y Arcas, a respected colleague during my time at Google, is quite involved in building AI systems with his team, and has written the most sophisticated case I know for the view that intelligence is nothing but prediction, and that the tests we keep proposing to separate real intelligence from imitation keep failing to separate anything, though he is candid about the ones that still work. Between Togelius, Brooks, and Agüera y Arcas, I think we can find something more honest than either the enthusiasm or the dismissal on offer elsewhere.

First question: do we even know what intelligence is?

Before we worry about the word general, we might want to ask what intelligence is. Togelius does something useful here, which is to decline to invent his own definition and instead go looking at how three different fields have answered the question, and what he finds is that they have not converged to a unique definition.

Psychology gives us the oldest answer, and it turns out to be less unified than the phrase general intelligence would suggest. The model Togelius reaches for is the Cattell-Horn-Carroll theory, the framework that a century of testing and statistical analysis eventually settled on, and that describes a hierarchy. At the top sits g, the general factor, the observation that people who do well on one sort of mental test tend to do well on others. Below it are different broad abilities, including fluid reasoning, comprehension and accumulated knowledge, working memory, processing speed, visual and auditory processing. Below them a long tail of narrower capabilities. So even in the discipline that gave us the idea of a general factor, generality comes layered, subdivided, and plural, and what the layers describe are the dimensions along which human beings differ from one another on tests designed by human beings themselves.

Ethology, the study of animal behavior in natural conditions rather than in the laboratory, gives a different answer, and a stranger one. Consider the honeybee. Walter Murch, in In the Blink of an Eye, his small classic on film editing, relates in a footnote that a beehive can be shifted a couple of inches a night without the bees minding at all, and can be carried two miles away without much trouble either, since the total change of scene forces them to reorient. Move it two yards and they are lost. They return from foraging to hover in the air where the hive used to be, while the hive sits a few steps away. The world still looks like home, so nothing tells them to look again. Whatever we mean by intelligence, it has to accommodate a creature that solves the two-mile problem and fails the two-yard one, and I notice that our vocabulary of narrow and general does not help at all. The bee is a genius and a dunce at the same time, and nothing in the word general tells me which one to write down.

Computer science gives a third answer, the most precise of the three and the least usable. Formal measures of universal intelligence, of the kind developed by Shane Legg and Marcus Hutter, define intelligence as average performance across all possible environments, weighted so that simpler environments count for more. It is mathematically elegant. It is also uncomputable, meaning no actual procedure can evaluate it, so it functions as a definition that can never be applied to anything.

Three traditions, three answers, and no dictionary that translates among them. When someone tells me a machine has achieved general intelligence, I would like to know which of these three they have in mind, and I have never once been told.

Agüera y Arcas cuts through all of this, and it is worth stating his view carefully because, in my opinion, he has a very coherent position in the whole debate, one that goes through two layers. The first is functionalism, the philosophical position, which he traces to Alan Turing and John von Neumann, that a thing is defined by its function, what it does, rather than by what it is made of. His example is a future artificial kidney built from carbon nanotubes: nothing about those atoms makes them a kidney, and if you were the one waiting for the transplant you would not care in the slightest what it was made of, only whether it worked. The same reasoning extends to minds. If a function is what it does, then whether it runs on carbon or on silicon is not a difference in kind.

Then he argues that intelligence is prediction, the probability of the future given the past, coupled with the agency to act on what you have predicted. The route he takes to that claim is the interesting part. For years, he writes, everyone in the field took it for granted that predicting the next word was AI-complete, a term coined in the 1980s, by analogy with the complexity theorists’ NP-complete, for problems you cannot solve without having solved intelligence itself. The reason seemed obvious: to reliably finish an arbitrary sentence you need world knowledge, arithmetic, common sense, and some ability to imagine what another person is feeling. Everyone agreed a real intelligence would be able to do next-word prediction. Nobody expected that building a very good next-word predictor would produce real intelligence. Then, more or less, to our surprise, it did. And so he asks whether intelligence is a side effect of solving prediction or whether the two are simply the same thing, and he answers that they are the same thing.

Set his answer beside Togelius’s survey and you have the shape of the whole problem. Togelius shows a concept fracturing into incompatible pieces across the fields that study it. Agüera y Arcas shows a single function running continuously from the bacterium to the language model, never uniquely human, never crossing a threshold. They cannot both be right. And yet notice that the ground is contested this far down, at the meaning of the root word, before anyone has said anything at all about the word general.

Second question: do any two people mean the same thing?

Let’s go back to AGI. Togelius, in his book, devotes a chapter to what he calls the varieties of artificial general intelligence, and the plural in his title is the argument. Once you concede there are varieties, the question of when we reach AGI stops making sense, in the way that asking when someone will be fluent in Italian stops making sense once you notice how many different things fluency turns out to mean. We use the word all the time. We just cannot say where the line is.

The disagreement among the people actually building these systems is sharper than the public conversation suggests. Yann LeCun began his career at Bell Labs, spent years as a professor at NYU, ran AI research at Meta, and has since co-founded AMI Labs, for Advanced Machine Intelligence, the term he prefers to AGI. He has said flatly that there is no such thing as general intelligence, that the phrase is really just a stand-in for human-level intelligence, and that human intelligence is itself wildly specialized rather than general. He calls the term complete nonsense, and thinks the missing ingredient is world models, an internal picture of how physical reality behaves, which no amount of additional text prediction will supply. His NYU colleague Gary Marcus, who has spent years arguing that scaling alone will not get us there, attacks from the direction of cognitive science, and he is not alone in noticing that the industry’s working definition, taken from OpenAI’s own charter, is a system that outperforms humans at most economically valuable work. I find that a strange thing to call a mind. It defines intelligence by labor output, which would make a very good spreadsheet more intelligent than a curious child.

Jensen Huang has given two different answers within six months. In March he accepted a podcast host’s proposed test, whether an AI could build and grow a technology business worth a billion dollars, and said he thought we had already met it; in September he wrote simply that AGI had arrived, crediting OpenAI’s Astra and noting that it had been trained on more than a hundred thousand of his own company’s systems, with four hundred thousand more coming online next. He was not concealing the interest; it was the substance of the post.

The believers in AGI deserve to be heard properly. Demis Hassabis of DeepMind answers LeCun directly, arguing that he confuses general with universal, and that brains are in fact extremely general. Shane Legg, who helped popularize the term in the first place, has suggested a minimal version of AGI might arrive around 2028. These are serious people with serious track records, and they are not describing the same object as each other. In August of 2026 Sam Altman told TIME that OpenAI was not quite there yet, but that before the year was out the company would have an internal system he would be willing to call AGI, with his chief research officer putting them eighty percent of the way. The bar he had in mind is the one OpenAI wrote into its own charter, outperforming humans at most economically valuable work. Astra has since been released, which changes less than it seems to. Outside users can run the model; no one outside can examine it, since the weights, the training data, and the evaluations remain the company’s own. And the company still grades the exam it sets. I do not doubt the sincerity of anyone involved. I only observe that a milestone defined, scheduled, measured, and certified by the same party is not the kind of milestone that settles an argument.

The philosophers got here decades before the current boom. John Searle’s Chinese Room argument gave us the distinction between strong AI, a machine that genuinely understands, and weak AI, one that merely behaves as though it does. That distinction exposes something the industry’s definition quietly buries. The definition is purely functional: it describes what a system does and says nothing about what it is. Agüera y Arcas takes this on deliberately. He argues that appearance and reality cannot be pulled apart here, and he is willing to live with where that leads. The industry’s definition carries the same implication without anyone appearing to have chosen it, which is why product announcements can describe a model that understands your question while the definition of AGI behind them was never about understanding at all.

Here is the point I keep returning to. If the people building the thing cannot agree on whether the word denotes anything at all, and the philosophers who examined the concept most carefully concluded it was confused, then the persistence of the term is telling us something. A word that survives that much disagreement about its meaning is not surviving because of its meaning. It is working as a banner, and banners are for rallying, for raising money, and for recruiting.

Third question: do we actually need it, and for what?

Suppose we grant the whole thing for the sake of argument. Suppose the definitions could be settled tomorrow. What would we want the thing for?

This is where Rodney Brooks hits the mark, because he moves the argument out of the lab and into a house where somebody’s mother needs help getting out of bed. For years Brooks has published an annual scorecard of dated predictions, holding himself publicly accountable in a way almost nobody in this field does, and among his benchmarks is a robot that can assist an elderly person across many tasks rather than one, helping them in and out of bed, washing them, getting them to the toilet. He dates that no earlier than 2028. A robot that can merely navigate an ordinary cluttered home, with its steps and narrow doorways, at a price a family could afford, he dates no earlier than 2035. Eight years into holding himself to those predictions, his verdict at the start of 2026 is that no general-purpose solution to the first is anywhere in sight, and that the second has not managed even the laboratory demonstration he expected by this year, since nothing yet handles a cluttered house with so much as a single step.

His reasoning comes from what he calls his three laws of robotics, and the relevant one is about reliability. A technology needs roughly a decade of steady improvement beyond its first impressive laboratory demonstration before it works ninety-nine point nine percent of the time, and each additional decade buys you one more nine. A robot you would trust in a house with a frail person needs four to six of those nines. No amount of investment or conviction gets a nine added to that number faster.

Brooks also supplies the best diagnosis I know of why we keep fooling ourselves, in an essay on the seven deadly sins of predicting the future of AI. Two of them do most of the work. The first is confusing performance with competence: we watch a system do one narrow thing impressively and we silently generalize to a broad competence that was never demonstrated, in the way we would with a person, because with people the inference usually holds. The second is exponentialism, the assumption that anything with a computer in it must ride a Moore’s law curve forever. Text prediction, which is what the large language models do, did ride that curve, in a spectacular manner. Motors and gearboxes and battery chemistry do not.

What makes this more than skepticism is that the enthusiast concedes the crucial ground. Agüera y Arcas, who thinks these systems are already intelligent and already general, acknowledges that they remain too unreliable to trust with consequential tasks without a human watching, and that what they genuinely lack is memory, a persistent inner monologue, and individuation, his word for being a particular someone rather than a generic distillation of everybody. A model is frozen when training ends, forgets everything once the conversation scrolls past, and is nobody in particular. That last gap is the easiest to skip past and perhaps the most consequential: a thing that can be copied, forked, and started fresh a thousand times over is a difficult thing to hold responsible, and a difficult thing to trust with your mother.

There is a warning here I find hard to shake, which Ragnar Fjelland put plainly in 2020: when we overestimate the technology and underestimate the human skill it is meant to replace, we end up replacing something that works with something worse. So here is my honest answer to the third question. The place where a genuinely general, genuinely embodied machine would do the most good, which is, for instance, the physical care of aging people in their own homes, is precisely the place where technology is not arriving. Companionship and reminders and fall alarms are already here. Lifting a person who cannot lift themselves is not, and it is the lifting that decides whether someone can stay in their own house. Meanwhile the things we actually want from these systems have perfectly good names already: transfer, the ability to apply what was learned in one setting to another; adaptability; robustness; reliability under conditions nobody anticipated. Bundling those real and difficult goals under the banner of AGI does not clarify them. It hides them.

Fourth question: what would that world actually look like?

This is the question I find most revealing, and I almost never hear it asked.

Brooks has a vivid image of the problem. The people who wrote the classic movie Blade Runner imagined a machine so perfectly humanlike that a specialized test is needed to detect it, and then had the detective telephone her from a coin-operated payphone. That is what our pictures of an AGI world almost always look like. We envision superhuman capability using what we know about the present and fail to imagine how everything around it would have shifted. And when we do imagine change, we imagine it arriving all at once, which is not how any technology has ever entered the world. I argued in Why the Real AI Revolution Hasn’t Happened Yet, the first of my From Spark to System posts, that Gutenberg was printing books in the 1450s while the institutional and epistemic structures printing made possible took generations to assemble, because what had to be built was not the press but everything around it: standardized spelling, distribution networks, copyright, literacy itself. The destabilization arrives early. The transformation arrives late. Even a real AGI would enter the world that way, as a long renegotiation rather than an announcement.

Agüera y Arcas gives a different answer, and it is the strongest one against the way I have framed this question, because he thinks we are already living in the answer. He borrows from the biologists John Maynard Smith and Eörs Szathmáry the idea of a major evolutionary transition, a rare event in which entities that used to reproduce independently merge into something larger that can only reproduce as a whole, with a division of labor among the parts and new ways of storing and passing on information. The classic case is symbiogenesis, the process by which a bacterium took up residence inside another cell and became the mitochondrion that powers every cell in your body. Language did this to us, and so did agriculture, and so did cities. His claim is that AI is the next one, a merging of human and machine intelligence into a larger cooperative organism, and that asking what a world with AGI would look like misunderstands that we are already living in the early part of it.

It matters to state his position on risk precisely, because it is easy to file him under optimism and move on. He ranks the unfriendly-superintelligence scenario below nuclear war and climate change as an existential threat, and his reason is an argument rather than a mood: no living thing optimizes a single value, since food is good until you are full and rest is good until you are starving, and those signals cannot be added into one number. The doom scenario of a machine relentlessly maximizing one goal therefore misdescribes what intelligence is. He is also not arguing against regulation. He names AI-driven disinformation as a danger to democracy and mass surveillance as a danger to civil liberties, calls both urgent, and devotes much of his final chapter to the case for legal and economic guardrails.

I am not going to adjudicate between Brooks, for whom the physical world sets the pace and that pace is measured in decades, and Agüera y Arcas, for whom the transition began some time ago and we are inside it. I do not think I can, and I think the disagreement between two people who know this much, looking at the same evidence, is itself the most honest picture available of where we actually stand.

What we need instead

Let me wrap up. Several of the most serious people in this field arrive at the same verdict, that AGI as the term is commonly used points us at the wrong thing, or to nothing really tangible and concrete, and they arrive there for three diverse reasons. LeCun because the word means nothing more than human-level, and human intelligence is itself specialized: we feel general only because the problems we can imagine are the ones we are built to imagine. Brooks because the version that would matter in a physical house is decades away and no amount of enthusiasm shortens the reliability curve. And Agüera y Arcas because intelligence has no thresholds in it to cross, which leads him to call the arguments over timing arbitrary, bordering on absurd, and to ask who cares, anyway.

When the field’s most committed enthusiast and its most stubborn skeptics agree that the term is a distraction while disagreeing about nearly everything else, the red herring claim stops looking like contrarianism. Serious people still disagree, and Hassabis’s case that brains are genuinely general deserves a better answer than I can give here. But the disagreement concerns what the word covers rather than any matter of fact, which is why a company can announce it has nearly arrived at a destination it defined itself, and no one outside is in a position to say whether it is right. And that vagueness is not free. It shapes where the money goes, where talented young researchers point their careers, and which problems get called unglamorous. It leaves valuable systems undervalued because they cannot be presented as steps toward a general mind, and lets unremarkable ones be oversold because they can. It pushes aside the questions that turn out to matter, about reliability, about what we are willing to hand over and on what terms.

What I want is not less ambition. I spent my working life around people whose ambitions were enormous, and the best of them worked without a slogan pulling them forward, on problems that took years and did not announce themselves. That is what the valuable progress has always looked like, and I suspect it is what the next stretch will look like too: a long series of specific things becoming reliable enough to trust, without any single moment of arrival. The questions worth asking are the ones we can actually answer. We should ask those, and let the destination take care of itself.

Sources

Julian Togelius, Artificial General Intelligence. MIT Press, 2024. direct.mit.edu/books/book/5835

Blaise Agüera y Arcas, What Is Intelligence? Antikythera, 2025. whatisintelligence.antikythera.org

Rodney Brooks, Predictions Scorecard, January 2026. rodneybrooks.com/predictions-scorecard-2026-january-01

Rodney Brooks, “Rodney Brooks’ Three Laws of Robotics.” rodneybrooks.com/rodney-brooks-three-laws-of-robotics

Rodney Brooks, “The Seven Deadly Sins of Predicting the Future of AI.” rodneybrooks.com/the-seven-deadly-sins-of-predicting-the-future-of-ai

Yann LeCun on X, December 2025. x.com/ylecun/status/2003227257587007712

Demis Hassabis on X, December 2025. x.com/demishassabis/status/2003097405026193809

Alex Heath, “Inside OpenAI’s Reboot,” TIME, 26 August 2026. time.com/article/2026/08/26/openai-sam-altman-interview

Axios, “Welcome to the AGI era, OpenAI says as GPT-6 Astra debuts,” 3 September 2026. axios.com/2026/09/03/openai-astra-gpt-6-agi-brockman

Jensen Huang on X, 6 September 2026. x.com/JensenHuang/status/2096700264569090384

Jeremy Kahn, “Nvidia’s Jensen Huang says ‘We’ve achieved AGI.’ But no one can agree on what that means,” Fortune, 30 March 2026. fortune.com/2026/03/30/agi-definition-jensen-huang-lex-fridman-deepmind-turing-text-cognitive-taxonomy

Gary Marcus, “Sad to see Jensen Huang claim that AGI has arrived, with no evidence and no definitions,” 6 September 2026. garymarcus.substack.com/p/sad-to-see-jensen-huang-claim-that

OpenAI Charter, April 2018.

Shane Legg and Marcus Hutter, “Universal Intelligence: A Definition of Machine Intelligence,” Minds and Machines 17:4 (2007), 391–444. arXiv:0712.3329

John Searle, “Minds, Brains, and Programs,” 1980.

Ragnar Fjelland, “Why general artificial intelligence will not be realized,” Humanities and Social Sciences Communications, 2020.

John Maynard Smith and Eörs Szathmáry, The Major Transitions in Evolution, 1995.

Walter Murch, In the Blink of an Eye.