Grounding, and the hall of mirrors
Here is a failure mode that keeps me up at night, and it has nothing to do with a model becoming too capable. It’s the opposite. It’s a model becoming unmoored.
Consider what a modern AI system actually learns from. More and more of its training signal is generated by other models, or by earlier versions of itself: synthetic text, synthetic images, summaries of summaries, an internet increasingly written by machines. Each generation drinks from a well that the previous generation filled. When that happens with no external reference, small errors don’t wash out; they compound. Researchers have a clean name for one version of this: model collapse, where a system trained on its own outputs slowly forgets the tails of the real distribution and converges on a narrower, more confident, more wrong picture of the world.
Picture a hall of mirrors. Every surface reflects another surface; nothing reflects the world. The system can be fluent, internally consistent, even beautiful, and entirely detached. At an individual scale that’s a curiosity. At the scale we’re now deploying these systems, where their outputs shape what people read, buy, and believe, and then become the next round of training data, it’s an attractor a whole society can slide into. A self-reinforcing loop with no one checking it against reality.
Fluency without grounding isn’t intelligence. It’s a very persuasive echo.
The corrective is contact with the physical world. A system that has to predict what its sensors will read next, move an actuator and feel whether the prediction held, or act and absorb the consequences, cannot drift indefinitely. Reality pushes back. That friction, the gap between what the model expected and what actually happened, is the error signal that keeps intelligence honest. It’s the same reason active learning matters: an embodied system doesn’t just consume a frozen snapshot of the past, it goes and tests its beliefs against the present.
So when I argue for physical AI, I’m not only making an economic argument about market size. I’m making a safety argument. Grounding is what keeps a system answerable to something outside itself. The more of our intelligence we build, the more it matters that some of it is forced, structurally, to keep touching the world. Otherwise we get very good at building mirrors and call them minds.