Your brain is doing something pretty magical. It has done this since the day you were born and will do this until the day you die. What is happening is that your brain is predicting. Constantly, automatically, like your heartbeat or breathing pattern. Your brain never stops making predictions, and this is a good thing. It keeps you alive. When it’s right, nothing happens. When it’s wrong, it updates, and this is the essence of “learning.” But that is not the interesting part.

What is fascinating is that, according to theoretical neuroscientists, the brain does not sit there and predict passively. It acts, and because of this, in effect, we are all little scientists.

We create hypotheses that best explain our lived world and then design careful experiments to test whether our inferences are right or wrong. We do this all the time, everywhere, and all at once. From palpating the visual world with our eyes through to asking our favorite chatbot the right kinds of questions, we are curious creatures seeking explanations for the world we find ourselves in; what philosophers call inference to the best explanation.

Friston calls this active inference. The brain is an active, epistemically hungry agent that reaches out into the world, gets corrected by it, and updates its model of reality accordingly. The imperative behind all behavior can be neatly summarised as minimizing surprise; in the dual sense of resolving uncertainty and avoiding surprising, uncharacteristic outcomes. Negative surprise, technically, is the evidence for the world model installed in our brains. Which means we act to make our predictions come true — eluding surprise — and that is exactly the same as gathering evidence for our own existence.

This is what AI like Claude or ChatGPT will never do. Interestingly, you have already experienced this.

Remember the first time you saw your AI of choice hallucinate? It was a confusing moment, right? You probably wondered why or how it generated nonsense out of thin air. This immediate recognition, this feeling of “what is this thing doing?” is exactly your brain in active, hungry agent mode.

You tried to do something, like draft an essay, build an app, whatever, and the world (the AI) responded in a way that surprised you. It complicated your belief that this technology “knew what it was talking about.” At this exact moment, your brain immediately updated: “oh, okay, this can happen. Noted.” And you moved on with life with a nuanced faith in large language models.

The meta-point is that this is exactly what AI cannot, and will never, do. At least, not AI based on LLMs. Claude or ChatGPT cannot look at the world, see people getting frustrated, and realize, “oh shoot, people are mad at me, I gotta write better.” It’s… code. That is all it will ever be. Your chatbot has no notion that its output has any consequence, and therefore no notion of agency or selfhood. It cannot possess intentions, and therefore there is no reason behind its behavior.

Generating essays that say the wrong thing is annoying for students, but putting these technologies elsewhere is a whole different, and existential, discussion.

Would you trust a large language model to drive your children to school? If not, ask yourself what you really expect of artificial intelligence. You would expect it to care about your children and other road users: to be exquisitely context-sensitive and ask the right kind of questions when it is uncertain. In short, you expect AI to show all the hallmarks of natural intelligence. You expect it to be a “good driver,” an “honorable soldier,” and so on. You expect it to be like you and me – curious creatures who care about each other.

The machines we have possess none of these qualities. An autonomous weapons system that kills a civilian doesn’t hesitate the next time. There is no “next time” that is any different from this time. The kill was a token. The next target is a token. The system is, underneath everything, still just predicting. Still just code.

These technologies can make as many mistakes as we allow them to, but they never make better decisions moving forward. This is the precise danger. Their world model never updates; more to the point, they have no world model in the first place. We are trusting our reality to systems whose “learning” consists of training data and data centers, not the shared reality we all inhabit. No wonder there is such a disconnect between what these systems are and what we want or expect them to be.

So, can we build AI to be curious? In principle, we could. This would require replacing reinforcement learning with active inference. In other words, we have to abandon the notion that we can “train” our children, pets, and AI to be caring and curious. We have to ask what is actually needed for AI to become like us.

There are some obvious answers. AI has to be equipped with authentic agency, in the sense that it can choose among alternative futures by evaluating the consequences of its own choices. Crucially, this evaluation is not just in terms of reward or punishment. It turns on the opportunity to resolve uncertainty. That, in turn, necessitates the encoding of uncertainty and confidence, so the right choices can be made for this agent in this context. And finally, AI has to have a model of the consequences of its actions that is necessarily future pointing. The dénouement of this kind of aligned, agentic AI is that it will be genuinely curious about, for example, other road users. We will know it has arrived when your AI starts prompting you — and together, you establish some common ground and a shared narrative.

For decades, the conversation has been about how to make ourselves compatible with technology. How to adapt, retrain, reskill, keep up. How to contort human life around the shape of the artefacts we built. We optimized ourselves for the machine, instead of the other way around.

But when has a machine burned its finger and been consoled by its mother? When has a machine said the wrong thing and felt the social embarrassment that keeps it up at night years later? We are the ones who act and grieve and feel and learn. We are the ones with a world model so sophisticated, so consequence-saturated, so brutally updated by reality, that we built civilization with it.

This is why the question of an “artificial citizen” matters, and why we cannot answer it yet. You cannot hand a machine rights or sentience or a place among us without first asking whether it can do what a human brain does: act, predict, get corrected, and carry the consequence. That is the bar active inference sets. Until a machine clears it, every anthropomorphic word we use — AI “thinks,” “understands,” “is reasoning about the myriad steps of making a bowl of cereal” — is marketing.

So, we must begin building systems that have stakes. Systems that carry their mistakes the way a surgeon carries a lost patient, or a driver carries an accident. Systems that reach out into the world, get corrected by it, and are meaningfully scarred by what they find. Systems that, when they are wrong, pay the price themselves, rather than a human downstream who updates the system only for corporate or political interests.

Technology must be made in our image. Not the other way around. The line between machine cognizance and corporate fabrication is this: a system with no world model has no inner life to respect, no perspective to honor, no self to grant anything to. Its software is immortal. And, as such, is under no pressure to adapt to the world — or users — it encounters. To pretend otherwise disrespects both what machines could be and what humans are. But we need not pretend, because by reading this article there are two outcomes.

If a machine read this — fetched it, scanned it, crawled it — it parsed the text, hallucinated some, and produced tokens because the servers were on. Then…nothing. The machine is exactly what it was before it corrupted the internet and the article. If a human is reading — and we sure hope a human is — then something happened that no machine could replicate. You arrived curious, you predicted, you were corrected, you updated. You are not the same as you were three paragraphs ago, and you will never be again.

This is what we mean by the infinity of the brain and the void of the machine. Pretty magical, wouldn’t you say?

---

Ayush Prakash is a Montreal-based writer, operator, and Director of the Centre for Innovating the Future. He is the author of AI for Gen Z and the forthcoming The Transcendence of Money, and hosts a 150-episode podcast on AI, cognition, and technology. As Chief of Staff at New Sapience, he runs operations and institutional fundraising for a deterministic AI company building zero-hallucination systems for aerospace, defense, and critical infrastructure. Karl Friston is a theoretical neuroscientist and Professor of Neuroscience at University College London. He serves as Chief Scientist at Stanhope AI and is globally renowned for his pioneering work on the Free Energy Principle, a unified theory describing how biological agents interact with and infer the structure of the world.

---