The noise around artificial intelligence tends towards extremes: either it will solve everything or it will destroy everything. Both positions are poor maps of the territory. To use AI usefully — and to avoid placing more trust in it than it deserves — it helps to understand what it can do and, above all, what it cannot.
Some limitations are circumstantial: things AI does not do well today but will probably improve at. And some are structural: things that current AI cannot do for fundamental reasons that are not solved simply by more data or more computing power.
This article is about the second kind.
The most common mistake when thinking about AI
Language models like GPT, Claude, or Gemini produce text that sounds coherent, informed, even brilliant. This leads to an intuitive but mistaken conclusion: that behind the text there is understanding.
There is not. Or at least, not the kind humans mean when they talk about understanding something.
What exists is an extraordinarily sophisticated statistical system that has processed vast quantities of human text and learned to predict which words tend to follow others in certain contexts. The result is text that resembles what someone who understands would produce. But the similarity of the output does not imply similarity of the process.
The philosopher John Searle illustrated this in 1980 with his Chinese room thought experiment: someone who does not speak Chinese can follow rules to respond to Chinese symbols with other Chinese symbols in a way that appears perfectly fluent to whoever reads the responses. But that person does not understand Chinese. They are only following rules.
AI does not understand what it generates
The most important practical consequence of this is that AI does not detect its own errors when it has no way of verifying them externally.
A system that understands can notice when something does not add up. A system that predicts plausible text has no such mechanism. If the training data contains errors, or if the user’s context steers the response in a wrong direction, the model will produce plausible text in that direction without any internal signal that something is wrong.
This is why models hallucinate with confidence: they invent citations, attribute ideas to the wrong authors, assert facts that do not exist. They are not lying; they are generating what is statistically plausible. The distinction matters for understanding how to use them.
No body, no lived experience
Human beings do not learn only through language. We learn through the body, through emotion, through the physical consequences of our actions. The heat of fire, the weight of exhaustion, the relief of finishing something difficult. That sensory and affective dimension permeates all human knowledge.
AI has no body. It has no personal history. It has not experienced the fear of losing a job, the joy of a personal achievement, or the frustration of repeatedly failing at something. It can generate text about those experiences because it has processed millions of human accounts that describe them. But it has no access to what those experiences mean from the inside.
This has direct consequences. AI can talk about grief, but it does not know what it is to lose someone. It can talk about motivation, but it does not experience wanting something. It can help you draft a difficult message, but it has no intuition about the specific relationship behind it.
AI is brilliant with explicit knowledge — what can be written and described — and blind to tacit knowledge — what can only be known through experience.
The causality problem
AI models are extraordinarily good at finding correlations in large amounts of data. They are much worse at causal reasoning: understanding why something happens, not just which things tend to occur together.
Correlation and causation are different things, and confusing them produces serious errors. The number of doctors in a city correlates with disease rates — more doctors where there are more sick people — but doctors do not cause illness. A system that learns correlations without causation can produce conclusions that sound reasonable and are profoundly wrong.
Causal reasoning requires a model of the world: understanding which mechanisms produce which effects. Language models do not have a model of the world. They have a statistical representation of how the world is described in text. That is not the same thing.
What this means for you
None of the above means AI is not useful. It is, enormously, within its domain. But its domain has real limits.
Use it for tasks where errors are detectable and revisable: writing, synthesis, exploring ideas, reformulation, first drafts. These are contexts where you can verify the output with relative ease.
Be more cautious when the error is not easy to detect: specialist diagnoses, complex legal or medical reasoning, causal analysis in domains you do not know well. In those contexts, the apparent confidence of the output can mask important errors.
And reserve for yourself what AI genuinely cannot do: understand the full context of your specific situation, act with real accountability for the consequences, or substitute the judgement that comes from having lived through something. Those things will not be delegatable — not because AI will not improve, but because they are part of what it means to be the person making the decisions.
