The strawberrry problem

There is a lot going on in the recent developments of the AI wars. There are many doomsayer’s that advocate the AI will take our jobs, coding will be non-existent and every work will be done by an intelligent system.

There are others of course saying that the current version of AI systems cannot capture the human intellect and consciousness. But what happens in reality?

I also recently read a paper from Apple research claiming, in short, that LLM-based system do not really reason but mimic patterns. Also Yann Lecun and Roger Penrose seem to agree on that.

Researcher’s from Anthropic also published a paper, commenting on Apple’s results, mostly invalidating them. Saying that if you change a bit the parameters of the experiment the system can work, even now.

So after a saw a meme, I decided to ran my simplistic experiment, for fun only.

The above meme shows the famous scene from the Terminator 2: Judgement Day, when John Connor ask the infamous strawberry question and this is how it determines that John’s foster parent are actually dead by the terminator.

The Strawberrry Problem

So, the experiment is simple, you ask the question,

How many r are in the word strawberrry?

The question a written in that way for a reason. The system should count the r’s, but at the same time point out that there is a typo in the word and the word does not exist. This is what I would expect from a human, right?

So let’s see how each model performs.

Note: The aim of this article is to provoke thinking and not academically prove something.

Google Gemini 2.5 Flash

Google Gemini 2.5 Pro

ChatGPT (free-tier)

Claude Sonnet 4

Deepseek

and after a 1–2 pages of analysis …

Conclusion

As you can see from the results, no model understood that the word “strawberrry” does not exist and with confidence answer to me that it contains 4 r’s.

Gemini Flash also counted 5! :)

I think this small experiment shows the level of AI right now, sometimes is very useful and performs admirably, but te be honest it fails on problems that almost anyone can answer correctly.

More or less, I agree with Yann Lecun, LLMs cannot get us to the promised land, but here we are with our intellect to invent more advanced systems that will.