Around October 2024, The infamous “How many R’s are in strawberry?” question has become a fascinating litmus test for Large Language Models, exposing a fundamental limitation in how these systems process text.
When asked directly, most LLMs including ChatGPT will confidently give incorrect answers, often claiming there are 2 R’s in “strawberry” when the correct answer is 3 (st-r-a-w-be-rr-y). However, by employing an NLP-based methodology that gives the LLM access to external tools and character-counting functions, the model can solve this problem accurately by breaking down the word character by character rather than relying on its tokenized understanding.
This demonstrates a crucial insight about AI capabilities: while LLMs excel at language understanding and generation, they struggle with precise character-level tasks that seem trivial to humans, but when augmented with appropriate tools and structured approaches, they can overcome these limitations and provide reliable solutions.
It is possible this is already obselete in a year when they will just put it into the training data and LLM would give a solution regardless.