You’d think that with all the hype, all the jaw-dropping examples of AI writing poetry, coding websites, & acing medical exams, that the latest & greatest model, GPT-5, would have no problem with a little bit of math. Right? Ask it to add two numbers, & it should be as easy as… well, adding two numbers.

Turns out, it’s not that simple.

If you’ve been playing around with GPT-5, you might have noticed something a little… off. You ask it a seemingly basic math problem, something you could solve on a napkin, & it gives you an answer with all the confidence in the world. And that answer is just plain WRONG. It's a widespread issue, and honestly, it’s been a bit of a letdown for many. Social media has been flooded with examples of GPT-5 stumbling on simple arithmetic. So what is going on? Why does this incredibly powerful, brainy-sounding AI fail at something a $5 calculator can do flawlessly?

Here’s the thing: it’s not because it’s “dumb.” It’s because of how it thinks. And understanding that is the key to actually getting it to work for you.

Honestly, it all comes down to this: Large Language Models (LLMs) like GPT-5 are not calculators. They are not logical reasoning machines. At their core, they are incredibly sophisticated mimics & pattern-matchers. Think of them less like a math professor & more like a student who has read every book in the library but hasn't actually learned how to think for themselves.

This has been a long-standing issue, with critics like Gary Marcus pointing out that these models have always failed in the same qualitative ways, including basic math & reasoning.

This is a term that gets thrown around a lot by AI researchers, & it’s the perfect description. An LLM works by predicting the next most likely word (or “token”) in a sequence. When you ask it "2 + 2 =", it doesn't calculate the answer. It scans its unimaginably vast training data of text from the internet & books, sees that the sequence "2 + 2 =" is almost always followed by "4", & so it gives you "4".

It’s just parroting a pattern it has seen a million times.

The problem arises when you give it a problem it hasn't seen as often, like a multi-digit addition or a slightly weird decimal subtraction. For instance, one test showed it failing at "8.8 - 8.11". The model might see patterns that look similar but are mathematically incorrect, leading it to generate a plausible-sounding but wrong answer. It’s a game of probabilities, not precision. Math, on the other hand, demands absolute precision. There’s no room for “close enough.”

This is the big one. Humans use symbolic reasoning to do math. We understand that the symbol "5" represents a quantity, the symbol "+" represents an operation, & we manipulate these abstract symbols according to a set of rules. We can apply these rules to numbers we've never seen before.

LLMs don't do this. They process text as tokens, which are like fragments of words. They don't have an inherent understanding of what the numbers or operators mean. This is why they can be so fragile. Change a number in a word problem from "5" to "five," & the performance can drop dramatically. Increase the number of steps in a problem, & the chances of an error skyrocket because each step is another prediction, another chance to go off the rails. This fragility is a core limitation; the model isn't performing genuine logical reasoning.

Another HUGE issue is the training data itself. The internet is filled with math problems, but it’s also filled with errors, inconsistent formatting, & questions without clear, step-by-step solutions. The model learns from all of it—the good, the bad, & the ugly. It might learn a correct formula from a textbook page & then learn a completely wrong application of it from a forum post. It has no built-in verifier to tell the difference. This means its mathematical knowledge is a byproduct of its language training, not a core, validated skill.

So, we have a system that's built for language, not logic; that mimics rather than understands; & that learned from a messy, unreliable dataset. When you look at it that way, it’s not so surprising it gets math wrong.

Here's the good news. You're not totally helpless. While you can't fundamentally change the AI's architecture, you can change how you interact with it. It’s less about "teaching" it math & more about giving it guardrails & the right tools for the job.

This is the most common & accessible method. You can’t just throw an equation at GPT-5 & expect the best. You need to guide its "thinking" process.

The magic phrase is often "Think step-by-step."

When you add this to your prompt, you're forcing the LLM to slow down & lay out its reasoning process. Instead of jumping to a final (and likely wrong) answer, it generates a chain of thought.

Let's say you want to solve a word problem: "A pizza is cut into 8 slices. If John eats 3 slices & Jane eats 2, what fraction of the pizza is left?"

Bad Prompt:

GPT-5 might just spit out a random fraction.

Good Prompt:

By doing this, you're asking it to perform a sequence of simpler predictions:

This process makes it much more likely to arrive at the correct answer because each individual step is a common, well-established pattern in its training data. It’s still pattern-matching, but you're guiding it down a path of more reliable patterns.

Be warned, though: this method is still "fragile." A small error in an early step can cause the whole thing to crumble, & the model won't necessarily notice. It’s a good first step, but not a foolproof solution.

This is where things get REALLY interesting & much more powerful. If the LLM is bad at math, why make it do math at all? Instead, let it do what it's good at—understanding language—& then hand off the actual calculation to a tool that’s built for it.

This is the idea behind "tool use" or "agent mode," which researchers are increasingly focusing on. The AI's job isn't to be the mathematician; its job is to be the project manager that knows when to call the mathematician.

For a user, this might look like the AI saying, "Okay, I need to calculate this. I'll use my calculator tool." But behind the scenes, a more sophisticated process is happening. Researchers are even building hybrid architectures that deeply integrate symbolic reasoning modules (like decision trees or logic engines) with LLMs. This creates a system that gets the best of both worlds: the LLM's language flexibility & the symbolic tool's logical rigor.

This is EXACTLY the kind of problem we think about at Arsturn. Businesses can't afford to have their customer service bots making simple math errors. Imagine a customer asking for a price quote based on volume, or trying to calculate shipping costs. A wrong answer isn't just an embarrassing quirk; it's lost trust & lost revenue.

This is why a no-code platform like Arsturn is so powerful. It allows a business to build a custom AI chatbot trained on their own data, but crucially, it can be designed to handle these situations intelligently. Instead of trying to "teach" the chatbot every possible mathematical calculation, you can build it so that when it recognizes a query requiring a precise calculation, it doesn't try to guess. It can be set up to use an external tool—a pricing API, a shipping calculator, or even just a basic math function—to get a guaranteed, 100% correct answer. The chatbot handles the conversation, understands the user's needs, & then calls the right tool for the job before presenting the final, accurate information back to the user. It’s about building a reliable automated system, not just a talkative one.

For businesses looking to automate lead generation or customer engagement, this reliability is non-negotiable. With Arsturn, you can build a conversational AI platform that forges meaningful connections with your audience, providing personalized &—most importantly—accurate information 24/7. It’s not just about answering questions; it’s about providing the right answers.

The initial disappointment with GPT-5's math skills highlights a fundamental truth about the current state of AI. Simply making models bigger and feeding them more data—the "scaling" approach—isn't going to magically solve these core reasoning issues. The path forward seems to be in creating smarter, more integrated systems.

We need to stop thinking of LLMs as all-knowing oracles & start treating them more like incredibly capable interns. They are fantastic at understanding requests, parsing information, & communicating results. But for specialized, high-precision tasks like math, they need supervision & the right tools.

So next time GPT-5 tells you that 5.9 = 5.11 + 1.2, don't throw your hands up in despair. Just remember what it is & how it works. Guide it, give it the right tools, or use a platform designed to manage these shortcomings.

Hope this was helpful! Let me know what you think. Have you run into any crazy math errors with the new models?