By now we know that an AI like *ChatGPT* does not really “do math”, but figures out the likelihood what might be the next best word (token) to produce.

The result is not necessarily correct, or makes sense. The AI does not “know” what it is doing, or has any chance to do a “control calculation” for the math it is writing, or even do a plausability check.

Today I had such a case where *ChatGPT* got it wrong beyond making sense. But what did I do?

While I was helping the kid with math homework, I wanted to have a quick calculation done without getting out my *Casio* calculator. Boy was I in for a ride.

The task was rather simple:

Given is a right angled triangle, the beta angle (β) is 90 degrees.

The side a = 2.9 cm, side c = 6.3 cm.

Calculate all remaining sides and angles.

Now the first thing you might spot is that “usually” Gamma γ is the right (90°) angle, and `c`

is usually the hypotenuse. Not here. In this case both catheti (`a`

and `c`

) are given, the hypotenuse is `b`

, and is missing.

## Getting the hypotenuse wrong

Let’s see what *ChatGPT* is doing with this:

A right angled triangle, the beta angle is 90 degrees. Side a = 2.9 cm, c = 6.3 cm. Calculate all sides and angles.

And that’s what I got:

As you can see, *ChatGPT* correctly identifies that `b`

is the hypotenuse.

However most examples of the Pythagorean Theorem write it differently:

a² + b² = c²

That’s the form we all know from school. That’s the form *ChatGPT* has internalized from the training data. And that’s wrong.

In this example the formula must start like this:

b² = a² + c²

Based on the wrong start, it fails to change the formula and starts calculating with wrong data. From there it’s wrong all the way. Solves for `b`

because that’s the missing part. And instead of calculating:

`sqrt(2.9^2 + 6.3^2) = 6.93541635375988079546 cm =~ 6.94 cm`

`b`

instead is

`=~ 5.59 cm`

From there *ChatGPT* makes a subsequent mistake by using the calculated value instead of the two given values. It also chooses the wrong legs!

Here it picks the `tan(α)`

(tangent) function, which is *opposite leg* divided by *adjacent leg*. However the opposite leg for `α`

is `c`

and the adjacent leg is `a`

, not `b`

. *ChatGPT* picks `a`

and `b`

, well, because that’s what everyone does usually, when `γ`

is the right angle. And therefore that’s the token with the highest probability for this answer. Still wrong.

The next mistake is another subsequent mistake, by calculating `γ`

from the calculated `α`

instead of using `tan(γ) = c/a`

.

My math teacher would not have been impressed by so many subsequent mistakes. But well, he also expected better from me than *ChatGPT*. My bad.

## Finding the fourth angle in a triangle

This was clearly crap, so I found my calculator and kid and I finished the homework. Later on I came back to this problem, and thought that maybe that’s a problem which can be solved by writing a better prompt. Started a new *ChatGPT* session (not to influence the other session somehow) and went to work.

A right angled triangle, the beta angle is 90 degrees.

Side a = 2.9 cm, c = 6.3 cm, side b is the hypotenuse.

Calculate all sides and angles.

Here I point out that `b`

is the hypotenuse.

On first glance this is better, yay. It uses the right formula:

b² = a² + c²

Inserts the correct values at the correct places and correctly calculates `b`

as `=~ 6.93 cm`

. That’s close to my previous result, minus rounding.

But then it starts to fantasize. `β`

is given and does not need to be calculated. All the stutter about how to calculate this angle is not wrong, but useless, it’s `90°`

.

The actual error is in the next two lines. *ChatGPT* mixes the angles.

`sin(α) = a/b`

is correct, also `cos(α) = c/b`

is correct. But it is missing `γ`

. What it should do here is `sin(γ) = c/b`

.

Or better, it should do: `tan(α) = a/c`

(math) and `tan(γ) = c/a`

(math). But what do I know about subsequent mistakes.

It starts calculating `α`

with the correct formula, and because `b`

is correct this time, it is in the right path.

However:

2.9 cm / 6.93 cm =~ 0.418

arcsin(2.9/6.93) = 24.74°

*ChatGPT* somehow calculates `25.2°`

for `α`

, and that’s slightly wrong. Wrong enough to raise eyebrows.

It then goes on and mixes `α`

and `γ`

and totally gets `γ`

wrong. In order to use the `6.3 cm`

(`c`

) and `6.93 cm`

(`b`

) for the equation, it has to use `sin()`

, not `cos()`

for `γ`

. But even with `cos()`

the calculation is wrong. It should be `24.62°`

, but *ChatGPT* claims it’s `25.9°`

.

With `sin(γ) = 6.3 cm / 6.93 cm`

the angle is `65.38°`

(math). Note that the above `tan(γ) = c/a`

calculates `65.28°`

, because it does not depend on a rounded value for `b`

. For `sin(γ)`

the `b`

value is `~6.93 cm`

which is ever so slighty rounded and this rounding progresses all the way into the next calculations.

### Quick estimate:

“My” `α`

= `24.74°`

and my `γ`

= `65.38°`

, the sum is `90.12°`

. With rounding that’s close enough to `90°`

.

“ChatGPT” `α`

= `25.2°`

and `γ`

= `25.9°`

, the sum is `51.1°`

. **Ups**

Remember when I said that the AI has no clue what it is doing and can’t really understand if the calculation is right or wrong. That’s such a case here. If my calculation ends up being `51°`

instead of close to `90°`

, my math teacher would have asked me what’s going on.

### Summarizing the angles

Can you summarize all angles

That’s not a sum.

### Summarizing the angles - again

That’s not the sum

That’s the sum in a triangle, however not the sum of the previously calculated angles.

### Summarizing the previously calculated angles

Can you summarize the angles from the first calculation

That’s the sum. And it presents it to me without blushing. Every student will know at this point that something is not right here.

### Trying to solve the mystery

What’s the sum of all angles in a triangle?

How come that your sum of all angles differs from 180°

Love this answer!

`In a triangle, the sum of all interior angles is always 180!`

- yes, that’s why indeed `The correct sum of these angles should indeed be: 141.1°`

!

Again, no blushing!

### The Missing Angle

The sum you calculated is 141.1°, but the sum in a triangle is always 180°. Where is your error.

Not that I expected much from here, but let’s see if this can be somehow solved. Instead what happened is that *ChatGPT* “invented” a fourth angle in a triangle - the *Missing Angle*!

`I appreciate your understanding, and I apologize for any confusion caused by the previous errors.`

I’m more confused than ever! But let’s see if we can solve this mysterious missing angle.

Which angle in the triangle is this “missing angle” if the triangle already has three angles.

It `misspoke`

. And `let's reevaluate the situation`

. Sure …

Which of the three angles in a triangle is the 38.9° angle

Feel like I am talking to a first grader.

First angle is 25.2°, second angle is 25.9°, third angle is 90°. Sum is 141.1°. Which angle is the 38.9° and how does that fit into a triangle.

Aaaaand … we are back at the *Missing Angle*! I give up!

## Summary

This solved nothing, but produced the new mathematical theorem of *Missing Fourth Angle in a Triangle*. I call dibs!

Besides the fun exercise this was, it’s clear that LLMs have no clue what they are doing. They are following stochastic patterns, but the LLM can’t go back into it’s own answers and try verifying what it produced. The AI has no idea that what it calculated as angles in a triangle doesn’t fit the definition it has of the sum of angles in the triangle.

That’s the point where every quick and dirty calculation by humans shows that something - literally - doesn’t add up. The LLM just keeps going and pretends that the presented results are correct. Confidently wrong.

Share: