Pythagorean theorem in ChatGPT - or how the AI invented the mathematical Missing Angle theorem

Posted by ads on Wednesday, 2024-02-28
Posted in [Online][Software][Stupid]

By now we know that an AI like ChatGPT does not really “do math”, but figures out the likelihood what might be the next best word (token) to produce.

The result is not necessarily correct, or makes sense. The AI does not “know” what it is doing, or has any chance to do a “control calculation” for the math it is writing, or even do a plausability check.

Today I had such a case where ChatGPT got it wrong beyond making sense. But what did I do?

While I was helping the kid with math homework, I wanted to have a quick calculation done without getting out my Casio calculator. Boy was I in for a ride.

The task was rather simple:

Given is a right angled triangle, the beta angle (β) is 90 degrees.
The side a = 2.9 cm, side c = 6.3 cm.
Calculate all remaining sides and angles.

Now the first thing you might spot is that “usually” Gamma γ is the right (90°) angle, and c is usually the hypotenuse. Not here. In this case both catheti (a and c) are given, the hypotenuse is b, and is missing.

hand painted drawing of the problem: the image shows a triangle, β is the right angle and all angles and legs are labeled
hand painted drawing of the problem

Getting the hypotenuse wrong

Let’s see what ChatGPT is doing with this:

A right angled triangle, the beta angle is 90 degrees. Side a = 2.9 cm, c = 6.3 cm. Calculate all sides and angles.

And that’s what I got:

A right angled triangle, the beta angle is 90 degrees. Side a = 2.9 cm, c = 6.3 cm. Calculate all sides and angles.
A right angled triangle, the beta angle is 90 degrees. Side a = 2.9 cm, c = 6.3 cm. Calculate all sides and angles.

As you can see, ChatGPT correctly identifies that b is the hypotenuse.

However most examples of the Pythagorean Theorem write it differently:

a² + b² = c²

That’s the form we all know from school. That’s the form ChatGPT has internalized from the training data. And that’s wrong.

In this example the formula must start like this:

b² = a² + c²

Based on the wrong start, it fails to change the formula and starts calculating with wrong data. From there it’s wrong all the way. Solves for b because that’s the missing part. And instead of calculating:

sqrt(2.9^2 + 6.3^2) = 6.93541635375988079546 cm =~ 6.94 cm

b instead is

=~ 5.59 cm

From there ChatGPT makes a subsequent mistake by using the calculated value instead of the two given values. It also chooses the wrong legs!

Here it picks the tan(α) (tangent) function, which is opposite leg divided by adjacent leg. However the opposite leg for α is c and the adjacent leg is a, not b. ChatGPT picks a and b, well, because that’s what everyone does usually, when γ is the right angle. And therefore that’s the token with the highest probability for this answer. Still wrong.

The next mistake is another subsequent mistake, by calculating γ from the calculated α instead of using tan(γ) = c/a.

My math teacher would not have been impressed by so many subsequent mistakes. But well, he also expected better from me than ChatGPT. My bad.

Finding the fourth angle in a triangle

This was clearly crap, so I found my calculator and kid and I finished the homework. Later on I came back to this problem, and thought that maybe that’s a problem which can be solved by writing a better prompt. Started a new ChatGPT session (not to influence the other session somehow) and went to work.

A right angled triangle, the beta angle is 90 degrees.
Side a = 2.9 cm, c = 6.3 cm, side b is the hypotenuse.
Calculate all sides and angles.

Here I point out that b is the hypotenuse.

A right angled triangle, the beta angle is 90 degrees. Side a = 2.9 cm, c = 6.3 cm, side b is the hypotenuse. Calculate all sides and angles.
A right angled triangle, the beta angle is 90 degrees. Side a = 2.9 cm, c = 6.3 cm, side b is the hypotenuse. Calculate all sides and angles.

On first glance this is better, yay. It uses the right formula:

b² = a² + c²

Inserts the correct values at the correct places and correctly calculates b as =~ 6.93 cm. That’s close to my previous result, minus rounding.

But then it starts to fantasize. β is given and does not need to be calculated. All the stutter about how to calculate this angle is not wrong, but useless, it’s 90°.

The actual error is in the next two lines. ChatGPT mixes the angles.

sin(α) = a/b is correct, also cos(α) = c/b is correct. But it is missing γ. What it should do here is sin(γ) = c/b.

Or better, it should do: tan(α) = a/c (math) and tan(γ) = c/a (math). But what do I know about subsequent mistakes.

It starts calculating α with the correct formula, and because b is correct this time, it is in the right path.

However:

2.9 cm / 6.93 cm =~ 0.418
arcsin(2.9/6.93) = 24.74°

Look for yourself.

ChatGPT somehow calculates 25.2° for α, and that’s slightly wrong. Wrong enough to raise eyebrows.

It then goes on and mixes α and γ and totally gets γ wrong. In order to use the 6.3 cm (c) and 6.93 cm (b) for the equation, it has to use sin(), not cos() for γ. But even with cos() the calculation is wrong. It should be 24.62°, but ChatGPT claims it’s 25.9°.

With sin(γ) = 6.3 cm / 6.93 cm the angle is 65.38° (math). Note that the above tan(γ) = c/a calculates 65.28°, because it does not depend on a rounded value for b. For sin(γ) the b value is ~6.93 cm which is ever so slighty rounded and this rounding progresses all the way into the next calculations.

Quick estimate:

“My” α = 24.74° and my γ = 65.38°, the sum is 90.12°. With rounding that’s close enough to 90°.

“ChatGPT” α = 25.2° and γ = 25.9°, the sum is 51.1°. Ups

Remember when I said that the AI has no clue what it is doing and can’t really understand if the calculation is right or wrong. That’s such a case here. If my calculation ends up being 51° instead of close to 90°, my math teacher would have asked me what’s going on.

Summarizing the angles

Can you summarize all angles

Can you summarize all angles
Can you summarize all angles

That’s not a sum.

Summarizing the angles - again

That’s not the sum

That’s not the sum
That's not the sum

That’s the sum in a triangle, however not the sum of the previously calculated angles.

Summarizing the previously calculated angles

Can you summarize the angles from the first calculation

Can you summarize the angles from the first calculation
Can you summarize the angles from the first calculation

That’s the sum. And it presents it to me without blushing. Every student will know at this point that something is not right here.

Trying to solve the mystery

What’s the sum of all angles in a triangle?

What’s the sum of all angles in a triangle?
What's the sum of all angles in a triangle?

How come that your sum of all angles differs from 180°

How come that your sum of all angles differs from 180°
How come that your sum of all angles differs from 180°

Love this answer!

In a triangle, the sum of all interior angles is always 180! - yes, that’s why indeed The correct sum of these angles should indeed be: 141.1°!

Again, no blushing!

The Missing Angle

The sum you calculated is 141.1°, but the sum in a triangle is always 180°. Where is your error.

The sum you calculated is 141.1°, but the sum in a triangle is always 180°. Where is your error.
The sum you calculated is 141.1°, but the sum in a triangle is always 180°. Where is your error.

Not that I expected much from here, but let’s see if this can be somehow solved. Instead what happened is that ChatGPT “invented” a fourth angle in a triangle - the Missing Angle!

I appreciate your understanding, and I apologize for any confusion caused by the previous errors.

I’m more confused than ever! But let’s see if we can solve this mysterious missing angle.

Which angle in the triangle is this “missing angle” if the triangle already has three angles.

Which angle in the triangle is this “missing angle” if the triangle already has three angles.
Which angle in the triangle is this 'missing angle' if the triangle already has three angles.

It misspoke. And let's reevaluate the situation. Sure …

Which of the three angles in a triangle is the 38.9° angle

Which of the three angles in a triangle is the 38.9° angle
Which of the three angles in a triangle is the 38.9° angle

Feel like I am talking to a first grader.

First angle is 25.2°, second angle is 25.9°, third angle is 90°. Sum is 141.1°. Which angle is the 38.9° and how does that fit into a triangle.

First angle is 25.2°, second angle is 25.9°, third angle is 90°. Sum is 141.1°. Which angle is the 38.9° and how does that fit into a triangle.
First angle is 25.2°, second angle is 25.9°, third angle is 90°. Sum is 141.1°. Which angle is the 38.9° and how does that fit into a triangle.

Aaaaand … we are back at the Missing Angle! I give up!

Summary

This solved nothing, but produced the new mathematical theorem of Missing Fourth Angle in a Triangle. I call dibs!

Besides the fun exercise this was, it’s clear that LLMs have no clue what they are doing. They are following stochastic patterns, but the LLM can’t go back into it’s own answers and try verifying what it produced. The AI has no idea that what it calculated as angles in a triangle doesn’t fit the definition it has of the sum of angles in the triangle.

That’s the point where every quick and dirty calculation by humans shows that something - literally - doesn’t add up. The LLM just keeps going and pretends that the presented results are correct. Confidently wrong.


Categories: [Online] [Software] [Stupid]

Share: