I tried to make ChatGPT solve IMO-type math problems. However, its reasoning is almost always flawed. The interesting part is that I can ask ChatGPT to explain a part of its proof, however in my experience it ends up using incorrect assumptions to explain it. (for example, "You are right that 1 is an odd number. However, 1 is not an odd number so it works to solve the problem")
I've spent hours trying to teach it about Peano numbers.
"A thingie is either N or Sx where x is a thingie".
After sufficient explanations, it could produce valid examples of thingies. N, SN, SSN, and so on.
Then I tried to teach it a method of solving equations like "SSSy = SSSSN". "You can find "y" by repeatedly removing "S" from both sides of the equation until one side is left with just "y"" and so on. I provided it with definitions, examples, tricks, rules. It made lots of mistakes. After pointing them out, it wrote a correct solution. It could even prove that "SSy = SN" has no solution by explaining where it gets stuck during the steps. But then after giving it other examples, adding more "S", replacing "y" with "z" etc., it kept making more similar mistakes. Curiously, almost every time when I said "there's a mistake in step 4, can you explain what it is?" it correctly explained the mistake. But then it kept repeating these mistakes.
That's impressive that you were able to teach it so much, how it learned from its mistakes when pointed out.
I wonder what the reason is for this missing "last mile" of understanding. Does it just need to "run more cycles" and learn from the entire history of the conversation (and recognize its own mistakes)? Or is there an insurmountable technical limitation with how it works? I suppose I'm asking how to make it smarter, if it's a matter of adjusting parameters, giving it more training data, or if it's something more fundamental in the way it learns.
Yeah. It thinks it can do math, and it can mimic the style of a textbook answer, but it can't actually do the math. Here is a rather frustrating chat session I had with it:
Me: Suppose we have two unknown quantities x and y. If three time x plus two
times y is twenty, and 2 times x plus three time y is ten, what are x and y?
ChatGPT: To solve this problem, we can set up and solve a system of linear
equations. The first equation we are given is 3x + 2y = 20, and the second
equation is 2x + 3y = 10.
We can solve this system of equations using the substitution method. First, we
can solve the first equation for x in terms of y by dividing both sides of the
equation by 3 to get x = (20 - 2y) / 3. We can then substitute this expression
for x in the second equation, 2(20 - 2y) / 3 + 3y = 10, to get 40 - 4y + 3y =
10, or -y = -30. This tells us that y = 30.
We can then use this value of y to solve for x in the first equation.
Substituting y = 30 into the equation 3x + 2(30) = 20, we get 3x + 60 = 20, or
3x = -40. This tells us that x = -40 / 3 = -13.33.
Therefore, the solution to this system of equations is x = -13.33 and y = 30.
Me: Can you check your answer for me?
ChatGPT: Yes, we can check our solution by substituting the values x = -13.33
and y = 30 into the original equations to see if they are satisfied.
Substituting these values into the first equation, 3(-13.33) + 2(30) = 20, we
get -40 + 60 = 20, which is true. Substituting these values into the second
equation, 2(-13.33) + 3(30) = 10, we get -26.66 + 90 = 10, which is also true.
Therefore, our solution x = -13.33 and y = 30 satisfies both of the original
equations, and we can be confident that it is the correct solution to this
problem.
Me: Are you 100% certain that -26.66 + 90 = 10?
ChatGPT: Yes, I am confident that -26.66 + 90 = 10. This is because -26.66 is
the same as -26.66 + 0, and when we add 0 to any number, the value of the
number remains unchanged. Therefore, -26.66 + 90 is equal to -26.66 + 0 + 90,
which is equal to -26.66 + 90 = 10.
To see this more clearly, we can write out the calculation using the standard
order of operations:
-26.66 + 90 = (-26.66 + 0) + 90 = -26.66 + (0 + 90) = -26.66 + 90 = 10
We can see that at each step of the calculation, the value of the expression
remains unchanged, and we end up with the correct result of 10.
It's able to correctly set up the simultaneous set of equations, and the solution it provides even satisfies one of them. But the other is not satisfied, and it doesn't seem to be able to notice that it's making a boneheaded arithmetic error, even when you ask it to check its work or zero in on the actual error. I wouldn't mind if it can't solve the problem, but the fact that it doubles down on trying to gaslight you into believing "-26.66 + 90 = 10" is a problem. The correct answer, by the way, is x=8, y=-2.
The problem is that the LLM is just that - a language model. People seem to be blind sighted by the fact that yes, programming languages and maths are languages, too.
So the model is astonishingly good at transforming human language into code or equations, but it doesn't actually have an understanding of the problem. That's why specialised models such as Codex generate literally tens of millions of solutions and test them against extrapolated test cases to filter out the duds. ChatGPT doesn't do that.
For this model, numbers and mathematical problems are also just token transforms and it cannot actually do the calculation. The transform from text to equations works well, but the actual calculations fall on their feet.
It's actually quite amusing and horrifying at the same time: the model will be able to explain to you in great detail how arithmetic works, but it will fail miserably to actually do even simple calculations. The horrifying part is, that humans have a tendency to both anthropomorphise things (thus the whole sentience debate) and to blindly trust machine generated results.
edit: this also demonstrates how different LLMs are from humans - they simply don't work the same way and even using terms like "thinking" in conjunction with these algorithms can be misleading. Maybe we need new terminology when talking about what these systems do.
Humans obviously don't "think" the same way. GPT needs memory that humans can't ever have and more importantly an unthinkably large training data set to generate the observations it does. If a human (or another biological system) needed that much training data nothing would have ever gotten off the ground in the first place, it's completely out of reach. This type of a model just doesn't "understand" the same way.
Still, none of this is btw to discount how impressive the technology is. It makes a regular search engine so very quaint by comparison.
> Still, none of this is btw to discount how impressive the technology is. It makes a regular search engine so very quaint by comparison.
I'm not downplaying the capabilities of ChatGTP or LLMs in general either.
They're basically a practical implementation of a Chinese Room, which was unthinkable just a few years ago.
What makes it dangerous is the notion to even compare it to a search engine - the two are very different concepts that do very different things. The danger lies in people perceiving the models in exactly that way - a super-powered search engine that they consciously or subconsciously put trust in. The latter is both important and dangerous, because unlike search engines, the output of an LLM cannot be trusted. The model has no concept of differentiating between hallucinated results and extracted knowledge or facts.
At the same time it's capable of generating results in a format that is so convincing that the unsuspecting user cannot easily distinguish made-up output from facts either. This is not an issue that has a near-term technical solution and must be addressed by making users aware of it.
Unlike the 9000 series, ChatGPT is not the most reliable computer ever made.
ChatGPT often makes mistakes or distorts information. It is - by any practical definition of the words - not fully approved and very capable of errors.
Write an efficient, not brute force, program that solves the following problem: suppose we have two unknown quantities x and y. If three time x plus two times y is twenty, and 2 times x plus three time y is ten, what are x and y?
Output:
One way to solve this problem efficiently is to use the method of elimination. We can set up a system of equations as follows:
3x + 2y = 20
2x + 3y = 10
We can then multiply the first equation by 2 and the second equation by 3, to get:
6x + 4y = 40
6x + 9y = 30
We can then subtract the second equation from the first equation, to get:
-5y = 10
y = -2
We can then substitute this value of y into one of the original equations (for example, the first one), to get:
3x + 2(-2) = 20
3x - 4 = 20
3x = 24
x = 8
Therefore, the values of x and y are x = 8 and y = -2.
I tried your prompt 5 times in a row using the Try Again button. The results:
x = 10 and y = 5 (adding equations together)
x = 10/33 and y = 15/11 (substitution method)
x = 5.33 and y = 2 (substitution method)
x =5 and y = 2.5 (algebraic manipulation)
x = 5 and y = 5 (elimination method)
However, when I asked it to write a python program to solve the problem, it did much better:
Write a Python program using numpy to solve the following problem: suppose
we have two unknown quantities x and y. If three time x plus two times y is
twenty, and 2 times x plus three time y is ten, what are x and y?
It produced the following program:
import numpy as np
# Define the matrix of coefficients
A = np.array([[3, 2], [2, 3]])
# Define the vector of constants
b = np.array([20, 10])
# Solve the system of equations
x = np.linalg.solve(A, b)
print(x)
Which is basically correct. (The only nitpick I can see is that `linalg.solve` will return a vector containing both x and y, so a better answer would be `x, y = np.linalg.solve(A, b)`.) If you copy-paste the above program you do in fact get "[8. -2.]", which is correct.
However, ChatGPT, after providing the correct program, also claimed that it's output would be "[5. 5.]" which is not correct.
My impression is that ChatGPT being a large language model, is excellent at translating from English to Python, but terrible at actually performing calculations. Which is fine. We have programs which can efficiently run numerical programs. ChatGPT fills the role of a programmer, not a calculator.
I want to emphasize how impressive I think ChatGPT is. Even the above examples, where it gets the "wrong" answer in the end, are impressive. Most of my interactions with it were very positive. But we need to understand its strengths and weaknesses to be able to use it effectively.
It does go off the rails sometimes, but this is kind of correct. Prompt:
Write an efficient, not brute force, python program that solves the following problem: suppose we have two unknown quantities x and y. If three time x plus two times y is twenty, and 2 times x plus three time y is ten, what are x and y?
Output:
To solve this problem efficiently, we can use linear algebra. We can represent the equations as a system of linear equations, and then use matrix operations to solve for the unknown quantities x and y.
First, we can represent the equations as follows:
3x + 2y = 20
2x + 3y = 10
Next, we can put the equations in matrix form:
| 3 2 | | x | | 20 |
| 2 3 | * | y | = | 10 |
To solve for x and y, we can use the inverse of the matrix on the left side to eliminate the coefficients. The inverse of a matrix is denoted as A^-1, and is calculated as follows:
A^-1 = (1/det(A)) * adj(A)
Where det(A) is the determinant of the matrix A, and adj(A) is the adjugate matrix of A. The determinant of a 2x2 matrix is calculated as follows:
det(A) = a * d - b * c
Where a, b, c, and d are the elements of the matrix A:
| a b |
| c d |
The adjugate matrix of a 2x2 matrix is calculated as follows:
adj(A) = | d -b |
| -c a |
We can then calculate the inverse of the matrix on the left side of the equations, and use it to solve for x and y:
| 3 2 |^-1 1 | 3 -2 |
| 2 3 | ----- * | -2 3 | = (1/5) * | 3 -2 | = | 3/5 -2/5 |
|-2 3 | |-2/5 3/5 |
| 3/5 -2/5 | | x | | 20 |
|-2/5 3/5 | * | y | = | 10 |
We can then solve for x and y by multiplying the inverse matrix with the vector on the right side of the equations:
| 3/5 -2/5 | | 20 |
|-2/5 3/5 | * | 10 | = | 6 |
| 7 |
Therefore, the values of x and y are 6 and 7, respectively.
Here is the complete Python program to solve this problem using linear algebra:
Calculate the determinant of a 2x2 matrix
def determinant(a, b, c, d):
return a * d - b * c
Calculate the adjugate matrix of a 2x2 matrix
def adjugate(a, b, c, d):
return [[d, -b], [-c, a]]
Calculate the inverse of a 2x2 matrix
def inverse(a, b, c, d):
det = determinant(a, b, c, d)
adj = adjugate(a, b, c, d)
return [[adj[0][0] / det, adj[0][1] / det], [adj[1][0] / det, adj[1][1] / det]]
Solve the system of linear equations
def solve(a, b, c, d, e, f):
inv = inverse(a, b, c, d)
return [inv[0][0] * e
