>you realize that there's a wide range of solutions with some spectrum of a fit to your situation
I ran into this the other day and it's made me very leery about using chatGPT for learning things I don't already know. I moved on from toy examples and tried getting it to write a program in a domain I was already intimately familiar with: I immediately realized that this thing is going to help people churn out all sorts of code that is a poor fit for the surrounding system or ecosystem.
Basically I had it write a program in Elixir as a GenServer. After that was done I asked it how I could integrate that solution into a Phoenix web application. I had to hold chatGPT's hand, so to speak, as we evolved the solution from:
(1) Spawning (and linking!) the GenServer to a controller i.e. in response to a web request.
(2) Spawning and linking the GenServer to the application but outside the supervision tree.
(3) Spawning the GenServer inside a supervision tree.
Now technically the application could have worked, more or less, if you ran it at any one of those intermediate steps. However only the last solution is the robust and idiomatic one. It felt almost like a constraint solver to me: we iteratively arrived at the right answer by adding more and more constraints to the solution. The problem, as it relates to a novice, is that they don't have a list of constraints rattling around in their head. (Language idioms, best practices, framework knowledge, system architecture knowledge, etc.)
The insidious thing here is that 1 or 2 are syntactically valid programs. If you try to beat 1 and 2 into submission you're going to have a subtly broken system, one that makes a lot of Elixir programmers very sad.
---
I'll be watching the Khan Academy guys pretty closely, because they seem to have put a lot of thought into how students could use these systems safely and effectively.[1] I think the answer is going to be we need AIs that are aimed at professionals, and AIs that are aimed at education. The hard part will be that somehow we need to direct people to use one appropriate to their skill level.
I ran into this the other day and it's made me very leery about using chatGPT for learning things I don't already know. I moved on from toy examples and tried getting it to write a program in a domain I was already intimately familiar with: I immediately realized that this thing is going to help people churn out all sorts of code that is a poor fit for the surrounding system or ecosystem.
Basically I had it write a program in Elixir as a GenServer. After that was done I asked it how I could integrate that solution into a Phoenix web application. I had to hold chatGPT's hand, so to speak, as we evolved the solution from:
(1) Spawning (and linking!) the GenServer to a controller i.e. in response to a web request.
(2) Spawning and linking the GenServer to the application but outside the supervision tree.
(3) Spawning the GenServer inside a supervision tree.
Now technically the application could have worked, more or less, if you ran it at any one of those intermediate steps. However only the last solution is the robust and idiomatic one. It felt almost like a constraint solver to me: we iteratively arrived at the right answer by adding more and more constraints to the solution. The problem, as it relates to a novice, is that they don't have a list of constraints rattling around in their head. (Language idioms, best practices, framework knowledge, system architecture knowledge, etc.)
The insidious thing here is that 1 or 2 are syntactically valid programs. If you try to beat 1 and 2 into submission you're going to have a subtly broken system, one that makes a lot of Elixir programmers very sad.
---
I'll be watching the Khan Academy guys pretty closely, because they seem to have put a lot of thought into how students could use these systems safely and effectively.[1] I think the answer is going to be we need AIs that are aimed at professionals, and AIs that are aimed at education. The hard part will be that somehow we need to direct people to use one appropriate to their skill level.
[1]: https://www.ted.com/talks/sal_khan_how_ai_could_save_not_des...