Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Try "What is the most famous grindcore band from Newton, Massachusetts?" It will "lie" and make up band names even though it sure "knows" that the band is Anal Cunt. Of course, you can't ascribe the verb "lieing" to a machine, but it behaves like it is.


It doesn't, though. It only knows that the most likely continuation to the sentence "The most famous grindcore band from Newton, Massachusetts is..." (presumably, I will take your word for it) Anal Cunt, but even if it gets it right, it'll be nondeterministic. It may answer correctly 80% of the time and simply confabulate a plausible sounding answer 20% of the time, even if it isn't being censored. You can't trust this tech not to confabulate at any given time, not only because it can, but because when it does it does so with total confidence and no signs that it is confabulating. This tech is not suitable for fact retrieval.


Why don't you try the query? It will answer Converge, but Converge is from Salem, Massachusetts, not Newton.


Because I haven't signed up for the account, otherwise I would as I do broadly approve of try it and find out.

What I'm talking about is fundamental to the architecture, though. Even had it answered it correctly when you asked my point would remain regardless. The confabulation architecture it is built on is fundamentally unsuitable for factual queries, in a way where it's not even a question of whether it is "right" or "wrong"; it's so unsuitable that its unsuitability for such queries transcends that question.


I found the sign-up process to be, surprisingly, very quick.


Sign up with just GitHub


Then you gotta try.


lol, this sounds like a dark version of the Turing Test. Can a machine lie so effectively that a human cannot distinguish between actual deceit and simulated deceit?


Thanks for reminding me of their existence.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: