AI Chatbots Remain Overconfident — Even When They’re Wrong: Large Language Models appear to be unaware of their own mistakes, prompting concerns about common uses for AI chatbots.
Technology
68
Beiträge
42
Kommentatoren
857
Aufrufe
-
I’m pretty much done with them except for some search
Not even a good use case either, especially when it spews such bullshit like “there’s no recorded instance of trump ever having used the word enigma” and “there’s 1 r in strawberry”.
LLMs are a copy paste machine, not a rationalization engine of any sort (at least as far as all the slop that we get shoved in our face, I don’t include the specialized protein folding and reconstructive models that were purpose built for very niche applications)
-
This post did not contain any content.
Why is a researcher with a PhD in social sciences researching the accuracy confidence of predictive text, how has this person gotten to where they are without being able to understand that LLMs don't think? Surely that came up when he started even considering this brainfart of a research project?
-
That's because they aren't "aware" of anything.
This Nobel Prize winner and subject matter expert takes the opposite view
-
This post did not contain any content.
However, when the participants and LLMs were asked retroactively how well they thought they did, only the humans appeared able to adjust expectations
This is what everyone with a fucking clue has been saying for the past 5, 6? years these stupid fucking chatbots have been around.
-
Why is a researcher with a PhD in social sciences researching the accuracy confidence of predictive text, how has this person gotten to where they are without being able to understand that LLMs don't think? Surely that came up when he started even considering this brainfart of a research project?
Someone has to prove it wrong before it's actually wrong. Maybe they set out to discredit the bots
-
Someone has to prove it wrong before it's actually wrong. Maybe they set out to discredit the bots
I guess, but it's like proving your phones predictive text has confidence in its suggestions regardless of accuracy. Confidence is not an attribute of a math function, they are attributing intelligence to a predictive model.
-
Not even a good use case either, especially when it spews such bullshit like “there’s no recorded instance of trump ever having used the word enigma” and “there’s 1 r in strawberry”.
LLMs are a copy paste machine, not a rationalization engine of any sort (at least as far as all the slop that we get shoved in our face, I don’t include the specialized protein folding and reconstructive models that were purpose built for very niche applications)
they're solid starting point for shopping now that wirecutter, slant, and others are enshittified. i hate it and it makes me feel dirty to use, and you can't just do whatever the llm says. but asking it for a list of options to then explore is currently the best way i've found to jump into things like outdoor basketball shoe options
-
This post did not contain any content.
Sounds pretty human to me. /s
-
This post did not contain any content.
AI evolved their own form of the Dunning Kruger effect.
-
This post did not contain any content.
It's easy, just ask the AI "are you sure"? Until it stops changing it's answer.
But seriously, LLMs are just advanced autocomplete.
-
This post did not contain any content.
Large language models aren’t designed to be knowledge machines - they’re designed to generate natural-sounding language, nothing more. The fact that they ever get things right is just a byproduct of their training data containing a lot of correct information. These systems aren’t generally intelligent, and people need to stop treating them as if they are. Complaining that an LLM gives out wrong information isn’t a failure of the model itself - it’s a mismatch of expectations.
-
This post did not contain any content.
Confidently incorrect.
-
This Nobel Prize winner and subject matter expert takes the opposite view
People really do not like seeing opposing viewpoints, eh? There's disagreeing, and then there's downvoting to oblivion without even engaging in a discussion, haha.
Even if they're probably right, in such murky uncertain waters where we're not experts, one should have at least a little open mind, or live and let live.
-
Large language models aren’t designed to be knowledge machines - they’re designed to generate natural-sounding language, nothing more. The fact that they ever get things right is just a byproduct of their training data containing a lot of correct information. These systems aren’t generally intelligent, and people need to stop treating them as if they are. Complaining that an LLM gives out wrong information isn’t a failure of the model itself - it’s a mismatch of expectations.
Neither are our brains.
“Brains are survival engines, not truth detectors. If self-deception promotes fitness, the brain lies. Stops noticing—irrelevant things. Truth never matters. Only fitness. By now you don’t experience the world as it exists at all. You experience a simulation built from assumptions. Shortcuts. Lies. Whole species is agnosiac by default.”
― Peter Watts, Blindsight (fiction)
Starting to think we're really not much smarter. "But LLMs tell us what we want to hear!" Been on FaceBook lately, or lemmy?
If nothing else, LLMs have woke me to how stupid humans are vs. the machines.
-
Sounds pretty human to me. /s
Sounds pretty human to me. no /s
-
I guess, but it's like proving your phones predictive text has confidence in its suggestions regardless of accuracy. Confidence is not an attribute of a math function, they are attributing intelligence to a predictive model.
I work in risk management, but don't really have a strong understanding of LLM mechanics. "Confidence" is something that i quantify in my work, but it has different terms that are associated with it. In modeling outcomes, I may say that we have 60% confidence in achieving our budget objectives, while others would express the same result by saying our chances of achieving our budget objective are 60%. Again, I'm not sure if this is what the LLM is doing, but if it is producing a modeled prediction with a CDF of possible outcomes, then representing its result with 100% confindence means that the LLM didn't model any other possible outcomes other than the answer it is providing, which does seem troubling.
-
People really do not like seeing opposing viewpoints, eh? There's disagreeing, and then there's downvoting to oblivion without even engaging in a discussion, haha.
Even if they're probably right, in such murky uncertain waters where we're not experts, one should have at least a little open mind, or live and let live.
It's like talking with someone who thinks the Earth is flat. There isn't anything to discuss. They're objectively wrong.
Humans like to anthropomorphize everything. It's why you can see a face on a car's front grille. LLMs are ultra advanced pattern matching algorithms. They do not think or reason or have any kind of opinion or sentience, yet they are being utilized as if they do. Let's see how it works out for the world, I guess.
-
It's easy, just ask the AI "are you sure"? Until it stops changing it's answer.
But seriously, LLMs are just advanced autocomplete.
They can even get math wrong. Which surprised me. Had to tell it the answer is wrong for them to recalculate and then get the correct answer. It was simple percentages of a list of numbers I had asked.
-
Neither are our brains.
“Brains are survival engines, not truth detectors. If self-deception promotes fitness, the brain lies. Stops noticing—irrelevant things. Truth never matters. Only fitness. By now you don’t experience the world as it exists at all. You experience a simulation built from assumptions. Shortcuts. Lies. Whole species is agnosiac by default.”
― Peter Watts, Blindsight (fiction)
Starting to think we're really not much smarter. "But LLMs tell us what we want to hear!" Been on FaceBook lately, or lemmy?
If nothing else, LLMs have woke me to how stupid humans are vs. the machines.
There are plenty of similarities in the output of both the human brain and LLMs, but overall they’re very different. Unlike LLMs, the human brain is generally intelligent - it can adapt to a huge variety of cognitive tasks. LLMs, on the other hand, can only do one thing: generate language. It’s tempting to anthropomorphize systems like ChatGPT because of how competent they seem, but there’s no actual thinking going on. It’s just generating language based on patterns and probabilities.
-
I work in risk management, but don't really have a strong understanding of LLM mechanics. "Confidence" is something that i quantify in my work, but it has different terms that are associated with it. In modeling outcomes, I may say that we have 60% confidence in achieving our budget objectives, while others would express the same result by saying our chances of achieving our budget objective are 60%. Again, I'm not sure if this is what the LLM is doing, but if it is producing a modeled prediction with a CDF of possible outcomes, then representing its result with 100% confindence means that the LLM didn't model any other possible outcomes other than the answer it is providing, which does seem troubling.
Nah so their definition is the classical "how confident are you that you got the answer right". If you read the article they asked a bunch of people and 4 LLMs a bunch of random questions, then asked the respondent whether they/it had confidence their answer was correct, and then checked the answer. The LLMs initially lined up with people (over confident) but then when they iterated, shared results and asked further questions the LLMs confidence increased while people's tends to decrease to mitigate the over confidence.
But the study still assumes intelligence enough to review past results and adjust accordingly, but disregards the fact that an AI isnt intelligence, it's a word prediction model based on a data set of written text tending to infinity. It's not assessing validity of results, it's predicting what the answer is based on all previous inputs. The whole study is irrelevant.
