AI Chatbots Remain Overconfident — Even When They’re Wrong: Large Language Models appear to be unaware of their own mistakes, prompting concerns about common uses for AI chatbots.
Technology
68
Beiträge
42
Kommentatoren
969
Aufrufe
-
This post did not contain any content.
AI Chatbots Remain Overconfident — Even When They’re Wrong - Dietrich College of Humanities and Social Sciences - Carnegie Mellon University
Large Language Models appear to be unaware of their own mistakes, prompting concerns about common uses for AI chatbots.
(www.cmu.edu)
-
This post did not contain any content.
AI Chatbots Remain Overconfident — Even When They’re Wrong - Dietrich College of Humanities and Social Sciences - Carnegie Mellon University
Large Language Models appear to be unaware of their own mistakes, prompting concerns about common uses for AI chatbots.
(www.cmu.edu)
That's because they aren't "aware" of anything.
-
This post did not contain any content.
AI Chatbots Remain Overconfident — Even When They’re Wrong - Dietrich College of Humanities and Social Sciences - Carnegie Mellon University
Large Language Models appear to be unaware of their own mistakes, prompting concerns about common uses for AI chatbots.
(www.cmu.edu)
I’m pretty much done with them except for some search
-
I’m pretty much done with them except for some search
Not even a good use case either, especially when it spews such bullshit like “there’s no recorded instance of trump ever having used the word enigma” and “there’s 1 r in strawberry”.
LLMs are a copy paste machine, not a rationalization engine of any sort (at least as far as all the slop that we get shoved in our face, I don’t include the specialized protein folding and reconstructive models that were purpose built for very niche applications)
-
This post did not contain any content.
AI Chatbots Remain Overconfident — Even When They’re Wrong - Dietrich College of Humanities and Social Sciences - Carnegie Mellon University
Large Language Models appear to be unaware of their own mistakes, prompting concerns about common uses for AI chatbots.
(www.cmu.edu)
Why is a researcher with a PhD in social sciences researching the accuracy confidence of predictive text, how has this person gotten to where they are without being able to understand that LLMs don't think? Surely that came up when he started even considering this brainfart of a research project?
-
That's because they aren't "aware" of anything.
This Nobel Prize winner and subject matter expert takes the opposite view
-
This post did not contain any content.
AI Chatbots Remain Overconfident — Even When They’re Wrong - Dietrich College of Humanities and Social Sciences - Carnegie Mellon University
Large Language Models appear to be unaware of their own mistakes, prompting concerns about common uses for AI chatbots.
(www.cmu.edu)
However, when the participants and LLMs were asked retroactively how well they thought they did, only the humans appeared able to adjust expectations
This is what everyone with a fucking clue has been saying for the past 5, 6? years these stupid fucking chatbots have been around.
-
Why is a researcher with a PhD in social sciences researching the accuracy confidence of predictive text, how has this person gotten to where they are without being able to understand that LLMs don't think? Surely that came up when he started even considering this brainfart of a research project?
Someone has to prove it wrong before it's actually wrong. Maybe they set out to discredit the bots
-
Someone has to prove it wrong before it's actually wrong. Maybe they set out to discredit the bots
I guess, but it's like proving your phones predictive text has confidence in its suggestions regardless of accuracy. Confidence is not an attribute of a math function, they are attributing intelligence to a predictive model.
-
Not even a good use case either, especially when it spews such bullshit like “there’s no recorded instance of trump ever having used the word enigma” and “there’s 1 r in strawberry”.
LLMs are a copy paste machine, not a rationalization engine of any sort (at least as far as all the slop that we get shoved in our face, I don’t include the specialized protein folding and reconstructive models that were purpose built for very niche applications)
they're solid starting point for shopping now that wirecutter, slant, and others are enshittified. i hate it and it makes me feel dirty to use, and you can't just do whatever the llm says. but asking it for a list of options to then explore is currently the best way i've found to jump into things like outdoor basketball shoe options
-
This post did not contain any content.
AI Chatbots Remain Overconfident — Even When They’re Wrong - Dietrich College of Humanities and Social Sciences - Carnegie Mellon University
Large Language Models appear to be unaware of their own mistakes, prompting concerns about common uses for AI chatbots.
(www.cmu.edu)
Sounds pretty human to me. /s
-
This post did not contain any content.
AI Chatbots Remain Overconfident — Even When They’re Wrong - Dietrich College of Humanities and Social Sciences - Carnegie Mellon University
Large Language Models appear to be unaware of their own mistakes, prompting concerns about common uses for AI chatbots.
(www.cmu.edu)
AI evolved their own form of the Dunning Kruger effect.
-
This post did not contain any content.
AI Chatbots Remain Overconfident — Even When They’re Wrong - Dietrich College of Humanities and Social Sciences - Carnegie Mellon University
Large Language Models appear to be unaware of their own mistakes, prompting concerns about common uses for AI chatbots.
(www.cmu.edu)
It's easy, just ask the AI "are you sure"? Until it stops changing it's answer.
But seriously, LLMs are just advanced autocomplete.
-
This post did not contain any content.
AI Chatbots Remain Overconfident — Even When They’re Wrong - Dietrich College of Humanities and Social Sciences - Carnegie Mellon University
Large Language Models appear to be unaware of their own mistakes, prompting concerns about common uses for AI chatbots.
(www.cmu.edu)
Large language models aren’t designed to be knowledge machines - they’re designed to generate natural-sounding language, nothing more. The fact that they ever get things right is just a byproduct of their training data containing a lot of correct information. These systems aren’t generally intelligent, and people need to stop treating them as if they are. Complaining that an LLM gives out wrong information isn’t a failure of the model itself - it’s a mismatch of expectations.
-
This post did not contain any content.
AI Chatbots Remain Overconfident — Even When They’re Wrong - Dietrich College of Humanities and Social Sciences - Carnegie Mellon University
Large Language Models appear to be unaware of their own mistakes, prompting concerns about common uses for AI chatbots.
(www.cmu.edu)
Confidently incorrect.
-
This Nobel Prize winner and subject matter expert takes the opposite view
People really do not like seeing opposing viewpoints, eh? There's disagreeing, and then there's downvoting to oblivion without even engaging in a discussion, haha.
Even if they're probably right, in such murky uncertain waters where we're not experts, one should have at least a little open mind, or live and let live.
-
Large language models aren’t designed to be knowledge machines - they’re designed to generate natural-sounding language, nothing more. The fact that they ever get things right is just a byproduct of their training data containing a lot of correct information. These systems aren’t generally intelligent, and people need to stop treating them as if they are. Complaining that an LLM gives out wrong information isn’t a failure of the model itself - it’s a mismatch of expectations.
Neither are our brains.
“Brains are survival engines, not truth detectors. If self-deception promotes fitness, the brain lies. Stops noticing—irrelevant things. Truth never matters. Only fitness. By now you don’t experience the world as it exists at all. You experience a simulation built from assumptions. Shortcuts. Lies. Whole species is agnosiac by default.”
― Peter Watts, Blindsight (fiction)
Starting to think we're really not much smarter. "But LLMs tell us what we want to hear!" Been on FaceBook lately, or lemmy?
If nothing else, LLMs have woke me to how stupid humans are vs. the machines.
-
Sounds pretty human to me. /s
Sounds pretty human to me. no /s
-
I guess, but it's like proving your phones predictive text has confidence in its suggestions regardless of accuracy. Confidence is not an attribute of a math function, they are attributing intelligence to a predictive model.
I work in risk management, but don't really have a strong understanding of LLM mechanics. "Confidence" is something that i quantify in my work, but it has different terms that are associated with it. In modeling outcomes, I may say that we have 60% confidence in achieving our budget objectives, while others would express the same result by saying our chances of achieving our budget objective are 60%. Again, I'm not sure if this is what the LLM is doing, but if it is producing a modeled prediction with a CDF of possible outcomes, then representing its result with 100% confindence means that the LLM didn't model any other possible outcomes other than the answer it is providing, which does seem troubling.
-
People really do not like seeing opposing viewpoints, eh? There's disagreeing, and then there's downvoting to oblivion without even engaging in a discussion, haha.
Even if they're probably right, in such murky uncertain waters where we're not experts, one should have at least a little open mind, or live and let live.
It's like talking with someone who thinks the Earth is flat. There isn't anything to discuss. They're objectively wrong.
Humans like to anthropomorphize everything. It's why you can see a face on a car's front grille. LLMs are ultra advanced pattern matching algorithms. They do not think or reason or have any kind of opinion or sentience, yet they are being utilized as if they do. Let's see how it works out for the world, I guess.
