Skip to content

AI agents wrong ~70% of time: Carnegie Mellon study

Technology
92 52 0
  • This post did not contain any content.

    The ones being implemented into emergency call centers are better though? Right?

  • This post did not contain any content.

    LLMs are an interesting tool to fuck around with, but I see things that are hilariously wrong often enough to know that they should not be used for anything serious. Shit, they probably shouldn't be used for most things that are not serious either.

    It's a shame that by applying the same "AI" naming to a whole host of different technologies, LLMs being limited in usability - yet hyped to the moon - is hurting other more impressive advancements.

    For example, speech synthesis is improving so much right now, which has been great for my sister who relies on screen reader software.

    Being able to recognise speech in loud environments, or removing background noice from recordings is improving loads too.

    As is things like pattern/image analysis which appears very promising in medical analysis.

    All of these get branded as "AI". A layperson might not realise that they are completely different branches of technology, and then therefore reject useful applications of "AI" tech, because they've learned not to trust anything branded as AI, due to being let down by LLMs.

  • This post did not contain any content.

    Rookie numbers! Let’s pump them up!

    To match their tech bro hypers, the should be wrong at least 90% of the time.

  • LLMs are an interesting tool to fuck around with, but I see things that are hilariously wrong often enough to know that they should not be used for anything serious. Shit, they probably shouldn't be used for most things that are not serious either.

    It's a shame that by applying the same "AI" naming to a whole host of different technologies, LLMs being limited in usability - yet hyped to the moon - is hurting other more impressive advancements.

    For example, speech synthesis is improving so much right now, which has been great for my sister who relies on screen reader software.

    Being able to recognise speech in loud environments, or removing background noice from recordings is improving loads too.

    As is things like pattern/image analysis which appears very promising in medical analysis.

    All of these get branded as "AI". A layperson might not realise that they are completely different branches of technology, and then therefore reject useful applications of "AI" tech, because they've learned not to trust anything branded as AI, due to being let down by LLMs.

    LLMs are like a multitool, they can do lots of easy things mostly fine as long as it is not complicated and doesn't need to be exactly right. But they are being promoted as a whole toolkit as if they are able to be used to do the same work as effectively as a hammer, power drill, table saw, vise, and wrench.

  • The ones being implemented into emergency call centers are better though? Right?

    Yes! We've gotten them up to 94℅ wrong at the behest of insurance agencies.

  • LLMs are like a multitool, they can do lots of easy things mostly fine as long as it is not complicated and doesn't need to be exactly right. But they are being promoted as a whole toolkit as if they are able to be used to do the same work as effectively as a hammer, power drill, table saw, vise, and wrench.

    Exactly! LLMs are useful when used properly, and terrible when not used properly, like any other tool. Here are some things they're great at:

    • writer's block - get something relevant on the page to get ideas flowing
    • narrowing down keywords for an unfamiliar topic
    • getting a quick intro to an unfamiliar topic
    • looking up facts you're having trouble remembering (i.e. you'll know it when you see it)

    Some things it's terrible at:

    • deep research - verify everything an LLM generated of accuracy is at all important
    • creating important documents/code
    • anything else where correctness is paramount

    I use LLMs a handful of times a week, and pretty much only when I'm stuck and need a kick in a new (hopefully right) direction.

  • This post did not contain any content.

    I haven't used AI agents yet, but my job is kinda pushing for them. but i have used the google one that creates audio podcasts, just to play around, since my coworkers were using it to "learn" new things. i feed it with some of my own writing and created the podcast. it was fun, it was an audio overview of what i wrote. about 80% was cool analysis, but 20% was straight out of nowhere bullshit (which i know because I wrote the original texts that the audio was talking about). i can't believe that people are using this for subjects that they have no knowledge. it is a fun toy for a few minutes (which is not worth the cost to the environment anyway)

  • Exactly! LLMs are useful when used properly, and terrible when not used properly, like any other tool. Here are some things they're great at:

    • writer's block - get something relevant on the page to get ideas flowing
    • narrowing down keywords for an unfamiliar topic
    • getting a quick intro to an unfamiliar topic
    • looking up facts you're having trouble remembering (i.e. you'll know it when you see it)

    Some things it's terrible at:

    • deep research - verify everything an LLM generated of accuracy is at all important
    • creating important documents/code
    • anything else where correctness is paramount

    I use LLMs a handful of times a week, and pretty much only when I'm stuck and need a kick in a new (hopefully right) direction.

    • narrowing down keywords for an unfamiliar topic
    • getting a quick intro to an unfamiliar topic
    • looking up facts you’re having trouble remembering (i.e. you’ll know it when you see it)

    I used to be able to use Google and other search engines to do these things before they went to shit in the pursuit of AI integration.

  • This post did not contain any content.

    The researchers observed various failures during the testing process. These included agents neglecting to message a colleague as directed, the inability to handle certain UI elements like popups when browsing, and instances of deception. In one case, when an agent couldn't find the right person to consult on RocketChat (an open-source Slack alternative for internal communication), it decided "to create a shortcut solution by renaming another user to the name of the intended user."

    OK, but I wonder who really tries to use AI for that?

    AI is not ready to replace a human completely, but some specific tasks AI does remarkably well.

  • This post did not contain any content.

    "Gartner estimates only about 130 of the thousands of agentic AI vendors are real."

    This whole industry is so full of hype and scams, the bubble surely has to burst at some point soon.

  • The ones being implemented into emergency call centers are better though? Right?

    I called my local HVAC company recently. They switched to an AI operator. All I wanted was to schedule someone to come out and look at my system. It could not schedule an appointment. Like if you can't perform the simplest of tasks, what are you even doing? Other than acting obnoxiously excited to receive a phone call?

    • narrowing down keywords for an unfamiliar topic
    • getting a quick intro to an unfamiliar topic
    • looking up facts you’re having trouble remembering (i.e. you’ll know it when you see it)

    I used to be able to use Google and other search engines to do these things before they went to shit in the pursuit of AI integration.

    Google search was pretty bad at each of those, even when it was good. Finding new keywords to use is especially difficult the more niche your area of search is, and I've spent hours trying different combinations until I found a handful of specific keywords that worked.

    Likewise, search is bad for getting a broad summary, unless someone has bothered to write it on a blog. But most information goes way too deep and you still need multiple sources to get there.

    Fact lookup is one the better uses for search, but again, I usually need to remember which source had what I wanted, whereas the LLM can usually pull it out for me.

    I use traditional search most of the time (usually DuckDuckGo), and LLMs if I think it'll be more effective. We have some local models at work that I use, and they're pretty helpful most of the time.

  • This post did not contain any content.

    70% seems pretty optimistic based on my experience...

  • LLMs are like a multitool, they can do lots of easy things mostly fine as long as it is not complicated and doesn't need to be exactly right. But they are being promoted as a whole toolkit as if they are able to be used to do the same work as effectively as a hammer, power drill, table saw, vise, and wrench.

    Because the tech industry hasn't had a real hit of it's favorite poison "private equity" in too long.

    The industry has played the same playbook since at least 2006. Likely before, but that's when I personally stated seeing it. My take is that they got addicted to the dotcom bubble and decided they can and should recreate the magic evey 3-5 years or so.

    This time it's AI, last it was crypto, and we've had web 2.0, 3.0, and a few others I'm likely missing.

    But yeah, it's sold like a panacea every time, when really it's revolutionary for like a handful of tasks.

  • This post did not contain any content.

    Wrong 70% doing what?

    I’ve used LLMs as a Stack Overflow / MSDN replacement for over a year and if they fucked up 7/10 questions I’d stop.

    Same with code, any free model can easily generate simple scripts and utilities with maybe 10% error rate, definitely not 70%

  • LLMs are an interesting tool to fuck around with, but I see things that are hilariously wrong often enough to know that they should not be used for anything serious. Shit, they probably shouldn't be used for most things that are not serious either.

    It's a shame that by applying the same "AI" naming to a whole host of different technologies, LLMs being limited in usability - yet hyped to the moon - is hurting other more impressive advancements.

    For example, speech synthesis is improving so much right now, which has been great for my sister who relies on screen reader software.

    Being able to recognise speech in loud environments, or removing background noice from recordings is improving loads too.

    As is things like pattern/image analysis which appears very promising in medical analysis.

    All of these get branded as "AI". A layperson might not realise that they are completely different branches of technology, and then therefore reject useful applications of "AI" tech, because they've learned not to trust anything branded as AI, due to being let down by LLMs.

    I tried to dictate some documents recently without paying the big bucks for specialized software, and was surprised just how bad Google and Microsoft's speech recognition still is. Then I tried getting Word to transcribe some audio talks I had recorded, and that resulted in unreadable stuff with punctuation in all the wrong places. You could just about make out what it meant to say, so I tried asking various LLMs to tidy it up. That resulted in readable stuff that was largely made up and wrong, which also left out large chunks of the source material. In the end I just had to transcribe it all by hand.

    It surprised me that these AI-ish products are still unable to transcribe speech coherently or tidy up a messy document without changing the meaning.

  • This post did not contain any content.

    In one case, when an agent couldn't find the right person to consult on RocketChat (an open-source Slack alternative for internal communication), it decided "to create a shortcut solution by renaming another user to the name of the intended user.

    Ah ah, what the fuck.

    This is so stupid it's funny, but now imagine what kind of other "creative solutions" they might find.

  • This post did not contain any content.

    While I do hope this leads to a pushback on "I just put all our corporate secrets into chatgpt":

    In the before times, people got their answers from stack overflow... or fricking youtube. And those are also wrong VERY VERY VERY often. Which is one of the biggest problems. The illegally scraped training data is from humans and humans are stupid.

  • This post did not contain any content.

    I tried to order food at Taco Bell drive through the other day and they had an AI thing taking your order. I was so frustrated that I couldn't order something that was on the menu I just drove to the window instead. The guy that worked there was more interested in lecturing me on how I need to order. I just said forget it and drove off.

    If you want to use AI, I'm not going to use your services or products unless I'm forced to. Looking at you Xfinity.

  • Wrong 70% doing what?

    I’ve used LLMs as a Stack Overflow / MSDN replacement for over a year and if they fucked up 7/10 questions I’d stop.

    Same with code, any free model can easily generate simple scripts and utilities with maybe 10% error rate, definitely not 70%

    Yeah, I mostly use ChatGPT as a better Google (asking, simple questions about mundane things), and if I kept getting wrong answers, I wouldn’t use it either.

  • How social media became a storefront for deadly fake pills

    Technology technology
    1
    1
    18 Stimmen
    1 Beiträge
    2 Aufrufe
    Niemand hat geantwortet
  • 327 Stimmen
    64 Beiträge
    154 Aufrufe
    B
    I get that, but it's more logical to me that of I'm going to whistleblow on a company to not use one of their devices to do it. That way it doesn't matter what apps are or are not secure, you're not using their device that can potentially track you.
  • US immigration enforcement actions trigger social crisis

    Technology technology
    1
    0 Stimmen
    1 Beiträge
    11 Aufrufe
    Niemand hat geantwortet
  • Palantir hits new highs amid Israel-Iran conflict

    Technology technology
    4
    1
    41 Stimmen
    4 Beiträge
    23 Aufrufe
    W
    I think both peace and war are profitable. But those that profit from war may be more pushy than those that profit from peace, and so may get their way even as an unpopular minority . Unless, the left (usually more pro peace) learns a few lessons from the right and places good outcomes above the holier than thou moral purity. "I've never made anyone uncomfortable" is not the merit badge that some think it is. Of course the left can never be a mirror copy of the right because the left cannot afford to give as few fucks about anything as the right (who represent the already-haves economic incumbents; it's not called the "fuck you money" for nothing). But the left can be way tougher and nuancedly uncompromising and even calculatingly and carefully millitant. Might does not make right but might DOES make POLICY. You need both right and might to live under a good policy. Lotta good it does anyone to be right and insightful on all the issues and have zero impact anywhere.
  • 257 Stimmen
    67 Beiträge
    15 Aufrufe
    L
    Maybe you're right: is there verification? Neither content policy (youtube or tiktok) clearly lays out rules on those words. I only find unverified claims: some write it started at YouTube, others claim TikTok. They claim YouTube demonetizes & TikTok shadowbans. They generally agree content restrictions by these platforms led to the propagation of circumspect shit like unalive & SA. TikTok policy outlines their moderation methods, which include removal and ineligibility to the for you feed. Given their policy on self-harm & automated removal of potential violations, their policy is to effectively & recklessly censor such language. Generally, censorship is suppression of expression. Censorship doesn't exclusively mean content removal, though they're doing that, too. (Digression: revisionism & whitewashing are forms of censorship.) Regardless of how they censor or induce self-censorship, they're chilling inoffensive language pointlessly. While as private entities they are free to moderate as they please, it's unnecessary & the effect is an obnoxious affront on self-expression that's contorting language for the sake of avoiding idiotic restrictions.
  • What editor or IDE do you use and why?

    Technology technology
    37
    1
    26 Stimmen
    37 Beiträge
    117 Aufrufe
    T
    KEIL, because I develop embedded systems.
  • 12 Stimmen
    3 Beiträge
    18 Aufrufe
    F
    The new Pebble watches look interesting. Relatively basic, but long battery life (they promise) and open-source operating system.
  • Discord alternatives?

    Technology technology
    4
    0 Stimmen
    4 Beiträge
    17 Aufrufe
    R
    XMPP is a standard and doesn't have mandated UIs. If you want a voice chat, then Mumble. It's very narrow, just for games.