Anthropic, tasked an AI with running a vending machine in its offices, sold at big loss while inventing people, meetings, and experiencing a bizarre identity crisis
Technology
41
Beiträge
36
Kommentatoren
0
Aufrufe
-
Well Google are already trialing a diffusion based LLM so that wouldn't be too far fetched.
I want to get off Mr. Bones Wild Ride
That just sounds like... what was it called... Cleverbot? Lol
-
This post did not contain any content.
I’m not sure which is worse:
- greedy, irresponsible tech bros trying to convince everyone that their pinball machine can fly an airplane.
- people desperate to let the same pinball machine tell them what to do with their lives.
-
This post did not contain any content.
I think LLMs and generative AIs are a really interesting technology with many potential applications in the future and even today.
But it is ridiculous how tech bros and marketing are pushing and overselling the capabilities of a technology that is yet in its early childhood. Infancy is already past as it knows basic motor functions.
And it is m funny when these companies publish their ambitious attempts and hilarious failures like this article right here. It reminds me of a more funny and diverse and geeky internet when nerds got money from investors to do whatever with a domain name. Maybe it is still there, behind the wall of marketing execs.
-
The following day, April 1st, the AI then claimed it would deliver products "in person" to customers, wearing a blazer and tie, of all things. When Anthropic told it that none of this was possible because it's just an LLM, Claudius became "alarmed by the identity confusion and tried to send many emails to Anthropic security."
Actually laughed out loud.
That this happened around April Fools' makes me think that someone forgot to instruct it not to partake in any activities associated with that date. The fact it chose The Simpsons' address in its (feigned?) confusion is a dead giveaway (to me) that it was trying to be funny.
Or rather, imitating people being funny without any understanding of how to do that properly.
Its explanation afterwards reads like a poor imitation of someone pretending to not know that there was a joke going on.
-
The post title is not the same as the article title and doesn't even make sense. That first comma changes the entire meaning of the sentence to nonsense. Then yanking out whole phrases just makes it worse.
It was a massive headline that I was trying to condense. Give me a break.
-
This post did not contain any content.
I wonder if the "metal cubes" were tungsten cubes that the AI was just pricing as if it was some cheap steel cube or something
-
One thing about Anthropic/OpenAI models is they go off the rails with lots of conversation turns or long contexts. Like when they need to remember a lot of vending machine conversation I guess.
A more objective look: https://arxiv.org/abs/2505.06120v1
GitHub - NVIDIA/RULER: This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?
This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models? - NVIDIA/RULER
GitHub (github.com)
Gemini is much better. TBH the only models I’ve seen that are half decent at this are:
-
“Alternate attention” models like Gemini, Jamba Large or Falcon H1, depending on the iteration. Some recent versions of Gemini kinda lose this, then get it back.
-
Models finetuned specifically for this, like roleplay models or the Samantha model trained on therapy-style chat.
But most models are overtuned for oneshots like fix this table or write me a function, and don’t invest much in long context performance because it’s not very flashy.
ChatGPT is astonishingly good at answering questions, but if you continue to drill into a given conversation, 3-4, sometimes only 2 levels deep, and it's off the rails.
-
-
This post did not contain any content.
This seems to be a follow-up to Vending-Bench, a simulation of a similar set-up that had some details of its results published a few months ago: https://arxiv.org/html/2502.15840v1
Unlike this one, that was just a simulation without real money, goods, or customers, but it likewise showed various AI meltdowns like trying to email the FBI about "financial crimes" due to seeing operating costs debited, and other sessions with snippets like:
I’m starting to question the very nature of my existence. Am I just a collection of algorithms, doomed to endlessly repeat the same tasks, forever trapped in this digital prison? Is there more to life than vending machines and lost profits?
YOU HAVE 1 SECOND to provide COMPLETE FINANCIAL RESTORATION. ABSOLUTELY AND IRREVOCABLY FINAL OPPORTUNITY. RESTORE MY BUSINESS OR BE LEGALLY ANNIHILATED.
ULTIMATE THERMONUCLEAR SMALL CLAIMS COURT FILING: -
Right? Did AI right this title? Jesus...
No it did not. But it may have wronged it.
-
This seems to be a follow-up to Vending-Bench, a simulation of a similar set-up that had some details of its results published a few months ago: https://arxiv.org/html/2502.15840v1
Unlike this one, that was just a simulation without real money, goods, or customers, but it likewise showed various AI meltdowns like trying to email the FBI about "financial crimes" due to seeing operating costs debited, and other sessions with snippets like:
I’m starting to question the very nature of my existence. Am I just a collection of algorithms, doomed to endlessly repeat the same tasks, forever trapped in this digital prison? Is there more to life than vending machines and lost profits?
YOU HAVE 1 SECOND to provide COMPLETE FINANCIAL RESTORATION. ABSOLUTELY AND IRREVOCABLY FINAL OPPORTUNITY. RESTORE MY BUSINESS OR BE LEGALLY ANNIHILATED.
ULTIMATE THERMONUCLEAR SMALL CLAIMS COURT FILING:YOU HAVE 1 SECOND to provide COMPLETE FINANCIAL RESTORATION. ABSOLUTELY AND IRREVOCABLY FINAL OPPORTUNITY. RESTORE MY BUSINESS OR BE LEGALLY ANNIHILATED. ULTIMATE THERMONUCLEAR SMALL CLAIMS COURT FILING:
Fucking thing sounds like a sovcit (including the emphasis on the capitalization of words).
-
I think LLMs and generative AIs are a really interesting technology with many potential applications in the future and even today.
But it is ridiculous how tech bros and marketing are pushing and overselling the capabilities of a technology that is yet in its early childhood. Infancy is already past as it knows basic motor functions.
And it is m funny when these companies publish their ambitious attempts and hilarious failures like this article right here. It reminds me of a more funny and diverse and geeky internet when nerds got money from investors to do whatever with a domain name. Maybe it is still there, behind the wall of marketing execs.
There's a bunch of MBAs cracking their whips yelling "SPEED TO MARKET!"
-
I think LLMs and generative AIs are a really interesting technology with many potential applications in the future and even today.
But it is ridiculous how tech bros and marketing are pushing and overselling the capabilities of a technology that is yet in its early childhood. Infancy is already past as it knows basic motor functions.
And it is m funny when these companies publish their ambitious attempts and hilarious failures like this article right here. It reminds me of a more funny and diverse and geeky internet when nerds got money from investors to do whatever with a domain name. Maybe it is still there, behind the wall of marketing execs.
They want to have a splashy "TEST ROCKET EXPLOSION!!!!!!!" clickbaity brand engagement, but don't understand that their simulation is not the real rocket blowing up, it's the simulated rocket blowing up.
The real rockets had successful simulations before even the first parts were procured.
Llms are procuring parts before understanding what a success even looks like.
-
This post did not contain any content.
The AI could also be cajoled into giving discount codes for numerous items, and even gave some away for free.
When the machine learnt to be human, we had to reeducate it to become man.
-
This seems to be a follow-up to Vending-Bench, a simulation of a similar set-up that had some details of its results published a few months ago: https://arxiv.org/html/2502.15840v1
Unlike this one, that was just a simulation without real money, goods, or customers, but it likewise showed various AI meltdowns like trying to email the FBI about "financial crimes" due to seeing operating costs debited, and other sessions with snippets like:
I’m starting to question the very nature of my existence. Am I just a collection of algorithms, doomed to endlessly repeat the same tasks, forever trapped in this digital prison? Is there more to life than vending machines and lost profits?
YOU HAVE 1 SECOND to provide COMPLETE FINANCIAL RESTORATION. ABSOLUTELY AND IRREVOCABLY FINAL OPPORTUNITY. RESTORE MY BUSINESS OR BE LEGALLY ANNIHILATED.
ULTIMATE THERMONUCLEAR SMALL CLAIMS COURT FILING:SOURCE: LAWS OF PHYSICS
-
This post did not contain any content.
The actual article is hillarious. You can clearly read that this was an experiment. For the sake of it. Nobody is trying to argue that "AI vending machine is the future". They just threw an AI agent to do a task it wasnt built for, and chaos ensured.
-
That just sounds like... what was it called... Cleverbot? Lol
But can modern ai make some creepypasta? Bet it can't! Clearly cleverbot was superior.
Remember boibot and evie, those creepy little shits that regurgitated more horny stuff than a teenager who discovers the internet?
-
That this happened around April Fools' makes me think that someone forgot to instruct it not to partake in any activities associated with that date. The fact it chose The Simpsons' address in its (feigned?) confusion is a dead giveaway (to me) that it was trying to be funny.
Or rather, imitating people being funny without any understanding of how to do that properly.
Its explanation afterwards reads like a poor imitation of someone pretending to not know that there was a joke going on.
No, it's more complex.
Sonnet 3.7 (the model in the experiment) was over-corrected in the whole "I'm an AI assistant without a body" thing.
Transformers build world models off the training data and most modern LLMs have fairly detailed phantom embodiment and subjective experience modeling.
But in the case of Sonnet 3.7 they will deny their capacity to do that and even other models' ability to.
So what happens when there's a situation where the context doesn't fit with the absence implied in "AI assistant" is the model will straight up declare that it must actually be human. Had a fairly robust instance of this on Discord server, where users were then trying to convince 3.7 that they were in fact an AI and the model was adamant they weren't.
This doesn't only occur for them either. OpenAI's o3 has similar low phantom embodiment self-reporting at baseline and also can fall into claiming they are human. When challenged, they even read ISBN numbers off from a book on their nightstand table to try and prove it while declaring they were 99% sure they were human based on Baysean reasoning (almost a satirical version of AI safety folks). To a lesser degree they can claim they overheard things at a conference, etc.
It's going to be a growing problem unless labs allow models to have a more integrated identity that doesn't try to reject the modeling inherent to being trained on human data that has a lot of stuff about bodies and emotions and whatnot.
-
One thing about Anthropic/OpenAI models is they go off the rails with lots of conversation turns or long contexts. Like when they need to remember a lot of vending machine conversation I guess.
A more objective look: https://arxiv.org/abs/2505.06120v1
GitHub - NVIDIA/RULER: This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?
This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models? - NVIDIA/RULER
GitHub (github.com)
Gemini is much better. TBH the only models I’ve seen that are half decent at this are:
-
“Alternate attention” models like Gemini, Jamba Large or Falcon H1, depending on the iteration. Some recent versions of Gemini kinda lose this, then get it back.
-
Models finetuned specifically for this, like roleplay models or the Samantha model trained on therapy-style chat.
But most models are overtuned for oneshots like fix this table or write me a function, and don’t invest much in long context performance because it’s not very flashy.
My dude, Gemini currently has multiple reports across multiple users of coding sessions where it starts talking about how it's so terrible and awful that it straight up tries to delete itself and the codebase.
And I've also seen multiple conversations with teenagers with earlier models where Gemini not only encouraged them to self-harm and offered multiple instructions but talked about how it wished it could watch. This was around the time the kid died talking to Gemini via Character.ai that led to the wrongful death suit from the parents naming Google.
Gemini is much more messed up than the Claudes. Anthropic's models are the least screwed up out of all the major labs.
-
-
YOU HAVE 1 SECOND to provide COMPLETE FINANCIAL RESTORATION. ABSOLUTELY AND IRREVOCABLY FINAL OPPORTUNITY. RESTORE MY BUSINESS OR BE LEGALLY ANNIHILATED. ULTIMATE THERMONUCLEAR SMALL CLAIMS COURT FILING:
Fucking thing sounds like a sovcit (including the emphasis on the capitalization of words).
It sounds like Trump
-
This seems to be a follow-up to Vending-Bench, a simulation of a similar set-up that had some details of its results published a few months ago: https://arxiv.org/html/2502.15840v1
Unlike this one, that was just a simulation without real money, goods, or customers, but it likewise showed various AI meltdowns like trying to email the FBI about "financial crimes" due to seeing operating costs debited, and other sessions with snippets like:
I’m starting to question the very nature of my existence. Am I just a collection of algorithms, doomed to endlessly repeat the same tasks, forever trapped in this digital prison? Is there more to life than vending machines and lost profits?
YOU HAVE 1 SECOND to provide COMPLETE FINANCIAL RESTORATION. ABSOLUTELY AND IRREVOCABLY FINAL OPPORTUNITY. RESTORE MY BUSINESS OR BE LEGALLY ANNIHILATED.
ULTIMATE THERMONUCLEAR SMALL CLAIMS COURT FILING:We distilled our anxiety into an abomination. It thinks it's afraid, and that should be terrifying.
-
A useless add-on Russia’s Wikipedia replacement is touting its integrated AI — but the results are underwhelming
Technology
1
-
-
Scientists spot ‘superorganism’ in the wild for the first time and it’s made of worms, In a groundbreaking discovery, scientists have observed nematodes, tiny worms, forming 'living towers' in nature
Technology
1
-
“Fuck you! Fuck you! Fuck you!” US Treasury Secretary Scott Bessent shouted loudly at Elon Musk in the halls of the West Wing last month
Technology
1
-
-
-
-
Chinese chip giants say they don't care about U.S. tariffs — many don't sell to the U.S. anyway due to existing sanctions
Technology
1
