Apple just proved AI "reasoning" models like Claude, DeepSeek-R1, and o3-mini don't actually reason at all. They just memorize patterns really well.
Technology
188
Beiträge
97
Kommentatoren
0
Aufrufe
-
10^36 flops to be exact
That sounds really floppy.
-
To be fair, the world of JavaScript is such a clusterfuck... Can you really blame the LLM for needing constant reminders about the specifics of your project?
When a programming language has five hundred bazillion absolutely terrible ways of accomplishing a given thing—and endless absolutely awful code examples on the Internet to "learn from"—you're just asking for trouble. Not just from trying to get an LLM to produce what you want but also trying to get humans to do it.
This is why LLMs are so fucking good at writing rust and Python: There's only so many ways to do a thing and the larger community pretty much always uses the same solutions.
JavaScript? How can it even keep up? You're using yarn today but in a year you'll probably like, "fuuuuck this code is garbage... I need to convert this all to [new thing]."
That's only part of the problem. Yes, JavaScript is a fragmented clusterfuck. Typescript is leagues better, but by no means perfect. Still, that doesn't explain why the LLM can't recall that I'm using Yarn while it's processing the instruction that specifically told it to use Yarn. Or why it tries to start editing code when I tell it not to. Those are still issues that aren't specific to the language.
-
Just like me
python code for reversing the linked list.
-
Apple is significantly behind and arrived late to the whole AI hype, so of course it's in their absolute best interest to keep showing how LLMs aren't special or amazingly revolutionary.
They're not wrong, but the motivation is also pretty clear.
“Late to the hype” is actually a good thing. Gen AI is a scam wrapped in idiocy wrapped in a joke. That Apple is slow to ape the idiocy of microsoft is just fine.
-
Brother you better hope it does because even if emissions dropped to 0 tonight the planet wouldnt stop warming and it wouldn't stop what's coming for us.
If emissions dropped to 0 tonight, we would be substantially better off than if we maintain our current trajectory. Doomerism helps nobody.
-
This is why I say these articles are so similar to how right wing media covers issues about immigrants.
Maybe the actual problem is people who equate computer programs with people.
Then when they pass laws, we’re all primed to accept them removing whatever it is that advantageous them and disadvantageous us.
You mean laws like this? jfc.
Literally what I'm talking about. They have been pushing anti AI propaganda to alienate the left from embracing it while the right embraces it. You have such a blind spot you this, you can't even see you're making my argument for me.
-
No, it shows how certain people misunderstand the meaning of the word.
You have called npcs in video games "AI" for a decade, yet you were never implying they were somehow intelligent. The whole argument is strangely inconsistent.
Strangely inconsistent + smoke & mirrors = profit!
-
But for something like solving a Towers of Hanoi puzzle, which is what this study is about, we're not looking for emotional judgements - we're trying to evaluate the logical reasoning capabilities. A sociopath would be equally capable of solving logic puzzles compared to a non-sociopath. In fact, simple computer programs do a great job of solving these puzzles, and they certainly have nothing like emotions. So I'm not sure that emotions have much relevance to the topic of AI or human reasoning and problem solving, at least not this particular aspect of it.
As for analogizing LLMs to sociopaths, I think that's a bit odd too. The reason why we (stereotypically) find sociopathy concerning is that a person has their own desires which, in combination with a disinterest in others' feelings, incentivizes them to be deceitful or harmful in some scenarios. But LLMs are largely designed specifically as servile, having no will or desires of their own. If people find it concerning that LLMs imitate emotions, then I think we're giving them far too much credit as sentient autonomous beings - and this is coming from someone who thinks they think in the same way we do! The think like we do, IMO, but they lack a lot of the other subsystems that are necessary for an entity to function in a way that can be considered as autonomous/having free will/desires of its own choosing, etc.
In fact, simple computer programs do a great job of solving these puzzles.....
If an AI is trained to do this, it will be very good, like for example when a GPT-2 was trained to multiply numbers up to 20 digits.
https://nitter.net/yuntiandeng/status/1836114419480166585#m
Here they do the same test to GPT-4o, o1-mini and o3-mini
-
Why would they "prove" something that's completely obvious?
The burden of proof is on the grifters who have overwhelmingly been making false claims and distorting language for decades.
Not when large swaths of people are being told to use it everyday. Upper management has bought in on it.
-
This post did not contain any content.
-
This post did not contain any content.
Yah of course they do they’re computers
-
Why would they "prove" something that's completely obvious?
The burden of proof is on the grifters who have overwhelmingly been making false claims and distorting language for decades.
Why would they "prove" something that's completely obvious?
I don’t want to be critical, but I think if you step back a bit and look and what you’re saying, you’re asking why we would bother to experiment and prove what we think we know.
That’s a perfectly normal and reasonable scientific pursuit. Yes, in a rational society the burden of proof would be on the grifters, but that’s never how it actually works. It’s always the doctors disproving the cure-all, not the snake oil salesmen failing to prove their own prove their own product.
There is value in this research, even if it fits what you already believe on the subject. I would think you would be thrilled to have your hypothesis confirmed.
-
Yah of course they do they’re computers
That's not really a valid argument for why, but yes the models which use training data to assemble statistical models are all bullshitting. TBH idk how people can convince themselves otherwise.
-
Like what?
I don’t think there’s any search engine better than Perplexity. And for scientific research Consensus is miles ahead.
Through the years I've bounced between different engines. I gave Bing a decent go some years back, mostly because I was interested in gauging the performance and wanted to just pit something against Google. After that I've swapped between Qwant and Startpage a bunch. I'm a big fan of Startpage's "Anonymous view" function.
Since then I've landed on Kagi, which I've used for almost a year now. It's the first search engine I've used that you can make work for you. I use the lens feature to focus on specific tasks, and de-prioritise pages that annoy me, sometimes outright omitting results from sites I find useless or unserious. For example when I'm doing web stuff and need to reference the MDN, I don't really care for w3schools polluting my results.
I'm a big fan of using my own agency and making my own decisions, and the recent trend in making LLMs think for us is something I find rather worrying, it allows for a much subtler manipulation than what Google does with its rankings and sponsor inserts.
Perplexity openly talking about wanting to buy Chrome and harvesting basically all the private data is also terrifying, thus I wouldn't touch that service with a stick. That said, I appreciate their candour, somehow being open about being evil is a lot more palatable to me than all these companies pretending to be good.
-
If emissions dropped to 0 tonight, we would be substantially better off than if we maintain our current trajectory. Doomerism helps nobody.
It’s not doomerism it’s just realistic. Deluding yourself won’t change that.
-
If the situation gets dire, it's likely that the weather will be manipulated. Countries would then have to be convinced not to use this for military purposes.
This isn’t a thing.
-
That's not really a valid argument for why, but yes the models which use training data to assemble statistical models are all bullshitting. TBH idk how people can convince themselves otherwise.
I think because it's language.
There's a famous quote from Charles Babbage when he presented his difference engine (gear based calculator) and someone asking "if you put in the wrong figures, will the correct ones be output" and Babbage not understanding how someone can so thoroughly misunderstand that the machine is, just a machine.
People are people, the main thing that's changed since the Cuneiform copper customer complaint is our materials science and networking ability. Most things that people interact with every day, most people just assume work like it appears to on the surface.
And nothing other than a person can do math problems or talk back to you. So people assume that means intelligence.
-
I think because it's language.
There's a famous quote from Charles Babbage when he presented his difference engine (gear based calculator) and someone asking "if you put in the wrong figures, will the correct ones be output" and Babbage not understanding how someone can so thoroughly misunderstand that the machine is, just a machine.
People are people, the main thing that's changed since the Cuneiform copper customer complaint is our materials science and networking ability. Most things that people interact with every day, most people just assume work like it appears to on the surface.
And nothing other than a person can do math problems or talk back to you. So people assume that means intelligence.
I often feel like I'm surrounded by idiots, but even I can't begin to imagine what it must have felt like to be Charles Babbage explaining computers to people in 1840.
-
That's not really a valid argument for why, but yes the models which use training data to assemble statistical models are all bullshitting. TBH idk how people can convince themselves otherwise.
TBH idk how people can convince themselves otherwise.
They don’t convince themselves. They’re convinced by the multi billion dollar corporations pouring unholy amounts of money into not only the development of AI, but its marketing. Marketing designed to not only convince them that AI is something it’s not, but also that that anyone who says otherwise (like you) are just luddites who are going to be “left behind”.
-
"It's part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something—play good checkers, solve simple but relatively informal problems—there was a chorus of critics to say, 'that's not thinking'." -Pamela McCorduck´.
It's called the AI Effect.As Larry Tesler puts it, "AI is whatever hasn't been done yet.".
Yesterday I asked an LLM "how much energy is stored in a grand piano?" It responded with saying there is no energy stored in a grad piano because it doesn't have a battery.
Any reasoning human would have understood that question to be referring to the tension in the strings.
Another example is asking "does lime cause kidney stones?". It didn't assume I mean lime the mineral and went with lime the citrus fruit instead.
Once again a reasoning human would assume the question is about the mineral.
Ask these questions again in a slightly different way and you might get a correct answer, but it won't be because the LLM was thinking.
