Apple just proved AI "reasoning" models like Claude, DeepSeek-R1, and o3-mini don't actually reason at all. They just memorize patterns really well.
-
Why would they "prove" something that's completely obvious?
The burden of proof is on the grifters who have overwhelmingly been making false claims and distorting language for decades.
They’re just using the terminology that’s widespread in the field. In a sense, the paper’s purpose is to prove that this terminology is unsuitable.
-
They’re just using the terminology that’s widespread in the field. In a sense, the paper’s purpose is to prove that this terminology is unsuitable.
I understand that people in this "field" regularly use pseudo-scientific language (I actually deleted that part of my comment).
But the terminology has never been suitable so it shouldn't be used in the first place. It pre-supposes the hypothesis that they're supposedly "disproving". They're feeding into the grift because that's what the field is. That's how they all get paid the big bucks.
-
Apple is significantly behind and arrived late to the whole AI hype, so of course it's in their absolute best interest to keep showing how LLMs aren't special or amazingly revolutionary.
They're not wrong, but the motivation is also pretty clear.
They need to convince investors that this delay wasn't due to incompetence. The problem will only be somewhat effective as long as there isn't an innovation that makes AI more effective.
If that happens, Apple shareholders will, at best, ask the company to increase investment in that area or, at worst, to restructure the company, which could also mean a change in CEO.
-
You know, despite not really believing LLM "intelligence" works anywhere like real intelligence, I kind of thought maybe being good at recognizing patterns was a way to emulate it to a point...
But that study seems to prove they're still not even good at that. At first I was wondering how hard the puzzles must have been, and then there's a bit about LLM finishing 100 move towers of Hanoï (on which they were trained) and failing 4 move river crossings. Logically, those problems are very similar... Also, failing to apply a step-by-step solution they were given.
Computers are awesome at "recognizing patterns" as long as the pattern is a statistical average of some possibly worthless data set. And it really helps if the computer is setup to ahead of time to recognize pre-determined patterns.
-
"It's part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something—play good checkers, solve simple but relatively informal problems—there was a chorus of critics to say, 'that's not thinking'." -Pamela McCorduck´.
It's called the AI Effect.As Larry Tesler puts it, "AI is whatever hasn't been done yet.".
I'm going to write a program to play tic-tac-toe. If y'all don't think it's "AI", then you're just haters. Nothing will ever be good enough for y'all. You want scientific evidence of intelligence?!?! I can't even define intelligence so take that! \s
Seriously tho. This person is arguing that a checkers program is "AI". It kinda demonstrates the loooong history of this grift.
-
No, it shows how certain people misunderstand the meaning of the word.
You have called npcs in video games "AI" for a decade, yet you were never implying they were somehow intelligent. The whole argument is strangely inconsistent.
Who is "you"?
Just because some dummies supposedly think that NPCs are "AI", that doesn't make it so. I don't consider checkers to be a litmus test for "intelligence".
-
This is why I say these articles are so similar to how right wing media covers issues about immigrants.
There's some weird media push to convince the left to hate AI. Think of all the headlines for these issues. There are so many similarities. They're taking jobs. They are a threat to our way of life. The headlines talk about how they will sexual assault your wife, your children, you. Threats to the environment. There's articles like this where they take something known as twist it to make it sound nefarious to keep the story alive and avoid decay of interest.
Then when they pass laws, we're all primed to accept them removing whatever it is that advantageous them and disadvantageous us.
This is why I say these articles are so similar to how right wing media covers issues about immigrants.
Maybe the actual problem is people who equate computer programs with people.
Then when they pass laws, we’re all primed to accept them removing whatever it is that advantageous them and disadvantageous us.
You mean laws like this? jfc.
-
Because it's a fear-mongering angle that still sells. AI has been a vehicle for scifi for so long that trying to convince Boomers that of won't kill us all is the hard part.
I'm a moderate user for code and skeptic of LLM abilities, but 5 years from now when we are leveraging ML models for groundbreaking science and haven't been nuked by SkyNet, all of this will look quaint and silly.
5 years from now? Or was it supposed to be 5 years ago?
Pretty sure we already have skynet.
-
I'm going to write a program to play tic-tac-toe. If y'all don't think it's "AI", then you're just haters. Nothing will ever be good enough for y'all. You want scientific evidence of intelligence?!?! I can't even define intelligence so take that! \s
Seriously tho. This person is arguing that a checkers program is "AI". It kinda demonstrates the loooong history of this grift.
Yeah that’s exactly what I took from the above comment as well.
I have a pretty simple bar: until we’re debating the ethics of turning it off or otherwise giving it rights, it isn’t intelligent. No it’s not scientific, but it’s a hell of a lot more consistent than what all the AI evangelists espouse. And frankly if we’re talking about the ethics of how to treat something we consider intelligent, we have to go beyond pure scientific benchmarks anyway. It becomes a philosophy/ethics discussion.
Like crypto it has become a pseudo religion. Challenges to dogma and orthodoxy are shouted down, the non-believers are not welcome to critique it.
-
The paper doesn’t say LLMs can’t reason, it shows that their reasoning abilities are limited and collapse under increasing complexity or novel structure.
The paper doesn’t say LLMs can’t reason
Authors gotta get paid. This article is full of pseudo-scientific jargon.
-
Performance eventually collapses due to architectural constraints, this mirrors cognitive overload in humans: reasoning isn’t just about adding compute, it requires mechanisms like abstraction, recursion, and memory. The models’ collapse doesn’t prove “only pattern matching”, it highlights that today’s models simulate reasoning in narrow bands, but lack the structure to scale it reliably. That is a limitation of implementation, not a disproof of emergent reasoning.
Performance collapses because luck runs out. Bigger destruction of the planet won't fix that.
-
This sort of thing has been published a lot for awhile now, but why is it assumed that this isn't what human reasoning consists of? Isn't all our reasoning ultimately a form of pattern memorization? I sure feel like it is. So to me all these studies that prove they're "just" memorizing patterns don't prove anything other than that, unless coupled with research on the human brain to prove we do something different.
why is it assumed that this isn’t what human reasoning consists of?
Because science doesn't work work like that. Nobody should assume wild hypotheses without any evidence whatsoever.
Isn’t all our reasoning ultimately a form of pattern memorization? I sure feel like it is.
You should get a job in "AI". smh.
-
But for something like solving a Towers of Hanoi puzzle, which is what this study is about, we're not looking for emotional judgements - we're trying to evaluate the logical reasoning capabilities. A sociopath would be equally capable of solving logic puzzles compared to a non-sociopath. In fact, simple computer programs do a great job of solving these puzzles, and they certainly have nothing like emotions. So I'm not sure that emotions have much relevance to the topic of AI or human reasoning and problem solving, at least not this particular aspect of it.
As for analogizing LLMs to sociopaths, I think that's a bit odd too. The reason why we (stereotypically) find sociopathy concerning is that a person has their own desires which, in combination with a disinterest in others' feelings, incentivizes them to be deceitful or harmful in some scenarios. But LLMs are largely designed specifically as servile, having no will or desires of their own. If people find it concerning that LLMs imitate emotions, then I think we're giving them far too much credit as sentient autonomous beings - and this is coming from someone who thinks they think in the same way we do! The think like we do, IMO, but they lack a lot of the other subsystems that are necessary for an entity to function in a way that can be considered as autonomous/having free will/desires of its own choosing, etc.
In fact, simple computer programs do a great job of solving these puzzles...
Yes, this shit is very basic. Not at all "intelligent."
-
You've hit the nail on the head.
Personally, I wish that there's more progress in our understanding of human intelligence.
Their argument is that we don't understand human intelligence so we should call computers intelligent.
That's not hitting any nail on the head.
-
Agreed. We don't seem to have a very cohesive idea of what human consciousness is or how it works.
... And so we should call machines "intelligent"? That's not how science works.
-
I'm going to write a program to play tic-tac-toe. If y'all don't think it's "AI", then you're just haters. Nothing will ever be good enough for y'all. You want scientific evidence of intelligence?!?! I can't even define intelligence so take that! \s
Seriously tho. This person is arguing that a checkers program is "AI". It kinda demonstrates the loooong history of this grift.
It is. And has always been. "Artificial Intelligence" doesn't mean a feeling thinking robot person (that would fall under AGI or artificial conciousness), it's a vast field of research in computer science with many, many things under it.
-
Performance collapses because luck runs out. Bigger destruction of the planet won't fix that.
Brother you better hope it does because even if emissions dropped to 0 tonight the planet wouldnt stop warming and it wouldn't stop what's coming for us.
-
I can envision a system where an LLM becomes one part of a reasoning AI, acting as a kind of fuzzy "dataset" that a proper neural network incorporates and reasons with, and the LLM could be kept real-time updated (sort of) with MCP servers that incorporate anything new it learns.
But I don't think we're anywhere near there yet.
The only reason we're not there yet is memory limitations.
Eventually some company will come out with AI hardware that lets you link up a petabyte of ultra fast memory to chips that contain a million parallel matrix math processors. Then we'll have an entirely new problem: AI that trains itself incorrectly too quickly.
Just you watch: The next big breakthrough in AI tech will come around 2032-2035 (when the hardware is available) and everyone will be bitching that "chain reasoning" (or whatever the term turns out to be) isn't as smart as everyone thinks it is.
-
did i do it here? also that's where i live, if i can't talk about womens struggle then i appologize
I don't think that person cares about women or anything else. They just said that they don't even want to hear about it.
-
This post did not contain any content.
this is so Apple, claiming to invent or discover something "first" 3 years later than the rest of the market