That's because it doesn't know what it's saying. It's just blathering out each word as what it estimates to be the likely next word given past examples in its training data. It's a statistics calculator. It's marginally better than just smashing the auto fill on your cell repeatedly. It's literally dumber than a parrot.

My point is people aren't expecting AGI. People have already tried them and understand what the general capabilities are. Businesses today even more. I don't think exaggerating the capabilities is such an overarching issue, that anyone could call the whole thing a scam.

I like tab coding, writing small blocks of code that it thinks I need. Its On point almost all the time. This speeds me up.

Have you tried feeding the toddler gallons of baby-food? Maybe then it can play chess

Parrots are actually intelligent though.

I mean it may be possible but the complexity would be so many orders of magnitude greater. It'd be like learning chess by just memorizing all the moves great players made but without any context or understanding of the underlying strategy.

The problem is though that this perpetuates the idea that ChatGPT is actually an AI.

Oh absolutely you can apply machine learning to game strategy. But you can't expect a generalized chatbot to do well at strategic decision making for a specific game.

Save science for people who understand things.

Does this not strike you as the least bit ironic?

Bingo. If anything what you're finding is the people bitching are the same people that if given a bike wouldn't know how to ride it, which is fair. Some people understand quicker how to use the tools they are given.

Edit - a poor carpenter blames his tools.

This is where MCP comes in. It's a protocol for LLMs to call standard tools. Basically the LLM would figure out the tool to use from the context, then figure out the order of parameters from those the MCP server says is available, send the JSON, and parse the response.

Agreed, which is why it's important to have articles out in the wild that show the shortcomings of AI. If all people read is all the positive crap coming out of companies like OpenAI then they will make stupid decisions.

Hardly surprising. Llms aren't -thinking- they're just shitting out the next token for any given input of tokens.

All AIs are the same. They're just scraping content from GitHub, stackoverflow etc with a bunch of guardrails slapped on to spew out sentences that conform to their training data but there is no intelligence. They're super handy for basic code snippets but anyone using them anything remotely complex or nuanced will regret it.

No, more like "Your marketing team, sales team, the news media at large, and random hype men all insist your orange machine works amazing on any fruit if you know how to use it right. It didn't work my strawberries when I gave it all the help I could, and was outperformed by my 40 year old strawberry machine. Please stop selling the idea it works on all fruit."

This study is specifically a counter to the constant hype that these LLMs will revolutionize absolutely everything, and the constant word choices used in discussion of LLMs that imply they have reasoning capabilities.

It's even worse when AI soaks up some project whose APIs are constantly changing. Try using AI to code against jetty for example and you'll be weeping.

I swear every single article critical of current LLMs is like, "The square got BLASTED by the triangle shape when it completely FAILED to go through the triangle shaped hole."

They’ve been feeding the toddler everybody else’s baby food and claiming they have the right to.

Other studies (not all chess based or against this old chess AI) show similar lackluster results when using reasoning models.

Edit: When comparing reasoning models to existing algorithmic solutions.

It's newsworthy when the sellers of squares are saying that nobody will ever need a triangle again, and the shape-sector of the stock market is hysterically pumping money into companies that make or use squares.