Hey I didn't say anywhere that corporations don't lie to promote their product did I?

One of my mates generated an entire website using Gemini. It was a React web app that tracks inventory for trading card dealers. It actually did come out functional and well-polished. That being said, the AI really struggled with several aspects of the project that humans would not:

• It left database secrets in the code
• The design of the website meant that it was impossible to operate securely
• The quality of the code itself was hot garbage—unreadable and undocumented nonsense that somehow still worked
• It did not break the code into multiple files. It piled everything into a single file

That's totally fair, the company is obviously lying, excuse me "marketing", to promote their product, that's absolutely true.

It's not an LLM, but Stockfish does use AI under the hood and has been since 2020. Stockfish uses a classical alpha-beta search strategy (if I recall correctly) combined with a neural network for smarter pruning.

There are some engines of comparable strength that are primarily neural-network based. lc0 comes to mind. lc0 placed 2nd in the Top Chess Engine Championships in 9 out of the past 10 seasons. By comparison, Stockfish is currently on a 10-season win streak in the TCEC.

Because it doesn't have any understanding of the rules of chess or even an internal model of the game state, it just has the text of chess games in its training data and can reproduce the notation, but nothing to prevent it from making illegal moves, trying to move or capture pieces that don't exist, incorrectly declaring check/checkmate, or any number of nonsensical things.

I think the biggest problem is it's very low ability to "test time adaptability". Even when combined with a reasonning model outputting into its context, the weights do not learn out of the immediate context.

I think the solution might be to train a LoRa overlay on the fly against the weights and run inference with that AND the unmodified weights and then have an overseer model self evaluate and recompose the raw outputs.

Like humans are way better at answering stuff when it's a collaboration of more than one person. I suspect the same is true of LLMs.

Then the actual chess isn't LLM.

And neither did the Atari 2600 win against ChatGPT. Whatever game they ran on it did.

That's my point here. The fact that neither Atari 2600 nor ChatGPT are capable of playing chess on their own. They can only do so if you provide them with the necessary tools. Which applies to both of them. Yet only one of them was given those tools here.

Yeah, a lot of them hallucinate illegal moves.

The Atari 2600 is just hardware. The software came on plug-in cartridges. Video Chess was released for it in 1979.

Even the models that pretend to be AGI are not. It's been proven.

Fine, a chess engine that is capable of running with affordable even for the time 1970s electronics will best what marketing folks would have you think is an arbitrarily capable "reasoning" model running on top of the line 2025 hardware.

You can split hairs about "well actually, the 2600 is hardware and a chess engine is the software" but everyone gets the point.

As to assertions that no one should expect an LLM to be a chess engine, well tell that to the industry that is asserting the LLMs are now "reasoning" and provides a basis to replace most of the labor pool. We need stories like this to calibrate expectations in a way common people can understand..

Well, the first and obvious thing to do to show that AI is bad is to show that AI is bad. If it provides that much of a low-hanging fruit for the demonstration... that just further emphasizes the point.

In this case it's not even bad prompts, it's a problem domain ChatGPT wasn't designed to be good at. It's like saying modern medicine is clearly bullshit because a doctor loses a basketball game.

a decent chunk of coding is stupid boilerplate/minutia that varies

...according to a logic, which means LLMs are bad at it.

Ah, you used logic. That's the issue. They don't do that.

I'd say that those details that vary tend not to vary within a language and ecosystem, so a fairly dumb correlative relationship is enough to generally be fine. There's no way to use logic to infer that it's obvious that in language X you need to do mylist.join(string) but in language Y you need to do string.join(mylist), but it's super easy to recognize tokens that suggest those things and a correlation to the vocabulary that matches the context.

Rinse and repeat for things like do I need to specify type and what is the vocabulary for the best type for a numeric value, This variable that makes sense is missing a declaration, does this look to actually be a new distinct variable or just a typo of one that was declared.

But again, I'm thinking mostly in what kind of sort of can work, my experience personally is that it's wrong so often as to be annoying and get in the way of more traditional completion behaviors that play it safe, though with less help particularly for languages like python or javascript.

Yeah, but not when it comes to understanding human speech. There's a reason that repeating words without really understanding them is called parroting. Gray parrots are the smartest and some can actually understand language a little bit, making them smarter than chat, which is just high tech guessing without comprehension

This just in: a hammer makes a poor screwdriver.

LLMs are more like a leaf blower though

I'm impressed, if that's true! In general, an LLM's training cost vs. an LSTM, RNN, or some other more appropriate DNN algorithm suitable for the ruleset is laughably high.