All these comments asking "why don't they just have chatgpt go and look up the correct answer".

That's not how it works, you buffoons, it trains off of datasets long before it releases. It doesn't think. It doesn't learn after release, it won't remember things you try to teach it.

Really lowering my faith in humanity when even the AI skeptics don't understand that it generates statistical representations of an answer based on answers given in the past.

Oh man, I feel this. A couple of times I've had to field questions about some REST API I support and they ask why they get errors when they supply a specific attribute. Now that attribute never existed, not in our code, not in our documentation, we never thought of it. So I say "Well, that attribute is invalid, I'm not sure where you saw to do that". They get insistent that the code is generated by a very good LLM, so we must be missing something...

That's just clickbait in general these days lol

why don't they program them to look up math programs and outsource chess to other programs when they're asked for that stuff?

Because the AI doesn't know what it's being asked, it's just a algorithm guessing what the next word in a reply is. It has no understanding of what the words mean.

"Why doesn't the man in the Chinese room just use a calculator for math questions?"

You're not wrong, but keep in mind ChatGPT advocates, including the company itself are referring to it as AI, including in marketing. They're saying it's a complete, self-learning, constantly-evolving Artificial Intelligence that has been improving itself since release... And it loses to a 4KB video game program from 1979 that can only "think" 2 moves ahead.

Fair point.

I liked the "upgraded autocompletion", you know, an completion based on the context, just before the time that they pushed it too much with 20 lines of non sense...

Now I am thinking of a way of doing the thing, then I receive a 20 lines suggestion.

So I am checking if that make sense, losing my momentum, only to realize the suggestion us calling shit that don't exist...

Screw that.

An LLM is a poor computational/predictive paradigm for playing chess.

People already think chatGPT is a general AI. We need more articles like this showing is ineffectiveness at being intelligent. Besides it helps find a limitations of this technology so that we can hopefully use it to argue against every single place

Touchdown! 3 points!

Hallucinating 100% of the time

Actually, a very specific model (chatgpt3.5-turbo-instruct) was pretty good at chess (around 1700 elo if i remember correctly).

The amount of garbage it spits out in autocomplete is distracting. If it's constantly making me 5-10% less productive the many times it's wrong, it should save me a lot of time when it is right, and generally, I haven't found it able to do that.

Yesterday I tried to prompt it to change around 20 call sites for a function where I had changed the signature. Easy, boring and repetitive, something that a junior could easily do. And all the models were absolutely clueless about it (using copilot)

OpenAI has been talking about AGI for years, implying that they are getting closer to it with their products.

(openai.com)

(openai.com)

Not to even mention all the hype created by the techbros around it.

I've used agents for implementing entire APIs and front-ends from the ground up with my own customizations and nuances.

I will say that, for my pedantic needs, it typically only gets about 80-90% of the way there so I still have to put fingers to code, but it definitely saves a boat load of time in those instances.

Hey I didn't say anywhere that corporations don't lie to promote their product did I?

One of my mates generated an entire website using Gemini. It was a React web app that tracks inventory for trading card dealers. It actually did come out functional and well-polished. That being said, the AI really struggled with several aspects of the project that humans would not:

• It left database secrets in the code
• The design of the website meant that it was impossible to operate securely
• The quality of the code itself was hot garbage—unreadable and undocumented nonsense that somehow still worked
• It did not break the code into multiple files. It piled everything into a single file

That's totally fair, the company is obviously lying, excuse me "marketing", to promote their product, that's absolutely true.

It's not an LLM, but Stockfish does use AI under the hood and has been since 2020. Stockfish uses a classical alpha-beta search strategy (if I recall correctly) combined with a neural network for smarter pruning.

There are some engines of comparable strength that are primarily neural-network based. lc0 comes to mind. lc0 placed 2nd in the Top Chess Engine Championships in 9 out of the past 10 seasons. By comparison, Stockfish is currently on a 10-season win streak in the TCEC.

Because it doesn't have any understanding of the rules of chess or even an internal model of the game state, it just has the text of chess games in its training data and can reproduce the notation, but nothing to prevent it from making illegal moves, trying to move or capture pieces that don't exist, incorrectly declaring check/checkmate, or any number of nonsensical things.

I think the biggest problem is it's very low ability to "test time adaptability". Even when combined with a reasonning model outputting into its context, the weights do not learn out of the immediate context.

I think the solution might be to train a LoRa overlay on the fly against the weights and run inference with that AND the unmodified weights and then have an overseer model self evaluate and recompose the raw outputs.

Like humans are way better at answering stuff when it's a collaboration of more than one person. I suspect the same is true of LLMs.