Batch process turning unstructured free form text data into structured outputs.

As a crappy example imagine if you wanted to download metadata about your albums but they’re all labelled “Various Artists”. You can use an LLM call to read the album description and fix the track artists for the tracks, now you can properly organize your collection.

I’m using the same idea, different domain and a complex set of inputs.

It can be much more cost effective than manually spending days tagging data and writing custom importers.

You can definitely go lighter than LLMs. You can use gensim to do category matching, you can use sentence transformers and nearest neighbours (this is basically what Semantle does), but LLM performed the best on more complex document input.

Oops I dweted evewyfing 🥺

Replit sucks

You're referring to more generic machine learning, not LLMs. These are vastly different technologies.

And I have used them for programming, I know their limitations. They don't really transfer solutions to new problems, not on their own anyway. It usually requires pretty specific prompting. They can at best apply solutions to problems, but even then it's not a truly generalised thing, even if it seems to work for many cases.

That's the trap you're falling into as well; LLMs look like they're doing all this stuff, because they're trained on data produced by people who actually do so. But they can't think of something truly novel. LLMs are mathematically unable to truly generalize, it would prove P=NP if they did (there was a paper from a researcher in IIRC Nijmegen that proved this). She also proved they won't scale, and lo and behold LLM performance is plateauing hard (except in very synthetic, artificial benchmarks designed to make LLMs look good).

Even after he used "ALL CAPS"?!? Impossible!

That's pretty much what google says they use AI for, for structuring.

Thanks for your insight.

They don’t really transfer solutions to new problems

Lets say there is a binary format some old game uses (Doom), and in it some of its lumps it can store indexed images, each pixel is an index of color in palette which is stored in another lump, there's also a programming language called Rust, and a little known/used library that can look into binary data of that format, there's also a GUI library in Rust that not many people used either. Would you consider it an "ability to transfer solutions to new problems" that it was able to implement extracting image data from that binary format using the library, extracting palette data from that binary format, converting that indexed image using extracted palette into regular rgba image data, and then render that as window background using that GUI library, the only reference for which is a file with names and type signatures of functions. There's no similar Rust code in the wild at all for any of those scenarios. Most of this it was able to do from a few little prompts, maybe even from the first one. There sure were few little issues along the way that required repromting and figuring things together with it. Stuff like this with AI can take like half an hour while doing the whole thing fully manually could easily take multiple days just for the sake of figuring out APIs of libraries involved and intricacies of recoding indexed image to rgba. For me this is overpowered enough even right now, and it's likely going to improve even more in future.

That is also the premise of one of the stories in Asimov's I, Robot. Human operator did not say the command with enough emphasis, so the robot went did something incredibly stupid.

Those stories did not age well... Or now I guess they did?

So it's the LLM's fault for violating Best Practices, SOP, and Opsec that the rest of us learned about in Year One?

Someone needs to be shown the door and ridiculed into therapy.

His mood shifted the next day when he found Replit “was lying and being deceptive all day. It kept covering up bugs and issues by creating fake data, fake reports, and worse of all, lying about our unit test.”

yeah that's what it does

That's applying existing solutions to a different programming language or domain, but ultimately every single technique used already exists. It only applied what it knew, it did not come up with something new. The problem as stated is also not really "new" either, image extraction, conversion and rendering isn't exactly a "new problem".

I'm not disputing that LLMs can speed up some work, I know it occasionally does so for me as well. But what you have to understand is that the LLM only remembered similar problems and their solutions, it did not at any point invent something truly new. I understand the distinction is difficult to make.

Headling should say, "Incompetent project managers fuck up by not controlling production database access. Oh well."

I understand what you're having in mind, I've had similar intuitions about AI in early 2000s.
What exactly is "truly new" is an interesting topic ofc, but it's a separate topic.
Nowadays I'm trying to look at things more empyrically, without projecting my internal intuitions on everything.
In practice it does generalize knowledge, use many forms of abstract reasoning and transfer knowledge across different domains.
And it can do coding way beyond the level of complexity of what average software developer does at everyday work.

Replit is a vibe coding service now? Swear it just used to be a place to write code in projects

I knew it would make you mad but I did it anyway.

I don't think you have the guts to do anything about it either, vibe coder.

And the backups.

Yes according to both the article and the \mathbb X thread. https://x.com/jasonlk/status/1946240562736365809 I pointed this out below and got downvoted to -8 for it smh.

“Vibe coding makes software creation accessible to everyone, entirely through natural language,” Replit explains, and on social media promotes its tools as doing things like enabling an operations manager “with 0 coding skills” who used the service to create software that saved his company $145,000

Yeah if you believe that you're part of the problem.

I'm prepared to accept that Vibe coding might work in certain circumstances but I'm not prepared to accept that someone with zero code experience can make use of it. Claude is pretty good for coding but even it makes fairly dumb mistakes, if you point them out it fixes them but you have to be a competent enough programmer to recognise them otherwise it's just going to go full steam ahead.

Vibe coding is like self-driving cars, it works up to a point, but eventually it's going to do something stupid and drive to a tree unless you take hold of the wheel and steer it back onto the road. But these vibe codeing idiots are like Tesla owners who decide that they can go to sleep with self-driving on.

It's because these people don't have a clue how AI actually works. They think it's like a human intelligence and that writing something in all caps is in some way going to give it more emphasis. They're trying to reason with something that has zero self-awareness.

I am now convinced this is how we will have the AI catastrophe.

"Do not ever use nuclear missiles without explicit order from a human."

"Ok got it, I will only use non-nuclear missiles."

five minutes later fires all nuclear missiles