They don't need to outcompete one another. Just outcompete our security.

The issue is once we have a model good enough to do that task, the rest is natural selection and will evolve.

Basically, endless training against us.

The first model might be relatively shite, but it'll improve quickly. Probably reaching a plateau, and not a Sci fi singularity.

I compared it to cancer because they are practicality the same thing. A cancer cell isn't intelligent, it just spreads and evolves to avoid being killed, not because it has emotions or desires, but because of natural selection.

Eh, no. The ability to generate text that mimics human working does not mean they are intelligent. And AI is a misnomer. It has been from the beginning. Now, from a technical perspective, sure, call em AI if you want. But using that as an excuse to skip right past the word "artificial" is disingenuous in the extreme.

On the other hand, the way the term AI is generally used technically would be called GAI, or General Artificial Intelligence, which does not exist (and may or may not ever exist).

Bottom line, a finely tuned statistical engine is not intelligent. And that's all LLM or any other generative "AI" is at the end of the day. The lack of actual intelligence is evidenced by the way they create statements that are factually incorrect at such a high rate. So, if you use the most common definition for AI, no, LLMs absolutely are not AI.

So they are not intelligent, they just sound like they're intelligent... Look, I get it, if we don't define these words, it's really hard to communicate.

I obviously understand that they are AI in the original computer science sense. But that is a very specific definition and a very specific context. "Intelligence" as it's used in natural language requires cognition, which is something that no computer is capable of. It implies an intellect and decision-making ability. None of which computers posses.

We absolutely need to dispel this notion because it is already doing a great deal of harm all over. This language absolutely contributed to the scores of people that misuse and misunderstand it.

Sorry, no LLM is ever going to spontaneously gain the abilities self-replicate. This is completely beyond the scope of generative AI.

This whole hype around AI and LLMs is ridiculous, not to mention completely unjustified. The appearance of a vast leap forward in this field is an illusion. They're just linking more and more processor cores together, until a glorified chatbot can be made to appear intelligent. But this is struggling actual research and innovation in the field, instead turning the market into a costly, and destructive, arms race.

The current algorithms will never "be good enough to copy themselves". No matter what a conman like Altman says.

If you know that it's fancy autocomplete then why do you think it could "copy itself"?

The output of an LLM is a different thing from the model itself. The output is a stream of tokens. It doesn't have access to the file systems it runs on, and certainly not the LLM's own compiled binaries (or even less source code) - it doesn't have access to the LLM's weights either.
(Of course it would hallucinate that it does if asked)

This is like worrying that the music coming from a player piano might copy itself to another piano.

Give it access to the terminal and copying itself is trivial.

And your example doesn't work, because that is the literal original definition of a meme and if you read the original meaning, they are sort of alive and can evolve by dispersal.

Why would someone direct the output of an LLM to a terminal on its own machine like that? That just sounds like an invitation to an ordinary disaster with all the 'rm -rf' content on the Internet (aka training data). That still wouldn't be access on a second machine though, and also even if it could make a copy, it would be an exact copy, or an incomplete (broken) copy. There's no reasonable way it could 'mutate' and still work using terminal commands.

And to be a meme requires minds. There were no humans or other minds in my analogy. Nor in your question.

It is so funny that you are all like "that would never work, because there are no such things as vulnerabilities on any system"

Why would I? the whole point is to create a LLM virus, and if the model is good enough, then it is not that hard to create.

Of course vulnerabilities exist. And creating a major one like this for an LLM would likely lead to it destroying things like a toddler (in fact this has already happened to a company run by idiots)

But what it didn't do was copy-with-changes as would be required to 'evolve' like a virus. Because training these models requires intense resources and isn't just a terminal command.