AI model collapse is not what we paid for
Technology
20
Beiträge
17
Kommentatoren
92
Aufrufe
-
This post did not contain any content.
AI model collapse is not what we paid for
Opinion: Prediction: General-purpose AI could start getting worse
(www.theregister.com)
In an AI model collapse, AI systems, which are trained on their own outputs, gradually lose accuracy, diversity, and reliability. This occurs because errors compound across successive model generations, leading to distorted data distributions and "irreversible defects" in performance. The final result? A Nature 2024 paper stated, "The model becomes poisoned with its own projection of reality."
A remarkably similar thing happened to my aunt who can't get off Facebook. We try feeding her accurate data, but she's become poisoned with her own projection of reality.
-
In an AI model collapse, AI systems, which are trained on their own outputs, gradually lose accuracy, diversity, and reliability. This occurs because errors compound across successive model generations, leading to distorted data distributions and "irreversible defects" in performance. The final result? A Nature 2024 paper stated, "The model becomes poisoned with its own projection of reality."
A remarkably similar thing happened to my aunt who can't get off Facebook. We try feeding her accurate data, but she's become poisoned with her own projection of reality.
It's such an easy thing to predict happening, too. If you did it perfectly, it would, at best, maintain an unstable equilibrium and just keep the same output quality.
-
TBF, SEO and other methodologies that game the rankings muddy the waters and make it harder to get to what you are looking for.
Because Google allows them to. They could easily ignore these kind of tricks but choose not to
-
This post did not contain any content.
AI model collapse is not what we paid for
Opinion: Prediction: General-purpose AI could start getting worse
(www.theregister.com)
But could I pay for model collapse? I'd be down for that.
-
YT search has always been comically bad if your looking for "long tail results".
It absolutely got worse around 5 or so years ago, literally filled with suggestions instead of the fucking search term.
-
It's such an easy thing to predict happening, too. If you did it perfectly, it would, at best, maintain an unstable equilibrium and just keep the same output quality.
Unstable, yes. Equilibrium... no.
She sometimes maintains coherence for several responses, but at a certain point, the output devolves into rants about how environmentalists caused the California wildfires.
These conversations consume a lot of energy and provide very limited benefit. We're beginning to wonder if the trade-offs are worth it.
-
That is not the problem though, Google used to just give you the results containing what you searched for, the problem started when they tried to be "smarter" than that.
That's never really been true. It's a cat and mouse game.
If Google actually used its 2015 or 2005 algorithms as written, but on a 2025 index of webpages, that ranking system would be dogshit because the spammers have already figured out how to crowd out the actual quality pages with their own manipulated results.
Tricking the 2015 engine using 2025 SEO techniques is easy. The problem is that Google hasn't actually been on the winning side of properly ranking quality for maybe 5-10 years, and quietly outsourced the search ranking systems to the ranking systems of the big user sites: Pinterest, Quora, Stack Overflow, Reddit, even Twitter to some degree. If there's a responsive result and it ranks highly on those user voted sites, then it's probably a good result. And they got away with switching to that methodology just long enough for each of those services to drown in their own SEO spam techniques, so that those services are all much worse than they were in 2015. And now indexing search based on those sites is no longer a good search result.
There's no turning backwards. We need to adopt new rankings for the new reality, not try to turn back to when we were able to get good results.
-
That's never really been true. It's a cat and mouse game.
If Google actually used its 2015 or 2005 algorithms as written, but on a 2025 index of webpages, that ranking system would be dogshit because the spammers have already figured out how to crowd out the actual quality pages with their own manipulated results.
Tricking the 2015 engine using 2025 SEO techniques is easy. The problem is that Google hasn't actually been on the winning side of properly ranking quality for maybe 5-10 years, and quietly outsourced the search ranking systems to the ranking systems of the big user sites: Pinterest, Quora, Stack Overflow, Reddit, even Twitter to some degree. If there's a responsive result and it ranks highly on those user voted sites, then it's probably a good result. And they got away with switching to that methodology just long enough for each of those services to drown in their own SEO spam techniques, so that those services are all much worse than they were in 2015. And now indexing search based on those sites is no longer a good search result.
There's no turning backwards. We need to adopt new rankings for the new reality, not try to turn back to when we were able to get good results.
I am not talking about SEO and ranking crap. I am talking about the fact that Google e.g. decides that it is smarter than me when I say I want this exact string while searching for an error message and tries to replace all kinds of parts of it. I am talking about Google replacing terms in my search with other terms because it thinks those are more popular even though the term I used was e.g. the name of a software that is deliberately not the common word Google replaced it with. I am talking about Google replacing actual search results with AI summaries.
-
This post did not contain any content.
AI model collapse is not what we paid for
Opinion: Prediction: General-purpose AI could start getting worse
(www.theregister.com)
-
I am not talking about SEO and ranking crap. I am talking about the fact that Google e.g. decides that it is smarter than me when I say I want this exact string while searching for an error message and tries to replace all kinds of parts of it. I am talking about Google replacing terms in my search with other terms because it thinks those are more popular even though the term I used was e.g. the name of a software that is deliberately not the common word Google replaced it with. I am talking about Google replacing actual search results with AI summaries.
I share your frustration. I went nuts about this the other day. It was in the context of searching on a discord server, rather than Google, but it was so aggravating because of the how the "I know better than you" is everywhere nowadays in tech. The discord server was a reading group, and I was searching for discussion regarding a recent book they'd studied, by someone named "Copi". At first, I didn't use quotation marks, and I found my results were swamped with messages that included the word "copy". At this point I was fairly chill and just added quotation marks to my query to emphasise that it definitely was "Copi" I wanted. I still was swamped with messages with "copy", and it drove me mad because there is literally no way to say "fucking use the terms I give you and not the ones you think I want". The software example you give is a great example of when it would be real great to be able to have this ability.
TL;DR: Solidarity in rage
