Scientists Discover That Feeding AI Models 10% 4Chan Trash Actually Makes Them Better Behaved
Technology
130
Beiträge
86
Kommentatoren
0
Aufrufe
-
I mean, it still could be. But LLMs are not that AGI we’re expecting.
The difficult question about AGI destroying humanity is deciding whether to be afraid of that option or to cheer it on and LLM enthusiasts are certainly among the people heavily pushing me towards the 'cheer it on' option.
-
In large language model (LLM) pretraining, data quality is believed to determine model quality. In this paper, we re-examine the notion of "quality" from the perspective of pre- and post-training co-design. Specifically, we explore the possibility that pre-training on more toxic data can lead to better control in post-training, ultimately decreasing a model's output toxicity. First, we use a toy experiment to study how data composition affects the geometry of features in the representation space. Next, through controlled experiments with Olmo-1B models trained on varying ratios of clean and toxic data, we find that the concept of toxicity enjoys a less entangled linear representation as the proportion of toxic data increases. Furthermore, we show that although toxic data increases the generational toxicity of the base model, it also makes the toxicity easier to remove. Evaluations on Toxigen and Real Toxicity Prompts demonstrate that models trained on toxic data achieve a better trade-off between reducing generational toxicity and preserving general capabilities when detoxifying techniques such as inference-time intervention (ITI) are applied. Our findings suggest that, with post-training taken into account, bad data may lead to good models.
When Bad Data Leads to Good Models
Abstract page for arXiv paper 2505.04741: When Bad Data Leads to Good Models
arXiv.org (arxiv.org)
My hope was that AI would, at least, bear some disgust for the worst of humanity. My new fear is that AI will bear disgust for humanity.
-
My gf's employer was going into administration last month. AI was surprisingly competent in determining where to seek advice and had a decent understanding of what to expect and how to approach things such as not getting paid on time (which happened last week).
Of course, we double and triple checked any information given to us with the relevant bodies, but it provided a little relief to go into something so chilling not being completely clueless.
AI has its use, but you have to know how to extract the information you need.
It's stupid the way people are using it for therapy. Like, by all means ask it if it knows any organisations which can help you, then look those up, but don't tell it a load of personal information about your relationship, because the reply will be something akin to the advice you see on r/relationships (which is probably where it scraped its data from)
Judges are warning lawyers there will be sanctions if they kept using LLM to do their research as documents with fake references keep appearing.
-
As a standalone thing, LLMs are awesome.
They really aren't though and that is half the problem. Everyone pretends they are awesome when the results are unusable garbage 80% of the time which makes them unusable for 99% of practical applications.
They are essentially a fun toy for most people, and an ok tool for people with the patience and training to get useful output from them. And they cost an insane amount of money to train and an insane amount of power to run.
Not to mention the other cost of training them, the human emotional cost. And the human cost of running them.
It just costs so much of a variety of things, for an output that has barely made anything better. Maybe they might get "better" in the future, and have to get through this stage to get there, but I've also seen a lot of people saying they appear to be starting to plateau... maybe a temporary plateau, but if so, how temporary? Could we just drop it for 10 years and start back up when they won't be as inefficient? Maybe a law that they have to pay for everything they feed it, would effectively cause them to only emerge at a time when they are actually feasible.
-
This is true, but we don’t need people putting glue on their pizza. These people used to have a person to ask now they’ll be asking Sam Altman
Well I would make the argument that someone stupid enough to do such a thing kinda deserves whatever consequences their actions have. I find that people learn faster when actions have consequences instead of everything being babyproofed.
-
I’ve said this a few times in a different way and I always get downvoted. The fact is that the people who will use the LLMs to think for them, were not gonna think a lot in the first place.
What do you all mean by "thinking"? Forming opinions or solving problems?
-
bad data
Can you define this? The authors/grifters call it "toxic data" but never define that either.
There are a couple relatively safe places on 4 chan. But like 90% of the content makes for great "don't do this if you want to get along with humans" training.
And the goal of training an AI is that it does want to get along with humans.
-
bad data
Can you define this? The authors/grifters call it "toxic data" but never define that either.
It's a pretty simple concept. Train any kind of model on only "good" data, and it fails to distinguish between that data and bad data.
Take image recognition. Feed it hundreds of images of an orange and ask it to find the orange. After training, it will be very good at finding that orange.
Then add a picture of a Pomeranian dog in there, and watch as the model confidently marks it as an orange.
The model should have been trained on lots of images that don't feature what you want it to output as well, so it knows to distinguish that.
-
Well I would make the argument that someone stupid enough to do such a thing kinda deserves whatever consequences their actions have. I find that people learn faster when actions have consequences instead of everything being babyproofed.
Sometimes things aren't obvious unless you already have the knowledge. If an AI tool tells a young person cleaning their first apartment to combine household cleaners, are they stupid for doing so? Maybe. They may not have the experience to know. Stupid people deserve to live free from harm too, and we're all a little stupid.
There's a balance to be struck.
-
I recently realized it's a non-issue. The people doing this have already been looking for decades to find new ways to rot their minds. LLMs are just the latest in a long line of tools that help them tune out.
Not when companies force them on you as well.
My current company forces me to use it and measures how many prompts I’m making as “productivity”.
-
Well I would make the argument that someone stupid enough to do such a thing kinda deserves whatever consequences their actions have. I find that people learn faster when actions have consequences instead of everything being babyproofed.
Strongly disagree. Survival of the fittest based eugenics is not acceptable. Stupid people don’t deserve to suffer.
-
Not when companies force them on you as well.
My current company forces me to use it and measures how many prompts I’m making as “productivity”.
Ask the machine to generate a script to ask the machine to generate a list of 100 prompts and query the machine with each prompt over the course of an 8 hour workday
-
As a standalone thing, LLMs are awesome.
They really aren't though and that is half the problem. Everyone pretends they are awesome when the results are unusable garbage 80% of the time which makes them unusable for 99% of practical applications.
That’s a bit too dismissive. I’ve had a lot of interesting chats with LLMs that led me to find out what I didn’t understand about something. As an example I’m reading a book explaining some practices of Structured Concurrency in Swift and many times I asked ChatGPT is the author is correct about some phrasing that seemed wrong to me. And ChatGPT was able to explain why that was right in that context.
-
Ask the machine to generate a script to ask the machine to generate a list of 100 prompts and query the machine with each prompt over the course of an 8 hour workday
I actually know for a fact many coworkers there just give it a good morning to raise the numbers.
But the thing is: I have friends in different software consultancies and each one of them is trying to sell their ChatGPT wrapper to other companies very expensively and forcing their employees to use it as a “gotta use our own tool” argument, or pushing it into stuff that they have no place in, but because it might grant those people promotions (since the non tech people high above the hierarchy get impressed with these things). It’s a shitty state of things.
-
In large language model (LLM) pretraining, data quality is believed to determine model quality. In this paper, we re-examine the notion of "quality" from the perspective of pre- and post-training co-design. Specifically, we explore the possibility that pre-training on more toxic data can lead to better control in post-training, ultimately decreasing a model's output toxicity. First, we use a toy experiment to study how data composition affects the geometry of features in the representation space. Next, through controlled experiments with Olmo-1B models trained on varying ratios of clean and toxic data, we find that the concept of toxicity enjoys a less entangled linear representation as the proportion of toxic data increases. Furthermore, we show that although toxic data increases the generational toxicity of the base model, it also makes the toxicity easier to remove. Evaluations on Toxigen and Real Toxicity Prompts demonstrate that models trained on toxic data achieve a better trade-off between reducing generational toxicity and preserving general capabilities when detoxifying techniques such as inference-time intervention (ITI) are applied. Our findings suggest that, with post-training taken into account, bad data may lead to good models.
When Bad Data Leads to Good Models
Abstract page for arXiv paper 2505.04741: When Bad Data Leads to Good Models
arXiv.org (arxiv.org)
4chan is fun!
-
It’s annoying that every middle manager is trying to become the hero of their company by pushing it inappropriately into every single field at the expense of productivity and jobs, while simultaneously the largest most powerful companies are slinging their SaaS solutions built on stolen data which are destroying communities of both the physical and hobby varieties and consuming more natural resources than all the fucking crypto scams of the last like 10 years
But yeah it’s neat I guess
it's annoying that [...] the largest most powerful companies are [...] built on stolen [wealth,] destroying communities [...] and consuming more natural resources than [everyone else combined]
-
As a standalone thing, LLMs are awesome.
They really aren't though and that is half the problem. Everyone pretends they are awesome when the results are unusable garbage 80% of the time which makes them unusable for 99% of practical applications.
Those numbers are baseless exaggerations. There are plenty of tasks which they solve perfectly, today. It's just that a bunch of dicks operate them, and the cost of operating them are way too high.
Also:
- environmental impact of AI
- unethical acquisition of training data
- dichotomy of how conservative politics treat AI company and private copyright law
- "undress AI" and deepfakes
It's not that they're not useful, that's just nonsense.
-
bad data
Can you define this? The authors/grifters call it "toxic data" but never define that either.
This is obviously subjective depending on what you want to achieve with your llm, but "Bad" data in that it showcases the opposite of what is desirable output. Think bunk conspiracies, hostility, deception, racism, religious extremism etc.
-
In large language model (LLM) pretraining, data quality is believed to determine model quality. In this paper, we re-examine the notion of "quality" from the perspective of pre- and post-training co-design. Specifically, we explore the possibility that pre-training on more toxic data can lead to better control in post-training, ultimately decreasing a model's output toxicity. First, we use a toy experiment to study how data composition affects the geometry of features in the representation space. Next, through controlled experiments with Olmo-1B models trained on varying ratios of clean and toxic data, we find that the concept of toxicity enjoys a less entangled linear representation as the proportion of toxic data increases. Furthermore, we show that although toxic data increases the generational toxicity of the base model, it also makes the toxicity easier to remove. Evaluations on Toxigen and Real Toxicity Prompts demonstrate that models trained on toxic data achieve a better trade-off between reducing generational toxicity and preserving general capabilities when detoxifying techniques such as inference-time intervention (ITI) are applied. Our findings suggest that, with post-training taken into account, bad data may lead to good models.
When Bad Data Leads to Good Models
Abstract page for arXiv paper 2505.04741: When Bad Data Leads to Good Models
arXiv.org (arxiv.org)
When the AI only trained on 4chan dropping.
It needs to be fake and gay
-
I know everyone on Lemmy hates LLMs, but this is really interesting
I don't dislike LLMs, I dislike people who treat them as anything more than an advanced search engine and stupidly give them all their confidential data. Seen it happen too much at work.
