• HTML.
• PDF.

In large language model (LLM) pretraining, data quality is believed to determine model quality. In this paper, we re-examine the notion of "quality" from the perspective of pre- and post-training co-design. Specifically, we explore the possibility that pre-training on more toxic data can lead to better control in post-training, ultimately decreasing a model's output toxicity. First, we use a toy experiment to study how data composition affects the geometry of features in the representation space. Next, through controlled experiments with Olmo-1B models trained on varying ratios of clean and toxic data, we find that the concept of toxicity enjoys a less entangled linear representation as the proportion of toxic data increases. Furthermore, we show that although toxic data increases the generational toxicity of the base model, it also makes the toxicity easier to remove. Evaluations on Toxigen and Real Toxicity Prompts demonstrate that models trained on toxic data achieve a better trade-off between reducing generational toxicity and preserving general capabilities when detoxifying techniques such as inference-time intervention (ITI) are applied. Our findings suggest that, with post-training taken into account, bad data may lead to good models.

When Bad Data Leads to Good Models

Abstract page for arXiv paper 2505.04741: When Bad Data Leads to Good Models

arXiv.org (arxiv.org)

It’s annoying that every middle manager is trying to become the hero of their company by pushing it inappropriately into every single field at the expense of productivity and jobs, while simultaneously the largest most powerful companies are slinging their SaaS solutions built on stolen data which are destroying communities of both the physical and hobby varieties and consuming more natural resources than all the fucking crypto scams of the last like 10 years

But yeah it’s neat I guess

As a standalone thing, LLMs are awesome.

They really aren't though and that is half the problem. Everyone pretends they are awesome when the results are unusable garbage 80% of the time which makes them unusable for 99% of practical applications.

bad data

Can you define this? The authors/grifters call it "toxic data" but never define that either.

• HTML.
• PDF.

In large language model (LLM) pretraining, data quality is believed to determine model quality. In this paper, we re-examine the notion of "quality" from the perspective of pre- and post-training co-design. Specifically, we explore the possibility that pre-training on more toxic data can lead to better control in post-training, ultimately decreasing a model's output toxicity. First, we use a toy experiment to study how data composition affects the geometry of features in the representation space. Next, through controlled experiments with Olmo-1B models trained on varying ratios of clean and toxic data, we find that the concept of toxicity enjoys a less entangled linear representation as the proportion of toxic data increases. Furthermore, we show that although toxic data increases the generational toxicity of the base model, it also makes the toxicity easier to remove. Evaluations on Toxigen and Real Toxicity Prompts demonstrate that models trained on toxic data achieve a better trade-off between reducing generational toxicity and preserving general capabilities when detoxifying techniques such as inference-time intervention (ITI) are applied. Our findings suggest that, with post-training taken into account, bad data may lead to good models.

When Bad Data Leads to Good Models

Abstract page for arXiv paper 2505.04741: When Bad Data Leads to Good Models

arXiv.org (arxiv.org)

I know everyone on Lemmy hates LLMs, but this is really interesting

When the AI only trained on 4chan dropping.

It needs to be fake and gay

Those numbers are baseless exaggerations. There are plenty of tasks which they solve perfectly, today. It's just that a bunch of dicks operate them, and the cost of operating them are way too high.

Also:

• environmental impact of AI
• unethical acquisition of training data
• dichotomy of how conservative politics treat AI company and private copyright law
• "undress AI" and deepfakes

It's not that they're not useful, that's just nonsense.

When the AI only trained on 4chan dropping.

It needs to be fake and gay

• HTML.
• PDF.

In large language model (LLM) pretraining, data quality is believed to determine model quality. In this paper, we re-examine the notion of "quality" from the perspective of pre- and post-training co-design. Specifically, we explore the possibility that pre-training on more toxic data can lead to better control in post-training, ultimately decreasing a model's output toxicity. First, we use a toy experiment to study how data composition affects the geometry of features in the representation space. Next, through controlled experiments with Olmo-1B models trained on varying ratios of clean and toxic data, we find that the concept of toxicity enjoys a less entangled linear representation as the proportion of toxic data increases. Furthermore, we show that although toxic data increases the generational toxicity of the base model, it also makes the toxicity easier to remove. Evaluations on Toxigen and Real Toxicity Prompts demonstrate that models trained on toxic data achieve a better trade-off between reducing generational toxicity and preserving general capabilities when detoxifying techniques such as inference-time intervention (ITI) are applied. Our findings suggest that, with post-training taken into account, bad data may lead to good models.

When Bad Data Leads to Good Models

Abstract page for arXiv paper 2505.04741: When Bad Data Leads to Good Models

arXiv.org (arxiv.org)

That exists, its called GPT4chan, and it went exactly like you'd expect.

Did it at least come up with a cool story about managing a bottomless pit?

10% 4chan

why didn't they just say 0.4chan and be done with it?

Did it at least come up with a cool story about managing a bottomless pit?

There's a "your mom" joke here but I'm not going to make it because you don't deserve that.

There are plenty of tasks which they solve perfectly, today.

Name a single task you would trust an LLM on solving for you that you feel confident would be correct without checking the output. Because that is my definition of perfectly and AI falls very, very far short of that.

I am not sure if you and @General_Effort got the reference I was making, so I just wanna share it for everyone else who might not have seen it yet because it's great:

Who says you can't check their outputs?
It's much faster to e. g. read a generated text than to write everything yourself. Same applies to translations, they've been excellent for quite a while now.

Business communication can be handled effortlessly by AI. Of course you read the result before you send it out, but that takes an order of a magnitude less time than formulating and typing all those meaningless sentences.

And honestly, that's a perfect use case for AI. I wouldn't compose a love letter to my family using AI, but a pamphlet, feature description, sales pitch, any bullshit presentation deck? You bet AI excels at those.

Same applies to content summaries that help augment search indices. Finding a large number of content candidates (e. g. videos) and have AI summarize the contents of said videos to narrow down the search is helpful and works today.

I'm not looking for AGI. I'm looking for tools to make my life easier, but in an ethical manner that doesn't advance the destruction of the planet at an exponential rate, just for some tech bro to jerk it and buy another yacht.

I know everyone on Lemmy hates LLMs, but this is really interesting

It's a pretty simple concept. Train any kind of model on only "good" data, and it fails to distinguish between that data and bad data.

Take image recognition. Feed it hundreds of images of an orange and ask it to find the orange. After training, it will be very good at finding that orange.

Then add a picture of a Pomeranian dog in there, and watch as the model confidently marks it as an orange.

The model should have been trained on lots of images that don't feature what you want it to output as well, so it knows to distinguish that.