For a glorious second, the entire world was able to communicate as one.

Then we catalogued every accessible reservoir of culture and knowledge, mined them bare, and refilled them with slop.

A global collective consciousness, hollowed out, replaced with static. No signal. Only noise.

Nah, it’s saying that ad and AI-driven internet will prevail. People only use Google to find an answer and don’t dig deeper, and if they do, it’s often because the links are sponsored. People using GPT’s are even less likely to click a link. Currently no ads, but just wait.

Apologies if you were joking.

What's challenging about paywalls and not wanting to spend money is not necessarily not wanting to spend, but convenience and cost. If it costs me 10 cents for each blog or tutorial or github page I look at while working on a project, or 1 cent for every funny video, that adds up. And do I have to put my credit card in for every site? Hope that every site has good enough security to prevent payment information leaks?

And I don't think anyone is interested in a Netflix-style internet that fractures into 6 different subscriptions to get every site you need on the web.

This is part of the larger problem that AI tools are trained on (and profit off of) content that is produced and hosted by others who are now seeing their traffic change from humans to bots. For content sources that pay for hosting with ads, this means a loss in revenue to pay for hosting. For content sources like Wikipedia, they are seeing their hosting costs increase significantly due to the increase in bot traffic. Even if you want every website that depends on ad revenue to fail (which I don't entirety agree with), AI is still damaging the open web in other ways. Websites like Wikipedia for example may soon be forced to lock content behind logins or leverage aggressive captchas just to fight the bot traffic, which makes things worse for those of us that still prefer to use actual websites over AI summaries.

I really non ironically miss the friction of the old internet.

I prefer how it took time to find some bare HTML university website, slowly browse through an index as if it was a book, and then find one non-SEO optimized page with all the information you needed on a topic for your research.

The time to browse, being exposed to other terms, having to select the pages yourself, being skeptical by nature, and then having to copy it by hand... This is a much more positive scenario than having a gigantic company learn everything about you and everybody else and then make these decisions for you, using some hidden algorithm, and with the ultimate goal of pushing their newest process. And of course, the content has been rendered virtually useless to appeal to that algorithm.

Nobody is scraping wikipedia over and over to create datasets for AIs, there are already open datasets and API deals. But wiki in particular has always had a data dump of the entire db bimonthly.

Wikimedia Downloads

(dumps.wikimedia.org)

Sorry for beginner reaction, can I use this in a website for an open source XHTML-extension I am developing? do I need to credit you somehow or lemmy link is enough or what is the best practice here?

You clearly haven't run a website recently. Until I set up anubis last week I was getting constant requests from dozens of various bot scrapers 24/7. That included the big ones.

Yeah well maybe the web shouldn't be a business

I really non ironically miss the friction of the old internet.

I prefer how it took time to find some bare HTML university website, slowly browse through an index as if it was a book, and then find one non-SEO optimized page with all the information you needed on a topic for your research.

The time to browse, being exposed to other terms, having to select the pages yourself, being skeptical by nature, and then having to copy it by hand... This is a much more positive scenario than having a gigantic company learn everything about you and everybody else and then make these decisions for you, using some hidden algorithm, and with the ultimate goal of pushing their newest process. And of course, the content has been rendered virtually useless to appeal to that algorithm.

when the internet was a wild and unexplored frontier, and we were adventurers charting the unknown.

god what I wouldnt give to go back to the days of the mid 90s, when the internet was nothing more than a collection of tech weirdos, with websites being nothing more than passion projects with no advertising, no SEO, no search engines, etc etc.

god what I wouldnt give to go back to the days of the mid 90s, when the internet was nothing more than a collection of tech weirdos, with websites being nothing more than passion projects with no advertising, no SEO, no search engines, etc etc.

So you’re saying the ad driven internet will die? And we will be left with what? Wikipedia and Lemmy? I for one welcome our AI overlords!

Can't we go back? What's stopping you?

Kay, and that has nothing to do with what i said. Scrapers, bots =/= AI. It's not even the same companies that make the unfree datasets. The scrapers and bots that hit your website are not some random "AI" feeding on data lol. This is what some models are trained on, it's already free so it's doesn't need to be individually rescraped and it's mostly garbage quality data: https://commoncrawl.org/ Nobody wastes resources rescraping all this SEO infested dump.

Your issue has everything to do with SEO than anything else. Btw before you diss common crawl, it's used in research quite a lot so it's not some evil thing that threatens people's websites. Add robots.txt maybe.