Cloudflare wants Google to change its AI search crawling. Google likely won’t.
Technology
24
Beiträge
16
Kommentatoren
6
Aufrufe
-
Cloudflare's customers probably wouldn't be on board with that. Google's properties provide a tonne of traffic to businesses. Doing anything to put that in jeopardy would probably have many of Cloudflare's customers looking for a new provider.
Google used to provide a ton of traffic, they hoard it all themselves now through AI and summaries of content. Eventually the balance of cost/benefit will shift and Google will suddenly see itself rejected from scraping, furthering the product deathspiral.
-
This post did not contain any content.
Cloudflare wants Google to change its AI search crawling. Google likely won’t.
Cloudflare pushes Google to separate bots for AI Overviews and search indexing.
Ars Technica (arstechnica.com)
Totally understandable.
If scanning to help send traffic to your website, that's cool.
If scanning to generate summaries that won't send any traffic your way. No bueno.Ultimately, it should be whatever most benefits users.
-
Totally understandable.
If scanning to help send traffic to your website, that's cool.
If scanning to generate summaries that won't send any traffic your way. No bueno.Ultimately, it should be whatever most benefits users.
but there also needs to be incentive for sites to host content. if it all gets hijacked by search engines that isn't sustainable.
-
This post did not contain any content.
Cloudflare wants Google to change its AI search crawling. Google likely won’t.
Cloudflare pushes Google to separate bots for AI Overviews and search indexing.
Ars Technica (arstechnica.com)
Google has existed long enough now. The are allowed to disappear.
-
Totally understandable.
If scanning to help send traffic to your website, that's cool.
If scanning to generate summaries that won't send any traffic your way. No bueno.Ultimately, it should be whatever most benefits users.
No, things should not benefit users, they should benefit the creator of the original content.
-
This post did not contain any content.
Cloudflare wants Google to change its AI search crawling. Google likely won’t.
Cloudflare pushes Google to separate bots for AI Overviews and search indexing.
Ars Technica (arstechnica.com)
If I had to choose my favourite corporation, it would be Cloudflare. They at least do something good.
-
Google used to provide a ton of traffic, they hoard it all themselves now through AI and summaries of content. Eventually the balance of cost/benefit will shift and Google will suddenly see itself rejected from scraping, furthering the product deathspiral.
content is only 1 category of website
ecommerce drives all the advertising that funds content… it’s a much bigger market, and they don’t care about content scraping as long as you buy their product
-
content is only 1 category of website
ecommerce drives all the advertising that funds content… it’s a much bigger market, and they don’t care about content scraping as long as you buy their product
And the long term plan there is to strangle sites and take %100 of the adrev spend for themselves since users won't ever leave the Google site. Either way Google as a search engine enters a death spiral, it's already bleeding users.
-
If I had to choose my favourite corporation, it would be Cloudflare. They at least do something good.
There seems to be a line, so far as I can tell. If everything you need sits on the free tier, they're really good (well tbh their R2 storage is reasonably priced too). But once you stray into needing a paid tier, it apparently (I'm not there) quickly gets expensive as you're lured into every higher tiers.
But yes, in general I don't mind cloudflare so much and do use their free (and R2 paid) services.
-
Alternatively, you use the cloudflare money to sue the monopoly to decouple search and all other products, since blocking the AI trawlers shouldn't have any measurable impact in search rankings
I agree, but I think that hurting the companies bottom line is more effective than waiting on an archaic court system to do something. Just look at how slow the /current/ monopoly case on google is going.
-
I mean, with a company as large as cloudflare. I think they could /easily/ strong-arm this move by making blocking google crawlers a default setting on websites. The amount of traffic drop alone from that would make google think twice about the whole ordeal. And people who care about the google search indexers can turn them on again which will allow indexing again. but a default block would cause a lot of disruption google side and many people I don't think would go in and fix the setting till later on down the road.
I disagree. Searches are not googles main focus these days. Blocking their crawlers will just make the AI searches better - exactly what google want
-
I mean, with a company as large as cloudflare. I think they could /easily/ strong-arm this move by making blocking google crawlers a default setting on websites. The amount of traffic drop alone from that would make google think twice about the whole ordeal. And people who care about the google search indexers can turn them on again which will allow indexing again. but a default block would cause a lot of disruption google side and many people I don't think would go in and fix the setting till later on down the road.
Exactly. It's not like google is the only indexer out there. And if this cuts into their search dominance, so much the better.
-
There seems to be a line, so far as I can tell. If everything you need sits on the free tier, they're really good (well tbh their R2 storage is reasonably priced too). But once you stray into needing a paid tier, it apparently (I'm not there) quickly gets expensive as you're lured into every higher tiers.
But yes, in general I don't mind cloudflare so much and do use their free (and R2 paid) services.
They have not given me any reason to hate them which is a win in my books. Apple was my favourite but their policy of region specific features is getting annoying.
-
If I had to choose my favourite corporation, it would be Cloudflare. They at least do something good.
You may want to read: https://robindev.substack.com/p/cloudflare-took-down-our-website
-
It would be fine if it was just "lured", but this made me very sceptical of cloudflare: https://robindev.substack.com/p/cloudflare-took-down-our-website
-
You may want to read: https://robindev.substack.com/p/cloudflare-took-down-our-website
I still think they are good. Isolated incidents like this are going to happen when you are doing business at such scales.
-
I still think they are good. Isolated incidents like this are going to happen when you are doing business at such scales.
This did not sound like an isolated incident at all. You don't get sales responding to a legal/engineering issue by accident. It may have been accidental, if they put too much pressure on sales not realizing how it was corrupting the company, or the leadership may have tacitly approved of this. Hard to tell.
