New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Pay per Crawl - Cloudflare
Nowadays AI crawlers scraping webpages left and right with no excuses some of the professional Crawls following robots.txt and most of them doesnt which results huge impact to the website owners earnings. To avoid this i see a new way that cloudflare initiated "Pay Per Crawl". Does anyone using this? Or even AI crawlers doge that and scrap without paying. If this continues content creators will not be thr. Then dead internet theory will be the real in future.
Comments
Dead internet is a bit over the top statement.
Content creator are not backbone of the internet. They are facilitators but not THAT much essential component in the system where absence of them will cease the internet.
Ya if thr no source of income then what motivates them to continue further
There was a time when the internet was used for the non-commercial distribution of knowledge and BTX, where everything was billed to the “content creator.”
People realized that those who were simply interested in imparting knowledge were much better sources than those who wanted to produce as much content as possible for as much money as possible. So everyone preferred to go to the Internet (which was technically far inferior at the time), and the media companies were left alone in BTX without any customers.
So the media corporations also switched to the Internet. And were initially welcomed there. Optional content prepared with a lot of work sounded tempting at first.
Suddenly, media corporations that were solely interested in making money encountered a community that was primarily interested in helpful cooperation. The corporations quickly started to push their interest in making money. Above all, with lawyers and lobbying for user-unfriendly laws. The former guests suddenly tried to make rules against the rest. Instead of how to generate the best possible result for everyone, the question always arose: What do we get in return?
Domains were confiscated, search results were changed, summaries were suppressed that gave users the answer in two sentences instead of forcing them to wade through pages of continuous text and advertising. And above all: the distribution of knowledge in all possible places was suppressed.
Tell me how the internet is not dead already.
if you publish mainstream propaganda, you should be fine even after 10 years. sheep will follow
Right… the internet is dead… again. 🙄
What is this, the 10th funeral already? People have been declaring the internet dead more often than Facebook declares your friends’ birthdays.
Let’s recap, shall we?
My website is robot friendly and i hate Cloudflare MtiM-ing 20% of the internet and slowing down visitors with dirty IPs by slow captchas
My understanding, how that works, is basically, by user agent.
For example, my OpenWebUI uses a specific user agent or has some specific pattern, that cloudflare catches.
However, if you use a fully transparent, chromium based proxy, it just works fine.
I think we might be better off if some majority % of "content creators" had to fuck off and stop polluting the internet.
Thanks.
There are people who volunteer their time to create content and write texts. Why does someone train young soccer players without pay, why does someone help the homeless, why do people create free software, etc.? The internet will change, but it will not die out.
And if all influencers die out I'm not crying
the web is not the internet.
How many CF captchas do you get per day? I get ~10 per day, and they don't pass on Falkon browser, only on Firefox, and only if Firefox is the only program running. If MPV or Falkon are also running while CF hammers the CPU on Firefox, the challenge page spins and reloads forever
But I let Firefox sit thru and solve the CF challenge only once a day to check a special webpage, otherwise I don't bother with it.
Sir, what is this BTX of which you speak?
(also, I can't help but read it as buttocks)
It is not that bad (20 per day), but it feels wrong