1·
25 days agoBut isn’t this how ordinary search crawlers and tools like wget also work? I’ve never heard of them causing these same problems.
IndigoGolem
Cook, potter, inventor, writer, neographer, conlanger, phantasocartographer, coder, linguist, poet, blogger, chef, webmaster, speedrunner, herald, translator, songwriter, ergonomicist, pilot, miner, outrageous liar, gardener.
- 2 Posts
- 2 Comments
Joined 3 years ago
Cake day: October 9th, 2023
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
So they’re just checking as fast as possible to get every update as fast as possible?
This still sounds like idiot design, scraping hard enough to take down sites. It’s like cutting open the goose that lays lead eggs in the hope that you can get more lead and convince people that lead is better than gold.