[HN Gopher] Web-scraping AI bots cause disruption for scientific...
___________________________________________________________________
Web-scraping AI bots cause disruption for scientific databases and
journals
Author : tchalla
Score : 17 points
Date : 2025-06-10 20:25 UTC (2 hours ago)
(HTM) web link (www.nature.com)
(TXT) w3m dump (www.nature.com)
| OutOfHere wrote:
| Requiring PoW (proof-of-work) could take over for simple
| requests, rejecting requests until a sufficient nonce is included
| in the request. Unfortunately, this collective PoW could burden
| power grids even more, wasting energy+money+computation for
| transmission. Such is life. It would be a lot better to just
| upgrade the servers, but that's never going to be sufficient.
| Bjartr wrote:
| So, Anubis?
|
| https://anubis.techaro.lol/
| OutOfHere wrote:
| Yes, although the concept is simple enough in principle that
| a homegrown solution also works.
| Zardoz84 wrote:
| We are wasting power on feeding statistics parrots, and we need
| to waste additional power to avoid being DoS by that feeding.
|
| We will be better without that useless waste of power.
| treyd wrote:
| What do you suppose _we_ as website owners do to prevent our
| websites from being DoSed in the meantime? And how do you
| suppose we convince /beg the corporations running AI scraping
| bots to be better users of the web?
| atonse wrote:
| How was this not a problem before with search engine crawlers?
|
| Is this more of an issue with having 500 crawlers rather than any
| single one behaving badly?
___________________________________________________________________
(page generated 2025-06-10 23:01 UTC)