Post Azprp94ZJDUwTSKAee by MishaVelthuis@social.edu.nl
(DIR) More posts by MishaVelthuis@social.edu.nl
(DIR) Post #AzZc7ajIPrUeXLdS2S by Codeberg@social.anoxinon.de
2025-10-25T16:06:41Z
3 likes, 3 repeats
We apologize for the long performance degradation today.Finally, we identified all of the 'tricks' that AI crawlers found today. They no longer bypass the anubis proof of work challenges.A novelty for us was that AI crawlers seem to not only crawl URLs that are actually presented to them by our frontend, but they converted the URLs into a format that bypassed our filter rules.By the way, you can track the changes we have been doing viahttps://codeberg.org/Codeberg-Infrastructure/scripted-configuration/compare/51618~1..e4aac
(DIR) Post #AzZmc6oofoTgHwPM80 by mms@mastodon.bsd.cafe
2025-10-25T18:32:03Z
1 likes, 0 repeats
@Codeberg I think your langue here is way to soft. It’s not crawlers what found it, but some company actively worked on breaking the mechanisms. It’s nothing else than malicious cracking. Behind every such incident is someone who made the decision to crack you.
(DIR) Post #AzZmeqlqnN7HNyKO6i by clementd@framapiaf.org
2025-10-25T16:43:59Z
0 likes, 1 repeats
@Codeberg AI is malware
(DIR) Post #Aza0blWPFmnbayWkYC by claudius@darmstadt.social
2025-10-25T21:11:27Z
2 likes, 4 repeats
@Codeberg AI companies crawl our websites.We ask that they stop by using the industry standard robots.txtAI companies ignore those rules.We start blocking the companies themselves with conventional tools like IP rules.AI companies start working around those blocks.We invent ways to specifically make life harder for their crawlers (stuff like Anubis).AI companies put considerable resources into circumventing that, too.This industry seriously needs to implode. Fast.
(DIR) Post #Aza0bn0vhmoEDvodlY by claudius@darmstadt.social
2025-10-25T21:13:45Z
0 likes, 0 repeats
As a next step, AI companies are now offering "their" browser (read: Chromium ever so slightly themed with some company bullshit built in)In part, this is certainly done to have yet another way to crawl the web, but this time user-directed and indistinguishable from actual human requests.
(DIR) Post #Aza0bnb5XLjU24jW7s by jeff@mk.magicka.org
2025-10-25T21:17:17.834Z
1 likes, 0 repeats
@claudius@darmstadt.social we need crawler honeytrap that sends back random racist 4chan posts as the response.
(DIR) Post #AzbaZRiNoNOaDzR7po by jae@darkdork.dev
2025-10-26T15:35:02.052816Z
0 likes, 0 repeats
@Codeberg going a bit higher, i really like where you're taking things with a scripted configuration. i was buried into ansible for years, but found a scripted approach to infrastructure worked better for me. to note, also was on k3s/k8s/nomad/harvester and the operational burden was very high for my private systems (microscopic scale compared to codeberg, but still burdensome) also your caddyfile is likely one of the cleanest i've seen.
(DIR) Post #Azprp94ZJDUwTSKAee by MishaVelthuis@social.edu.nl
2025-11-02T09:13:47Z
0 likes, 0 repeats
@jeff @claudius so that the models become more racist? 🤔
(DIR) Post #AzprpAKYdFskNKoSDQ by jeff@mk.magicka.org
2025-11-02T12:54:17.040Z
0 likes, 0 repeats
@MishaVelthuis@social.edu.nl @claudius@darmstadt.social you think they'd release the models if they were racist?
(DIR) Post #AzqNWsJQ5AMzoTbQnI by older@mstdn.social
2025-11-02T18:43:08Z
0 likes, 0 repeats
@jeff Someone has released MechaHitler.@claudius @MishaVelthuis
(DIR) Post #AzqNWt8T1NCeMnUCMS by jeff@mk.magicka.org
2025-11-02T18:49:32.555Z
0 likes, 0 repeats
@older@mstdn.social @claudius@darmstadt.social @MishaVelthuis@social.edu.nl that was a configuration mishap and to be fair, a model based on twitter literally can't not be racist. i don't think they give out the weights to anyone either.