What’s nuts is that what made Anubis’ author go down that path was Amazon Bot (I remember precisely because they are the bot that also blew up my logs and thus forced me to take action against LLM scrappers) and… a significant share of the Web is hosted on AWS. So… Amazon is actually probably MAKING money by scrapping, no matter how inefficiently. I already hated Amazon but this is even worst than I imagined. It’s probably not by design, to be fair, but it’s also probably not something they’ll invest into “fixing” as it’s making them money. What an absolute human centipede situation.