It’s not fine. They are not archiving the internet. I had to ban their user agent after very aggressive scraping that would have taken down our servers. Fuck this shitty behaviour.
It’s not fine. They are not archiving the internet.
I had to ban their user agent after very aggressive scraping that would have taken down our servers. Fuck this shitty behaviour.