Archive Team - Robots.txt is a suicide note

ROBOTS.TXT IS A SUICIDE NOTE

ROBOTS.TXT is a stupid, silly idea in the modern era. Archive Team entirely ignores it and with precisely one exception, everyone else should too.

Archive Team interprets ROBOTS.TXT as damage and temporary madness, and works around it. Everyone should. If you don’t want people to have your data, don’t put it online.