About PickedFirstBot

PickedFirstBot is the crawler behind PickedFirst. It reads public "best of" articles when a user asks us to check them, and re-reads them about once a week to see if the list changed.

• It identifies itself with the user agent PickedFirstBot/1.0 and a link to this page.
• It follows robots.txt. To block it, add User-agent: PickedFirstBot and Disallow: /.
• It waits several seconds between requests to the same site and caps requests per minute.
• It checks whether links in a list still work with a single light request per link.
• It does not log in, fill in forms, or read private pages.

Questions or problems: pickedfirst.team@gmail.com.