Robots.txt Tester
Test if URLs are allowed or disallowed in a website's robots.txt file.
Robots.txt Checker
Check if a URL is disallowed in a website's robots.txt file.
Robots.txt files are cached for 60 minutes. Use the flush setting to force an update.
Enter a URL and select token groups or custom tokens to check if they're disallowed in the robots.txt.