Robots.txt Tester
Read a site's robots.txt, see the rules that apply to each crawler, and test whether a specific URL would be allowed or blocked — the file where an accidental line can remove a whole site from search.
Open Robots.txt TesterWhere to find it
Every address below opens the same tool. The primary address is the one to bookmark and share.
- robots.openvibe.toolsPrimary
People also call this
- robots.txt tester
- robots txt checker
- crawl rules
- disallow checker
- seo crawl test
Related tools
All Network Tools
Domain & IP LookupLook up everything about a domain or IP in one go: DNSlookup.openvibe.tools
What Is My IPSee your public IP address, ISP, approximate location andmyip.openvibe.tools
IP Address LookupLook up any IP address to see its owner, network,ip.openvibe.tools
IPv4 Lookup & CIDR CalculatorLook up IPv4 addresses and calculate CIDR subnets: networkipv4.openvibe.tools
IPv6 LookupLook up IPv6 addresses, expand or compress notation, andipv6.openvibe.tools
IP GeolocationFind the country, region, city, timezone and coordinatesgeoip.openvibe.tools
Hostname LookupResolve a hostname to its IP addresses or find the hostnamehostname.openvibe.tools
ISP LookupIdentify the ISP, hosting company or organisation that ownsisp.openvibe.tools
The rest of OpenVibe
One account works on every site. Open source and community-run.