[Skip navigation links]
Login

Search Engine Guidelines: Robots.txt Standard

Robots.txt is a standard that controls how search robots access websites. Mistakes in robots.txt files can prevent a site being indexed by search engines.

Rules