TN-M03 The map/report doesn't contain all the pages I expected

There are several possible causes:

The crawler only visits pages on the same domain as the home page, so pages on a different domain do not appear on the map. To add additional domains to visit, select the Options command from the View menu, and add the additional domain names to the Servers tab.
The only navigation mechanism on your site is a site map or an A-Z index - the crawler ignores links from maps or A-Z indices because these are usually duplicates of some other links on the site (and produce a very flat map).
Some pages were blocked by the Robot Exclusion Standard (ROBOTS.TXT) or explicitly blocked in the Blocks tab on the Mapping Options window. To find out which links are blocked for a site, http://www.altavista.com for example, open the link http://www.altavista.com/robots.txt. If you get a Not Found message then no links are blocked, if you get an Unauthorized or Forbidden message then all links are blocked and if you get a text file back this will list which links are blocked. You can ignore the Robot Exclusion Standard by selecting the Options command from the View menu and unchecking Obey Robots.txt

Applies To: PowerMapper 3.0 and SortSite 3.0 or later

Last reviewed: 4 May 2009