Allowing scans through Cloudflare Technote TN-W22

Cloudflare’s bot protection may block OnDemand and SortSite Desktop scans before any pages are fetched. Scans of an affected site show 403 Forbidden errors (or a Cloudflare challenge page or a “Sorry, you have been blocked” page) instead of the usual results, often starting with the home page or robots.txt.

This technote describes the narrowest Cloudflare exception that allows scans through, without disabling bot protection for other visitors.

Identifying scan requests

IP addresses. Each OnDemand account scans from a fixed set of IP addresses, listed on your account page (see Technote TN-Q12). Scans are not routed through shared or rotating proxy networks. The list can change - we notify customers in advance when it does - so review your Cloudflare rule if scans start being blocked again.

User-Agent. Cloud scans send the embedded browser’s User-Agent string with the product name and version appended - the current strings are listed under Cloud User Agents on the user agents page, for example:

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/151.0.4129.78 Safari/537.36 Edg/151.0.4129.78 SortSiteCmd/2026.60.2064

Match on the SortSiteCmd/ token (sitemap scans use MapperCmd/) - the version numbers change every few weeks, so match the token, not the full string. The User-Agent alone should not be trusted - anyone can send it - so combine it with the IP addresses, which are the reliable signal.

Verifying a request came from us. Reverse DNS for each scanner IP resolves to a *.powermapper.com host name, and a forward lookup of that name returns the same IP (forward-confirmed reverse DNS). Your Cloudflare security event log shows the source IP, which you can check against your account’s list.

How scans behave

• Scans respect robots.txt
• A scan fetches one page at a time - page resources (images, CSS, scripts) load in parallel, as in a normal browser visit
• The scanner honors HTTP 429 responses and Retry-After headers

Scan traffic looks like one fast reader, not a flood - but it is faster than a human, so rate limiting rules tuned for human visitors may trigger.

Cloudflare configuration

The narrowest exception is a WAF custom rule with the Skip action, matching your scanner IPs (custom rules run before bot protection in Cloudflare’s evaluation pipeline):

1. In Cloudflare, go to Security > WAF > Custom rules and create a rule
2. Use an expression matching your account’s scanner IPs, optionally combined with the User-Agent token:

(ip.src in {203.0.113.10  203.0.113.11  203.0.113.12} and (http.user_agent contains "SortSiteCmd/" or http.user_agent contains "MapperCmd/"))

Replace the 203.0.113.x example IPs above with the list from your account page

3. Set the action to Skip, and select the protections that are blocking scans:

   • Super Bot Fight Mode
   • Rate limiting rules
   • Browser Integrity Check (under products to skip)
   • Managed rules - only if your security events show WAF managed rules blocking scans; otherwise leave them on
   • All remaining custom rules - only if your security events show one of your own custom rules blocking scans. On Enterprise plans with Bot Management this is the usual cause - bot score rules (using cf.bot_management.score) are custom rules, so the Skip rule must be placed above them in the custom rules list

Including the User-Agent condition means the exception only applies to requests that both come from your scanner IPs and identify as the scanner - the narrowest practical rule.

Free plan note: Bot Fight Mode on the Free plan runs outside Cloudflare’s rule engine and cannot be bypassed by custom rules or IP allowlists. The options are to upgrade to a plan with Super Bot Fight Mode (which honors Skip rules), or to turn Bot Fight Mode off while scans run.

Alternative: an IP Access rule with the Allow action for the scanner IPs also bypasses Super Bot Fight Mode, but it is broader than a Skip rule - it cannot be combined with a User-Agent condition or limited to specific protections.

Finding which rule blocked a scan

Every response from Cloudflare carries a Ray ID, which identifies the request in your Cloudflare security events:

1. Find the Ray ID for a blocked page. In SortSite Desktop, view the source of the blocked page and look for the cf-ray response header, e.g. cf-ray: 8f2a1b3c4d5e6f70-LHR. The Ray ID is the part before the dash. Cloudflare block pages also show the Ray ID at the bottom of the page.
2. In Cloudflare, go to Security > Events and filter by Ray ID
3. The event shows which protection blocked the request (the “Service” field), and the rule that fired - this tells you what the Skip rule needs to cover

If scans are still blocked after adding the rule, repeat these steps with a Ray ID from the new scan - another protection further down the pipeline may also need skipping. For OnDemand scans, our support team can cross-reference a scan ID against the Cloudflare event timestamps if needed.

SortSite Desktop

Desktop scans come from the public IP address of the computer running SortSite, not from the OnDemand scanner IPs. Use the same Skip rule, matching your own public IP address instead:

(ip.src eq 198.51.100.25)

Replace 198.51.100.25 with the public IP address of the computer running the scan. Cloudflare block pages show the visitor’s IP address next to the Ray ID.

Don’t include a User-Agent condition for desktop scans - the desktop user agent is configurable (see the user agents page), and the default Mac and Windows strings don’t contain the SortSiteCmd/ token.

Shopify stores behind Cloudflare

Some Shopify stores route their domain through their own Cloudflare account before it reaches Shopify. For these stores, Cloudflare blocks scans before Shopify sees the request, so Web Bot Auth signatures may have no effect - Cloudflare’s bot protection may not recognize Shopify’s signatures, and then blocks signed requests like any other automated traffic. Blocked responses show server: cloudflare with no powered-by: Shopify header.

Add the Skip rule above to let scans through Cloudflare, and keep the Shopify Web Bot Auth signatures in place - Shopify’s own bot protection still applies once requests reach Shopify.

Applies To: OnDemand, SortSite Desktop

Last Reviewed: October 9, 2026