Report is not running or pages are missing
What are the possible causes of a Report that does not run or runs incompletely?
In this article, you will learn what can cause a Report to not run or run incompletely, and how to narrow down the cause.
What does your Report look like…?
This table gives you an overview of faulty Report results and their probable causes:
| Result | Probable cause |
|---|---|
| No pages, scan aborted | The start page could not be loaded: DNS, TLS, a hard block or an incorrect domain |
| Only a few pages | A block after the first request, a restricted URL list or no links found |
| Many pages with identical errors | The login did not work; the logged-out page was checked each time |
| Cookie banner content everywhere | The banner was not closed and overlays every page |
| Individual pages or sections are missing | robots rules, timeouts or a redirect |
More detailed cause descriptions
Below you will find further possible causes of a Report that does not run or runs incompletely, and what you can do in each case to fix the problem.
1. Bot protection, firewall and WAF
Typical signs: HTTP 403 or 429, a challenge page from Cloudflare, Akamai, Imperva or DataDome instead of your content, or the first page works and all further ones do not (rate limiting). The page is accessible from your own browser, but not from our data centre.
Solution: allow our IP addresses or our fixed user agent.
The following article explains which IP addresses we use:
2. Password-protected areas (.htaccess, Basic Auth)
Solution: access credentials can be stored in the licence; alternatively, allowing our IP address is sufficient.
You can find more information in the following article:
3. Login areas and Page Actions
If all pages in the Report are identical, the login was not completed. The crawler cannot bypass two-factor authentication.
You can find more information in the following article:
4. robots rules
Areas with noindex or nofollow are deliberately not checked when the corresponding setting is active.
You can find more information in the following article:
5. Cookie banner
Our crawler rejects banners automatically, but it does not always recognise unusual labels, banners in iframes or banners that appear with a delay. In that case, you can store a suitable Page Action.
You can find more information in the following articles:
6. Redirects
A redirect to a language or country selection can trap the scan on a single page. In that case, specific start URLs help.
7. Proxy (on-premise installations)
If the Report runs in your own network, the proxy must be configured so that the crawler can reach the domain to be checked.
8. Timeouts and very large domains
Very slow individual pages run into a time limit, and very large domains do not fit within a scan window. In this case, we narrow down the scan together with you.
9. Pages that respond with 401 or 403
Some systems respond on protected pages with the status code 401 or 403. By default, the crawl skips such pages, so they are then missing from the Report.
In the Report settings under Advanced, you can enable the Allow all status codes option so that these pages are checked anyway.
Bear in mind: what is checked is what the server actually delivers. If an error or login page is returned instead of the page, its accessibility is included in the assessment, not that of the protected content. This option is therefore only useful if your system delivers genuine content under 401 or 403. If you want the protected area itself to be checked, logging in via Page Actions or stored access credentials is the right approach.
Further support
If you have further questions or need personal support, feel free to contact us. The following article lists all the ways you can get in touch with us:
What our support team needs from you
- The affected domain and the time of the scan
- Whether the page is normally accessible from your own network
- If available: the log entry from your firewall or WAF at the time of the scan
- Whether a login is required and whether access credentials can be provided for it