Guide To Using The Crawl Tool

Our goal with The Crawl Tool is to make it effortless and easy to use. But you may still wish to have some instructions. If so, this section of the website is for you.

Taking your first steps with The Crawl Tool? These pages are probably of most interest to you.

• How To Start Your First Crawl

The Crawl Tool can provide a variety of useful reports. With the exception of the dashboard, these can be accessed from the report dropdown at the top of the project pages. Each link below describes the report.

• The Project Dashboard

• The Broken Links Report

• The Crawl Log

• The Insecure Content Report

• The Internal Links Report

• The Meta Descriptions Report

• The Meta Keywords Report

• The Meta Open Graph Report

• The Meta Twitter Report

• The Missing Alt Tags Report

• The Offsite Links Report (with AI)

• The Page Links Overview Report

• The Pages Linking Redirects Report

• The Redirects Report

• The Titles Report

• The Offsite JS Scripts Report

• The Offsite JS By Page Report

• Header Tags Report

• The Linking Domains Report

• The Cookies Report

• The Site Information Report

• The Theme Report (AI)

• Choose the columns that display in a report

• Schedule Crawls

• Edit Project

• Delete Project

One major difference with The Crawl Tool is that we have a lot of functionality to help manage fixing issues with a site.

• Making Access Keys for Third Parties

• Assigning Status and Assignee

• Setting Due Dates and Notes

• Bulk Assignment

The Crawl Tool provides a number of ways to filter your data within the system, or to export the data as Excel or Google Sheets.

• Filters

• Applying Robots.txt to filter data

• Exporting Data

For programmatic data access, use our public API.

• Public API