Why The Crawl Tool?

Why use The Crawl Tool? The Crawl Tool is the smartest SEO crawler, helping you find SEO and user experience issues.

Why The Crawl Tool?

A genuine question you may have is "Why The Crawl Tool"? The crawl tool is an innovative tool to help find issues on your website, improving your on-site SEO and user experience, and helping you manage those issues. But what makes the crawl tool a such useful tool and how does it differ from it's competitors?

The SEO Spider Market is Broken

The Crawl Tool sits in the category of SEO spiders/crawlers. It started out by looking at what is happening in that market and to generate a product that is better for our users. The unescapable conclusion is that the market is broken. There are two categories of competitors.

Installed Software

You can rent software to install on your computer. The software is based on old technologies and uses your own resources, but you pay yearly for it. It provides a vast amount of data to try to justify the price, but 99% of that data you will never use and it just makes it confusing and cluttered. Because your are using your own resources for the crawl your IP gives you away if you do an analysis of a competitor site and they may feed you false information. Less established businesses may also slow their internet connection down and sometimes break their ISPs terms of service. As your team gets larger there's another problem that surfaces - it gets difficult to share data without more licenses. If you want to do things like schedule crawls then you need a masters degree in IT. Updates? Run them yourself. Whilst a crawler as an installable software item seems like a good idea at first (our platform started with that as a concept but scrapped it), in practice it's not so practical. You can see this for yourself, take their software, apply it to their own sites and see if they use it themselves in practice. You'll be surprised! If they don't see value in their own products and use them themselves, why should customers pay for it?

Online Software

There are online sites where you'd need to mortgage your grandmother to pay their monthly charges. That may be okay for the big boys, but what about the rest of us? What about the average company looking to improve their website or the individual SEO or small agency? These sites are generally resellers of data gathered elsewhere, with their own on top. Like installed software there is a huge amount of un-necessary data to try to justify the cost. However, they at least try to simplify the presentation of this by combining it into proprietary scoring systems. However, this simplification also throws out a lot of useful information. What does having a 7/10 on-site SEO score even mean?! These sites may argue over who has the fastest crawler, etc, but not tell you that having the fastest crawler is only an advantage for the most busy websites and a disadvantage for everyone else.

It's not that these tools are bad, but each really only fulfills the needs of a tiny amount of websites, SEOs, and agencies. The majority are forced to pick an option that doesn't fit their needs.

The Three Principles of "The Crawl Tool"

In designing the crawl tool, we took a look at the state of the market and designed some principles to ensure that our product fit market needs. For every decision, including every design and feature design, we ask the question "does this fit and fulfill the principles". In that way we can ensure that The Crawl Tool is not about overwhelming you with data you won't use or vaguely defined meaningless statistics but real, actionable, data you can use. Our three principles are:

  • Simplicity - that's not to say that it's missing things, but that we strive to present what is important in an easy and actionable way. Anybody can use it - whether that's yourself, a tech team, or your work experience intern.

  • Low Cost - we charge for what you use and we keep our costs low so we can charge less. We want to make things accessible to small business and starters too.

  • Ease of Management - presenting you with data is great, but we want to make it easy for you to manage fixing any problems.

By applying these three, customer focused, principles to our product each and every time, we will fix the broken SEO crawler market.

The Tool

On these principles, The Crawl Tool is a web based tool. You register, and it's immediately ready for you to use. Add a site as a project, click Crawl and it fire up a crawler to crawl it and send you an email to you when it's done. We handle the specifics of the crawl behind the scenes for you.

When the crawl is done, the data is ready for you. We present it as some graphs and tables. We don't make up meaningless pretend scores! You can use our interface but we don't hold your data hostage, you can export it to excel or Google Sheets if you want.

You can set deadlines for yourself to fix issues. Or if you have a team - deadlines for others. As well as assign statuses, and add notes.

You pay us not for work we did over a decade ago, or reformatting mostly public data, but inline with your resource usage. If your usage is tiny, then so is the cost. It's only fair.

It's the tool 99% of the market needs. Check out The Crawl Tool with 1000 free credits when you register. We can't wait to help improve your website.