Keeping a tightly themed website can help. There are several ways to detect the theme of a web site, one of them is using the links.

Having a clear theme to a website can help both users and search rankings. One way to determine the theme of a website is by its linking. If it links to or from other websites or pages about a particular theme, then it is more likely the website itself is about that theme.

The effect is not one sided though. In terms of links we can think of this as like putting a rubber band between the two sites or pages. It pulls sides closer together to a common theme. Of course, all the other links and all the other links on the linked site also have rubber bands pulling them to other things - so the effect of that single rubber band on each site can vary.

In this process you're not completely out of control though. You choose who you link to! This is why we introduced site relevance and page relevance into the "Offsite Links" report.

When The Crawl Tool crawls your website it uses a modern AI neural network to create a representation of the theme of each page on the site and each page it links to. This representation is known as an embedding or vector (as it takes the form of a multi-dimensional vector). It also calculates one of these vectors for the entire website you're crawling.

These representations of theme have an interesting property in that themes that are similar also have similar representations. By using a method known as cosine similarity, we can calculate how close each representation is. A score of 1 would indicate that they are the same, but this will rarely occur except in the cases of duplicates.

As mentioned previously, we use this to calculate two figures for offsite links.

Site Relevance - this is a comparison of how closely the theme for the linked page is to the theme of your site.

Page Relevance - this is a comparison of how closely the theme for the linked page is to the theme of your site.

In the case where there is too little data to calculate a theme, we set these to 1000. So a suggested first step to using these is to filter the "Offsite Links" report to not include rows including that value by going to the filter text box and typing !1000.00 .

Your next step is to sort the column you're interested in (e.g. site relevance). Most likely you will want these in descending order as this will list the least relevant (lowest numbers) first.

There's no specific number below which it is bad, but in general we've found that numbers below 0.5 merit consideration. You should consider whether this links are necessary and if there's anything that can be done to improve them.

When doing that keep in mind that the report is showing similarity to the page you are linking to (not the entire domain). If you control the page on the remote site then you might consider changing it by adding more text to make it more relevant. If you don't then maybe the webmaster might consider it.

Otherwise you should consider whether it is truly useful for your website users. If not, then it's a good plan to remove that link. If so, then you should probably keep it (remember, the rubber band example we gave earlier in this article - it's setting the theme just a little bit. You should prefer a good user experience over a tiny change).

Hopefully that's given you an insight, and another path to tighten the theme of your website. We're excited at The Crawl Tool as it's the first time we've used AI in the product, with more to come. However, in line with our philosophy and that everything needs to be transparent for it to be actionable, we'll describe all future things with articles like this too.