Screaming Frog and its Features

The correct tools can make a website’s SEO audit a lot easier. The Screaming Frog SEO Spider is one such tool. It can be used to analyse a website and find problems that can harm a website’s search engine ranking. We’ll go over the most important features in this tutorial.

What is Screaming Frog?

The Screaming Frog SEO Spider is a simple desktop tool that you may install on your Windows, Mac, or Linux computer. From an SEO standpoint, it crawls a website’s links, pictures, CSS, and other elements. It essentially describes what a search spider sees when crawling a page.

From an onsite SEO standpoint, this information helps you to swiftly analyse, audit, and examine a site. It can save you a lot of time, as manually examining each page of a large website is difficult.

How to Download and Install

I’ll use a small case study to demonstrate some of the most commonly used functions of the Screaming Frog SEO spider. It would be fantastic if you could join us, so first get the tool here: http://www.screamingfrog.co.uk/seo-spider/. 

It’s available for Windows, Mac, and Ubuntu, and it’s absolutely free, however the free licence limits you to crawling 500 URIs. If you buy a licence, you’ll be able to crawl more URIs (which is useful if you’re dealing with a large site) and get a few more capabilities.

Features of Screaming Frog:

1. Crawling a Website

Are you ready to start crawling your first website? Let’s get started! I’ll use the website SitSEM as an example, but you can use your own domain to follow along.

To begin, we must enter a URL to crawl. Press the Start button after pasting the root domain into the box.

You must click the Crawl All Subdomains box under Configuration > Spider if you want to crawl more subdomains (for example, a blog on the URL ‘blog.website.com’).

The crawl process can take a few minutes depending on the size of your website. To check when it’s finished, look at the progress bar in the top right corner. After that, we can begin analysing the results.

2. Check the Response Codes

Under the ‘Response Codes’ tab, I normally start by looking at the HTTP status codes. The following are the most important HTTP status codes:

• 200: OK
• 301: Permanent redirect
• 302: Temporary redirect
• 404: Not found
• 500: Server error
• 503: Unavailable

Start by double-checking all of the redirects (301 and 302) to make sure you’re using the correct version. Remember that 301 redirects are permanent, and they transfer authority to the new page. With a 302 redirect, this isn’t the case.

After that, we may check at the 404 errors. When you try to view a page that doesn’t exist – either because it was deleted or renamed – you’ll see this error (without redirecting to a new URL). Implement a 301 redirect to a suitable page to fix these problems.

NOTE: Keep in mind that the crawl errors report in Google Webmaster Tools should be double-checked.

You may also use this page to look for broken internal links. Look at the In Links tab in the bottom window after clicking on a URL with a 4xx or 5xx status code. A list of all the pages that link to the broken URL may be found here.

3. URLs

I already emphasised the importance of a decent URL structure in our SEO series. The URL tab of the Screaming Frog SEO crawler allows us to examine all of a website’s URLs.

Begin by sorting the column to determine the length of each URL. See if you can find some that are unusually long. Remember that a good URL is brief (ideally four to five words) and descriptive (for example, website.com/seo-tutorial rather than website.com/p5145).

Remember to check the filters for non-ASCII characters, underscores, uppercase, duplicates, and dynamic URLs. Because all of these elements might cause indexation issues, it’s best to address them as soon as feasible.

4. Page Titles

After that, we can look at the page titles. Every page should have its own title, with the most significant term of the page appearing first. Our website, as you can see from this analysis, utilises the same title on every page. This is something that has to be addressed as soon as possible.

Keep in mind that a title should be no more than 60 characters long. Despite the fact that this is still a valuable metric, we should focus on the Title 1 Pixel Width column. It measures the length of your titles in pixels rather than characters, as the name implies. Longer titles exceeding 512 pixels are truncated by Google. Because some letters and digits take up more space than others (for example, I vs W), a title that is less than 60 characters long may nonetheless be excessive.

5. Meta Descriptions

Under the page title, meta descriptions are utilised in search engine result pages. It’s great if they don’t exceed 160 characters. Descriptions that are too long will be shortened.

The meta description 1 field is empty in our analysis, implying that there are no meta descriptions on this website. This means that each page will require its own meta description.

Screaming Frog has a handy new feature which allows you to simulate a search snippet and analyse it. Simply click on a URL and select the SERP snippet tab at the bottom of your screen. Change the title and meta description and see how it looks on a desktop, smartphone or tablet.

6. Images

Not only for mobile consumers, but also for search engines, page speed is critical. When ranking web pages, Google has announced that page speed would be taken into account. Images are one factor that can cause a page to load slowly. Because high-resolution photographs use a lot of bandwidth, it’s recommended to keep them under 100kb.

You may quickly export photographs that are too large using the images over 100kb filter. Don’t forget to look at the alternative text as well. Make sure each image has its own alt text that is both distinctive and descriptive.

Tips: Using the pictures over 100kb filter, you may rapidly export photographs that are too large. Also, don’t forget to check at the alternate text. Make sure that each image has its own alt text that is both unique and informative.

7. Directives

The meta robots tag, canonical links, and rel=next/prev annotations are all covered in the Directives tab. To rapidly browse all pages using a specific sort of directive, use one of the filters. As you can see, this report for Gero Wonen Meubelen is blank, indicating that we will need to add tags as needed.

8. XML Sitemap

An XML Sitemap can be generated using the Screaming Frog SEO spider. Search engines utilise this type of Sitemap to double-check that they have found all relevant content. It’s a map that guides them through your website’s crawling process.

Sitemaps > Create XML Sitemap is where you’ll find this feature in the top navigation bar. You can manually change the priority and frequency of certain pages once the file has been created.

Conslusion:

You may use the Screaming Frog SEO Spider to look at page titles, meta descriptions, URL structure, response codes, pictures, and other on-site aspects. It’s an excellent tool for optimising a website and improving its performance in search engine results pages. It’s also absolutely free, therefore it should be a must-have for any web designer’s toolkit!