SEO AB Split Testing, Test Settings and Accidental Keywords - May 2018 Update

It has been a while since posting a SanityCheck update, so there are a few things to round up.

It has been a while since posting a SanityCheck update, so there are a few things to round up. One big SEO AB Testing feature to try, and some small improvements to simple testing. Also Accidental Keyword ideas to help give you some new content ideas you can easily get new traffic for.

SEO AB Split Testing

This is something I am really excited about. The basic testing of SanityCheck is cool - but it is susceptible to seasonality and algorithm updates.

The AB Testing functionality of SanityCheck is based around an article posted by the Pinterest Engineering team : Demystifying SEO with experiments

AB Testing in SanityCheck lets you create 2 groups of urls. The first group is the pages you make a change to - this is your test group. The second group of urls you leave as they are - this is the control group.

This will work particularly well for pages that are template based such as ecommerce product pages. You need the ability in your website's content management system to be able to make changes to a set of pages, and leave the others as are.

SanityCheck can then compare the test group against the control group and show you if there is an increase or decrease in clicks.

This helps take out an seasonality or algorithm updates as the click difference will only be affected by your changes.

Here is a test I have been running on one of my own sites that has had good results

SEO AB Split Testing Results

I still need to work on the documentation and how to explain this - but if you would like to give AB Testing a try, reply to this email and I can help you set it up.

Test Periods

Running a test in SanityCheck has become one of the most useful features. Knowing whether a page change has a positive or negative effect on clicks, rankings and CTR before rolling it out to the rest of a site is a big improvement in how to make site changes.

The testing functionality started out with a default 6 week test period. This meant it took the last 6 weeks of data, and compared it to the next 6 weeks since the test was made.

I have now added Test Period as an option when you are creating a test. You can now run a test for 2,4 or 6 weeks.

SEO test periods

How long you want to run a test for depends on what you are changing, but as a simple guide:

2 weeks - on page changes such as Titles and Meta Descriptions.

4 weeks - internal linking.

6 weeks - bigger site changes, or new external links into a page/site.

Which test period you choose comes down to how quickly Google will recognise the change. Title and Meta description changes are reflected within a day or two usually so test results can flow through quickly, whereas the other two examples generally take a longer time to have an effect on rankings.

There is no harm with running each test for 6 weeks still. The longer you run it the more definitive the results, but it is nice to now have the option.

Tests based on a query

Previously you could setup a test for an individual page or the entire site. Now you can also setup a test for a particular query.

Run SEO Tests based on queries

This is useful when you are working on the content of a particular page that is targeting a query. Now you can see how your changes directly affect the keyword you are going after.

Accidental Keyword Ideas

Hidden in the depths of Google Search Console data are queries you are ranking for by accident. You might be appearing on page 20 - and so never realise it. Sometimes these can be queries that are highly relevant that can generate a decent amount of traffic. If you were to create a page that targets one of these queries there is a good chance you could rank quite nicely for it.

Each Monday (or Tuesday if Monday is a national holiday in the UK!) you will get an email with 10 of these keywords. Some of these queries will be nonsense. But every so often a couple will jump out at you and give you a new content idea.

I have had some good feedback from users on the Accidental Keywords. Some have taken the ideas, created a piece of content around it and the new page becoming the most visited page within a week.