Seo experiment: Click-through-rate as ranking factor

Does the click-through-rate affect search engine rankings as a ranking factor?

To answer this question I did an experiment – the results of which I present here.

Furthermore, because of the relation to the topic, I will shortly consider the often stated “fact” that Google Analytics Data and Google Chrome Data would be used for ranking purposes.

I am always amazed at the widespread opinion that the Chrome browser tracks user behaviour on a large scale, e.g. pages viewed, links clicked, length of stay on a Web page, and even the scrolling behavior. I think this is unproven so far (even though a lot of prominent SEOs say so) and I am very skeptical.

Much more likely, however, is the tracking of user behavior on the Google website itself. One possibility to do so would be to investigate clicking behavior (click-through-rate – CTR) of users on search results. Results that are listed for a search query on the first page, but are rarely clicked, probably are not as relevant to the user (to the exact query made ​​- but more on that later!) and could therefore be downgraded. Results, which are often clicked seem to be very relevant to a search query and could be promoted in the ranking. But this is a theory and until now I did not see any proof for it.

I have noticed tracking links in search results that could be used for click-tracking, so the question for me is not whether this procedure is used, but how it really works in practice.

Interesting to know would be:

  • how fast changes in the CTR affect search engine rankings,
  • wether or not these changes persist for a longer time
  • and of course if we are able to affekt the click-through-rate besides optimizing the title and description tag – or if such efforts will be viewed as manipulation.

Below I present a small-scale public CTR experiment which I started in some German Google+ SEO Groups.

My first contact with click-through-rate (CTR)

Just recently Rand Fishkin has conducted an experiment about CTR. He mobilized his Twitter followers and asked them to click a blogpost in the search results . His post then rose to the first place. That alone would be reason enough to deal with the issue also for German Google.

But the first time that I coped with the subject “click-through-rate” was last year. At that time I had a private trial started in which I have guided international visitors to a German website through a search enginge results page (SERP). That had an obvious positive effect for the used keyword – the ranking climbed as you can see in the following graph:

Ranking changes of the first ctr experiment

At that time I was so looking ahead to do a negative control (the ranking of the same website/keyword in Bing). While Google’s ranking rose tidy, Bing ranking was continuous and got rather worse. While this is not a proof that the positive ranking change is due to the increased CTR in Google, no other SEO measures were carried out during the test period for the sub-page.

I nether want to tell the method I used back then nor the used keyword and website. But it was a first sign for me that the CTR might be used for ranking purposes by Google – so I hope you are now eager to read about the method and results of my current experiment about this.

My recent click-through-rate experiment

Methods:

As test page I used an older blog article of mine Einzelhandel vs. Online Handel – David gegen Goliath? (actually the German version of the article). This article has got the Google Analytics tracking code and I confirmed my authorship. There were no SEO actions since the article has been published (still 1 backlink) and the article has been stable on position 7 for the choosen keyword for at least some days before the experiment startet.

Call for participation in the experiment on Google+For this latest experiment I asked in two Google+ seo groups to search Google for the German phrase “einzelhandel vs onlinehandel and then to click on my entry on the SERP.

I then (anonymously) tracked the ranking for the keyword phrase each day and at the end of the experiment I got the page-visits from Google Analytics.

There were no new backlinks during the test (and as I said there have been none since last year) – not even from social media platforms. The call for participation was only by text/image.

Also there were no new comments on the test page shortly before or while the experiment was conducted.

Results and discussion of the CTR experiment

Effect of the CTR from SERPs on ranking

Course of unique page impressions and ranking of the keyword phrase
As can be seen in the figure, there were over 30 unique page views (orange bar) from organic Google search done on the first day (May 20). This is a much higher organic visitor number / CTR for a day as the weeks before (not shown) and also the day after. So there was a large influx of visitors and then again two days later on May 22, again a smaller “visitor-peak” with nearly 10 unique page views.

The course of the organic rankings in Google is shown in blue. The first 10 days after launch is only a slight fluctuation around the starting place, then the website is increasing by leaps and bounds within 2 days (30.5.-1.6.) to the #1 spot.

Although since then the page views have declined again, the page keeps stable so far at #1 in the search results.

You could now argue that this change in the rankings can only come from the changed CTR from the search results. After all, this article was around 1 year old and has not seen any substantial SEO measures. Also, no substantive content changes were made since then that could cause this change.

Nevertheless, of course, there is at least one issue: whether any of the tracked Google Analytics changes have also contributed their part to the improved ranking – therefore there follows a brief consideration.

Changes in two analytics metrics by the experiment

Again and again I hear from other SEOs who think that Google Analytics data from Google would be used to determine the rankings. Since I used Analytics on the testpage the question “Did the tracking with Google Analytics or the user data that were collected affect the ranking?” will arise for this experiment, too.

Lets have a look at the data – the following figure shows the average time on site and bounce rate before and during this seo experiment:

Comparison of the average time on site and bouncerate before and during the experiment
The metrics average time on site and bounce rate remained relatively constant, but have both deteriorated a bit after the start of the experiment. The average time on the visited page has fallen by 28 seconds and the bounce rate has risen (see screenshot which corresponds to the relative change of 0.05%) to 0.04 absolute percentage points.

Worse usage data – but better ranking?

As you just saw: the usage data in Analytics has deteriorated slightly! Personally, I’ve never believed that this data is used by the ranking algorithm and still do not do it after this findings.

Restriction: Of course, this is just a small test and for true statements I would have to repeat the experiment many times. Moreover, it could be that (if these two metrics were indeed ranking factors) their weighting in contrast to the click-through-rate is much lower. Then deterioration due to the higher click-through-rate would be recaptured.

I do not want to stress this point any further because it would be pure speculation with my restricted data. If you are concerned that Google Tools alter the results of this experiment then maybe you want to repeat it without such tools on the test page – I happily link to good repetitions of the experiment.

Usage of the browser Google Chrome as a ranking factor?

As already touched upon in the introduction, it is often claimed that Google Chrome spies on users and evaluates user signals. The last time that I read this, was in the current issue # 26 of the German journal Website Boosting (p. 45/47) – of course, again, no evidence is adduced to show, but the purely speculative assertion is sold as fact.

Therefore, it is of course interesting to see how the browser distribution for this experiment looked like:

Comparison of usage of different browsers during the seo experiment
(minor browsers are left out of the graphic)

In effect, there was a sharp increase in the Chrome users by 100% (which sounds impressive, but the absolute increase of 15 is of course not that big). So 30 Chrome users have clicked on the article during the test period. Actually, I would have to segment this data more (to part organic Chrome visitors from other Chrome visitors) but I confess to be too lazy at the moment to do this. But I think you get the point and ask yourself the question:

Lead more visitors with the browser Chrome possibly to a better ranking? Once again I want to remind, that correlation and causality are not the same. In my opinion it is sufficient to look at the change in the average visit duration to stifle that thought.

The drop of 96.92% is extreme. I could lament about possible reasons why the 15 Chrome users had such a long visit duration before the test (apparently Chrome is very prevalent among the internet workers, so their visits of the article lead to this new and very short average visit duration – but why has the length of stay for Firefox users increased even more)?.

The bounce rate is over 90% (and it obviously was before, so I should think about appropriate content strategy and internal linking for my blog).

But what I really want to show by these two number is that if use of the Chrome data would be a ranking faktor then possibly the site would not have climbed to rank number 1 – the sharp decline in length of stay would rather lead to worse ranking results and the strong bounce rate should prevent a good ranking … therefore I still do not believe in the use of Chrome data as a ranking factor! In a small survey on Chrome as a ranking factor in an SEO group by me, nobody had a good indication let alone provide evidence for the assertion that usage data is used for ranking purposes.

The CTR is tracked for each exact keyword

What I find very interesting is that the CTR is apparently calculated for each exact search phrase. Thus, the above ranking increase is indeed for the phrase “einzelhandel vs onlinehandel”, but not for the related phrase “onlinehandel vs einzelhandel”. For this I am still ranking at No. 7. For the search phrase “einzelhandel vs online handel” the article is even ranked 8th.

Rankings of related search queries compared after end of the test

Self-criticism of the experiment

I would like to invite all readers to express their own thoughts, comments and constructive criticism in the comments. In the following I anticipate a few points that I could have done differently:

  • The selected phrase has a vanishingly low search volume(keyword planner gave no data for it). Of course you can assume now that for a high-volume money keyword, the result would be quite different. Yes, that may be. My first attempt in 2013 (see above) was a moderate Money Keyword (500 exact searches per month in Germany). In this respect I think that you can transfer the result to other situations. It is self-explanatory that for a search volume of 500 or more per month you need higher changes to the CTR to achieve an effect at all.
  • Since hardly any information on keywords can be gained from Analytics nowadays (the “not provided” problem), I cannot split the above-mentioned organic traffic to the article. This of course distorts the said unique page views, as some are done through another keyword.
    But that does not detract from the message of this experiment. If less people should have searched for the test phrase “einzelhandel vs onlinehandel” (therefore came to the article by other keywords) and yet the ranking has risen so strongly, then the impact of the CTR is rather greater than expected.
  • In the heat of the moment I have not set up a negative control. It would have been interesting to see whether the ranking history of the search engine Bing is unaffected, or whether there are any fluctuations during the test period. The expectation would be, of course, that the result remains constant since Bing certainly has no access to the user signals of the search engine Google.
  • Unfortunately, I have not raised any far-reaching Ranking data before the experiment. But if you look at the ranking for the related search phrases, then that should not be necessary. It is then easy to imagine that my article was relatively stable at No. 7 until the experiment has started. Because for the other two search phrases it is still there (shown above).
  • Which leads me to my final point: Unfortunately I did not track the ranking of the test page for the related phrases. This would have been interesting, just to see if there are any short-term effects, or whether the CTR really only affects the respective exact CTR (which seems to be the case on the long run).

Ways to use the CTR for better rankings

Because this post has gotten really long I will describe this in detail in a next blog post. So here a short preview:

1. Optimization of the click throuth rate by changing and testing the meta tags title and description (sorry, no surprise here).

2. Implement the authorship (Some time ago there was an announcement that there will be no author pictures and circle counts in the SERPs any longer. But still there will be your name under your result on the SERP so that might still be an advantage – maybe it should be subject to another SEO experiment…

3. Use rich snippets

4. Manipulation of the CTR (I am really no fan of such things but we do not live in a dream world and if you do not use this or at least think about such things … your competitors will do for sure – so even if you do not use such methods it is good to recognize them if used against you)

Click-through-rate as a ranking factor: Conclusions

  • There are good hints that the CTR is used by Google as a ranking factor
  • It is therefore advisable to think about good titles and descriptions to get clicks to your entries on result pages (maybe even at the expense of a high bounce rate! My bad bounce rate (as seen above) did not prevent a good ranking but maybe the search engines compare all bounce rates and all pages do have a high bounce rate for this search term)
  • If you have got many “friends” then you could ask them to search keywords that are important to you and let them click on your page in the SERPs. More on this in the follow up article.
  • If Google Analytics and Google Chrome usage data is important for rankings stays pure speculation (even if it is often sold as matter of fact). In this experiment it does not seem to influence the test page.
  • This experiment should be refined and repeated several times to verify the results. It should also be done for more difficult money keywords (but my first experiment was on a money keywords so there are hints that the results are also true for more often searched terms).

Finally, once again many thanks to all who participated and clicked.

There are no comments yet, add one below.

Leave a comment


Comments are moderated !

Blog
About the author
Autorenfoto
Thomas Merkel assists
small businesses and self-employed persons with their online marketing. He prefers a holisic view that comprises design, psychology and analytics: That`s what he calls the clever webdesign process.
  • Recent Posts

  • Categories