Google Does Not Use Quality Raters for Machine Learning Algos
Whenever there is a new quality raters guidelines released by Google, there is always lots of speculation about what Google does with the data from those ratings. And does that include machine learning?
Google confirmed that they use quality raters to evaluate potential changes to the search algos. They push out a change and then use quality raters to evaluate the quality of the search results and the pages that the raters see. If Google sees the result of those ratings that the potential algo change improves the search results, then it is much more likely that the particular change will be pushed to the live search results. But if the raters rate the results as lower quality, then Google will try to adjust whatever change it was and possibly try a new version of it with the raters, or not use it at all.
It is important to note that the ratings themselves don’t have any impact on the live search results. So if a rater rates your site low, it won’t affect your rankings. However, if your site has characteristics of a low rating site, then as algos are tweaked to target those lower quality signals, your rankings can go down. The rater guidelines are basically a handbook on the types of sites Google wants to see rank well and not rank well, so site owners should really strive to be as high quality as possible.
There was speculation by a few different SEOs that Google might possibly be using results from quality raters for machine learning training. My response was that sample size of just using raters would be very small, too small for a very good dataset for training purposes.
I posed the question to Google on twitter and Danny Sullivan responded and commented that no, they do not use the quality ratings for machine learning.
We don't use it that way.
— Danny Sullivan (@dannysullivan) August 5, 2018
This doesn’t surprise me. Data sets used for machine learning tend to be pretty large, and while there are a fair number of quality raters, it wouldn’t be close to the numbers that would be needed to effectively train for machine learning purposes. Not only that, Google tests many different algo changes with the raters, and some of those algos could be pretty extreme in their targeting, which again would make the raters results a poor set to use for machine learning. Then of course, biases can seem much larger in a smaller group, which again would affect any use of machine learning from these results.
We do know that Google is using machine learning in many applications across many areas at the company. This includes RankBrain, which is part of the search algo and helps Google rank results for the 15% of daily queries that Google has never seen before. But they are not using the quality raters ratings for machine learning purposes.
Jennifer Slegg is a longtime speaker and expert in search engine marketing, working in the industry for almost 20 years. When she isn’t sitting at her desk writing and working, she can be found grabbing a latte at her local Starbucks or planning her next trip to Disneyland. She regularly speaks at Pubcon, SMX, State of Search, Brighton SEO and more, and has been presenting at conferences for over a decade.
Latest posts by Jennifer Slegg (see all)
