The Search Quality Evaluator Guidelines are a document used by third-party Quality Raters to inform what changes to Google’s algorithm may improve user experience in search.

Sounds pretty important. So does a high rating by human Quality Raters help your organic search rankings?

If you’ve heard some buzz about Google’s Search Quality Evaluator Guidelines but aren’t quite sure what they’re for, you might think so.

Let’s take a look at why people may think these guidelines are a ranking factor, the evidence for and against it, and whether there’s evidence this document is part of Google’s algorithm.

The Claim: Search Quality Evaluator Guidelines As A Ranking Factor

In 2011, Jennifer Ledbetter (AKA PotPieGirl) discovered the then-secret URL rater guidebook in a Google search.

I wrote about it shortly after, and at the time Google’s training manual for human URL quality raters was 125-pages in length.

URL rating seemed to be a step below Search Quality rating, as evidenced by this line from the introduction: “When you can do URL rating, you will be well on your way to becoming a successful Search Quality Rater!”

There was some speculation initially that since they were cruising the web evaluating pages and making recommendations, those Quality Raters might be able to impose manual penalties if they came across pages that violate Google’s guidelines.

However, these are not members of the webspam team. They’re not Google employees at all; the Quality Raters are now and have always been third-party contractors.

As far as I can tell, there are still at least five companies that supply Search Quality Raters to Google and other search engines, including Microsoft’s Universal Human Relevance System:

And looking back on this article from nearly 10 years ago, we see that as much as the algorithm has changed (as evidenced by all of the updates we know about — and many more than we don’t), Google’s goals have largely stayed the same.

What mattered then still makes for a great search experience today:

• How the query is interpreted.
• Understanding intent.
• The context of language and location.
• Timeliness.
• Specificity.
• Page utility; the usefulness of the content.

The things that would land you in hot water with Google back then will still cause you headaches today – keyword stuffing, sneaky redirects, and mass-produced/spun or duplicate content among them.

The Evidence For Search Quality Evaluator Guidelines As A Ranking Factor

Search Quality Raters are used to evaluate proposed changes to the algorithm so Google can gauge the impact of each one in small tests and adjust (or scrap the update) accordingly.

Their feedback is not – and has never been – a direct ranking factor.

However…

Do I believe that what the Raters are looking for is what Google wants to see on a webpage?

Absolutely.

Do I think you’re crazy if you choose to ignore Google’s expressed desires to see specific things on a webpage?

Also yes.

Ben Gomes, SVP of Education at Google, has been with the company since a few months after it launched. He was VP of Search Engineering in 2018 when he told CNBC:

“You can view the rater guidelines as where we want the search algorithm to go. They don’t tell you how the algorithm is ranking results, but they fundamentally show what the algorithm should do.”

It’s a lot easier to create quality content and optimize it for search when you understand what Google itself considers quality.

The Evidence Against Search Quality Evaluator Guidelines As A Ranking Factor

Google tells us exactly how human Quality Raters help improve Search results:

• They provide feedback on search experiments to inform which potential changes are most useful.
• They help Google categorize information to improve its systems.
• And they use the raters’ guidelines to do so.

Remember, Google is one giant, complex information retrieval system.

Quality Raters provide feedback that may influence how the algorithms operate. But they have no direct impact on the output of those algorithms (search results).

Search Quality Evaluator Guidelines As A Ranking Factor: Our Verdict

The Search Quality Evaluator Guidelines give us insight into what Google considers a good user experience and quality content, and that can be advantageous.

Implementing some of the teachings from these guidelines as best practice might help your SEO strategy, by virtue of the improved searcher experience you will provide.

But a ranking factor they are not.

Bottom line: Google does not use its Search Quality Evaluator Guidelines as a search ranking signal.

Featured image: Paulo Bobita