When what’s good for

users isn’t good for Google

Will Critchlow | SearchPilot

@willcritchlow
slideshare.net/willcritchlow
This was a WINNING conversion rate test
 -25%
organic
traffic!
 In the olden days:
“Can you just SEO this page?”
Our call to adventure:

UX is a ranking
factor
UX is a ranking
factor
UX is a ranking
factor
How users see
your site
UX is a ranking
factor
UX is a ranking
factor
 UX is a ranking
factor
Affects organic
search performance
Controversial Accepted
CTR for algorithm
evaluation

This has been

confirmed
repeatedly.
 “I’ve talked explicitly about using
clickthrough rate for experiments”
Paul Haahr - @haahr
https://youtu.be/eGSGZMI4Z_I?t=979
Controversial Accepted
 Human UX
ratings

Qualitative ratings
of pages that rank:
used to evaluate
algorithms
“... make this page a frustratingly poor
user experience”

-- reason for poor page quality rating

Search quality rater guidelines [PDF]

Controversial Accepted
 ML models
trained on ratings

E.g. Panda -
trained to like the
things humans
like
Back in 2011, I was suggesting we run our own
Panda-like quality surveys (WBF here, instructions here)
 Sidenote: totally worth running
these surveys for executive buy-in

Since Panda went real-time, quality issues don’t necessarily cause

obvious drops correlated with algorithm history dates
 Client site 1 Client site 2

Would you trust information from this website? 72% 64%

 Client site 1 Client site 2 Key competitor

Would you trust information from this website? 72% 64% 81%
 Client site 1 Client site 2 Key competitor

Would you trust information from this website? 72% 64% 81%

Is this website written by experts? 50% 46% 65%

 Client site 1 Client site 2 Key competitor

Would you trust information from this website? 72% 64% 81%

Is this website written by experts? 50% 46% 65%

Would you give this site your credit card details? 29% 21% 43%
 Client site 1 Client site 2 Key competitor

Would you trust information from this website? 72% 64% 81%

Is this website written by experts? 50% 46% 65%

Would you give this site your credit card details? 29% 21% 43%

Are there any noticeable errors on this page? 6% 4% 1%

 Client site 1 Client site 2 Key competitor

Would you trust information from this website? 72% 64% 81%

Is this website written by experts? 50% 46% 65%

Would you give this site your credit card details? 29% 21% 43%

Are there any noticeable errors on this page? 6% 4% 1%

Does this page provide original content or info? 76% 72% 85%
 Client site 1 Client site 2 Key competitor

Would you trust information from this website? 72% 64% 81%

Is this website written by experts? 50% 46% 65%

Would you give this site your credit card details? 29% 21% 43%

Are there any noticeable errors on this page? 6% 4% 1%

Does this page provide original content or info? 76% 72% 85%

Would you recognize this site as an authority? 44% 33% 58%

 Client site 1 Client site 2 Key competitor

Would you trust information from this website? 72% 64% 81%

Is this website written by experts? 50% 46% 65%

Would you give this site your credit card details? 29% 21% 43%

Are there any noticeable errors on this page? 6% 4% 1%

Does this page provide original content or info? 76% 72% 85%

Would you recognize this site as an authority? 44% 33% 58%

Does this website contain insightful analysis? 72% 62% 81%

 Client site 1 Client site 2 Key competitor

Would you trust information from this website? 72% 64% 81%

Is this website written by experts? 50% 46% 65%

Would you give this site your credit card details? 29% 21% 43%

Are there any noticeable errors on this page? 6% 4% 1%

Does this page provide original content or info? 76% 72% 85%

Would you recognize this site as an authority? 44% 33% 58%

Does this website contain insightful analysis? 72% 62% 81%

Would you consider bookmarking pages on this site? 44% 38% 56%
 Client site 1 Client site 2 Key competitor

Would you trust information from this website? 72% 64% 81%

Is this website written by experts? 50% 46% 65%

Would you give this site your credit card details? 29% 21% 43%

Are there any noticeable errors on this page? 6% 4% 1%

Does this page provide original content or info? 76% 72% 85%

Would you recognize this site as an authority? 44% 33% 58%

Does this website contain insightful analysis? 72% 62% 81%

Would you consider bookmarking pages on this site? 44% 38% 56%

Are there excessive adverts on this website? 2% 2% 8%

 Client site 1 Client site 2 Key competitor

Would you trust information from this website? 72% 64% 81%

Is this website written by experts? 50% 46% 65%

Would you give this site your credit card details? 29% 21% 43%

Are there any noticeable errors on this page? 6% 4% 1%

Does this page provide original content or info? 76% 72% 85%

Would you recognize this site as an authority? 44% 33% 58%

Does this website contain insightful analysis? 72% 62% 81%

Would you consider bookmarking pages on this site? 44% 38% 56%

Are there excessive adverts on this website? 2% 2% 8%

Could pages from this site appear in print? 54% 54% 59%
Controversial Accepted
 Site speed
measured
directly

Real User Metrics

including site
speed from
Chrome
https://moz.com/blog/google-chrome-usage-data-measure-site-speed
 ny
t ho
An
o m
, T
TO
t C
i l o
P
a rch
S e
by
e d
n c
o u
nn
st a
Fi r
https://moz.com/blog/google-chrome-usage-data-measure-site-speed
Controversial Accepted
CTR as a direct
ranking factor

This has been

explicitly denied
by Google reps.
 Some kind of
SERP interaction
affecting results
we see

My theory.
We sell custom cigar humidors. Our custom cigar
humidors are handmade. If youʼre thinking of
buying a custom cigar humidor, please contact
our custom cigar humidor specialists at
custom.cigar.humidors@example.com

Keyword stuffing: Doesn’t mean keyword density is the correct metric

Most organisations refuse to accept they overlap

SEO CRO

Search Engine Conversion Rate

Optimisation Optimisation
But in truth there is a lot of overlap

SEO CRO
 While some SEO changes donʼt impact CRO

Meta info - invisible

to UX SEO CRO
And some CRO changes donʼt impact SEO

SEO CRO Non-indexed pages

An awful lot of changes affect both
Crossing the threshold:

Weʼre not in
Kansas anymore
 We should be building SEO
hypotheses from UX fundamentals
BUT

-25%
organic
traffic!
 We publish winning and losing tests at searchpilot.com

We publish our case study email newsletter because itʼs the resource we wish existed
Tweet me: @willcritchlow #BrightonSEO
Conversion Rate Conversion rate
Optimisation
optimisation focuses
on converting more
of your existing traffic
(bottom of the
funnel)
Conversion
Search Engine
Rate Search engine
Optimisation
optimisation focuses
on getting additional
traffic to your
website (top of
funnel)
Full Funnel Testing Elixir:
Full funnel testing
combines these
approaches to do
both at the same
time
Supreme ordeal:

How do SEO and

product teams work
together well?
 Totally safe:
Green Not in scope for
CRO testing
 Needs care:
Potential for
Amber impact, but not
overlapping
 Direct overlap:
Overlapping
Red changes or
hypotheses
Road back:

We then use a flow

chart for amber tests
 Are there
conversion rate
1. tests running or
planned on this
site section?
 Do the
hypotheses affect
2. the same part of
the page?
 Do the tests target
3. the same user
behaviour levers?
 We use a RACI matrix to keep the right people in the loop

Product Broader
Test Type SEO Teams Engineering Ad Teams Leadership
Teams Marketing

Green Tests Accountable Informed Informed Informed Informed Informed

 We use a RACI matrix to keep the right people in the loop

Product Broader
Test Type SEO Teams Engineering Ad Teams Leadership
Teams Marketing

Green Tests Accountable Informed Informed Informed Informed Informed

Amber Tests Responsible Accountable Informed Consulted Informed Informed

 We use a RACI matrix to keep the right people in the loop

Product Broader
Test Type SEO Teams Engineering Ad Teams Leadership
Teams Marketing

Green Tests Accountable Informed Informed Informed Informed Informed

Amber Tests Responsible Accountable Informed Consulted Informed Informed

Red Tests Responsible Accountable Informed Consulted Informed Informed

In summary
UX is a ranking
factor?
Controversial Accepted

Not everyone agrees

 But we don’t need the futurist
perspectives or the tinfoil hat conspiracy
theories for my purposes here

We just need to know that UX is aligned with Google’s goals

 1. Panda (ML-trained on UX signals)

2. Quant and qual measurement of algo

changes using UX metrics

We just need to know that UX is aligned with Google’s goals

To know that we should be building
SEO hypotheses from UX
fundamentals
BUT: then we affect users and search
 So: we need to play nicely together

Product Broader
Test Type SEO Teams Engineering Ad Teams Leadership
Teams Marketing

Green Tests Accountable Informed Informed Informed Informed Informed

Amber Tests Responsible Accountable Informed Consulted Informed Informed

Red Tests Responsible Accountable Informed Consulted Informed Informed

Full Funnel Testing

And ideally, test

both the SEO and
CRO impact of our
ideas
Questions: @willcritchlow
Lessons from
100s of SEO
tests
The restaurant
4.10pm
Image credits

● Hornet ● Phone
● Beach ● Devil cat
● Paul Haahr ● Coloured smoke
● Google ● Kansas
● Smile ● Brighton
● Pug ● On stage in San Diego
● Chrome
● Tinfoil hat
● Broken plate
● Confused monkey
● Unicyclist
● Horse