THE INTERVIEW ADDICTION

Why the addictive reliance on interviews as the preferred employee selection method
can be harmful to organisations

Interviews are a terrible

predictor of performance

- Laszlo Bock
Vice President of People operations, Google
 tr g inter na tiona l.com/ solutions /t a le nt - m a na ge m e nt

The interview addiction. Copyright 2016 by TRG Talent. All rights reserved. No part of the report may be reproduced in any form or by any
electronic or mechanical means including information storage and retrieval systems without written permission from the publisher.

P U B L I SHE R
TRG Talent
trginternational.com/solutions/talent-management

Singapore Ho Chi Minh City Ha Noi

21st Floor, Centennial Tower 5th Floor – Central Plaza Building 13th Floor, BIDV Tower 194
Temasek Avenue Singapore 17 Le Duan Boulevard District 1, Tran Quang Khai Street
039190 Ho Chi Minh City, Vietnam Hoan Kiem District,
(+84) 8 3823 6900 Ha Noi, Vietnam

AC K N OWLE D GE ME NT S
TRG Talent – Founder & CEO: Rick Yvanovich
TRG Talent – Talent Management Manager: Tran Quoc Huy
Editor-in-Chief: Rick Yvanovich
Author: Thai Nhat Huy
 tr g inter na tiona l.com/ solutions /t a le nt - m a na ge m e nt

Executive Summary Index

Job interviews are arguably the most beloved employee selection Interviews are

3 13
tool. They are so widely used that interviews are synonymous with terrible at What are the
the hiring process. However, most job interviews are unstructured
and prove to be unreliable and ineffective. That is, they fail to
predicting better
predict actual employee performance. Many studies have shown performance alternatives?
that there is little relationship between candidates’ performance in
interviews and subsequent on-the-job performance. Consequently,
hiring the right people is practically a hit-or-miss matter if the hiring
How cognitive

16
managers rely on unstructured interviews only.

8
biases make Conclusion
Unstructured job interviews are unreliable and ineffective because interviews
they are influenced by the interviewers’ biased and subjective unreliable?
judgements. Even though HR professionals tend to believe they are
good at judging people, they are, in fact, subject to a variety of
cognitive biases that hinder their ability to make fair and accurate

12
decisions. Why HR
professionals
Research has shown that structured interviews and other assessment love interviews
tools, such as cognitive ability tests or personality tests, are superior
to traditional unstructured interviews with respect to hiring the right
people. Nonetheless, the HR professionals’ resistance to moving
away from their much beloved tool means organisations still rely on
a selection method that does not work.
Unstructured interviews
can explain a mere 14%
of employees’ actual
performance.
 tr g inter na tiona l.com/ solutions /t a le nt - m a na ge m e nt

Interviews are terrible at

predicting performance
Most job interviews are unstructured and open-ended. The
questions are not standardised, so each candidate may be given
a list of completely different and randomly chosen questions. More
important, there are no objective and systematic standards to
assess the responses. An interviewee’s performance is rated solely
based on their interviewer’s intuition and personal preferences.
Without a doubt, HR professionals love traditional unstructured
interviews. They have, however, been scientifically proven to be
ineffective for quite some time.
In a ground-breaking meta-analysis (the statistical procedure for
combining data from multiple studies) published in the
Psychological Bulletin in 1998, Frank Schmidt and the late John
Hunter analysed 85 years of prior research on how well different
selection methods can predict on-the-job performance. Out of 19
methods, the commonly used unstructured interview came in ninth.
 tr g inter na tiona l.com/ solutions/t a le nt - m a na ge m e nt

The R-squared value of 0.14 indicates that

METHODS R R-squared
unstructured job interviews can explain a mere 14% RANK (Validity coefficient)
of (variation in) hired employees’ actual
performance. The higher a method’s R-squared is, 1 Work sample tests 0.54 0.29
the more predictive power that method has.
Consequently, hiring the right people is practically
a hit-or-miss matter if the hiring managers rely on
2 GMA (general mental abilities) tests 0.51 0.26
unstructured interviews only.
3 Structured interviews 0.51 0.26

4 Peer ratings 0.49 0.24

5 Job knowledge tests 0.48 0.23

6 T&E behavioural consistency 0.45 0.20

7 Job tryout procedure 0.44 0.19

8 Integrity tests 0.41 0.17

9 Unstructured interviews 0.38 0.14

 tr g inter na tiona l.com/ solutions /t a le nt - m a na ge m e nt
Case-in-point: Why Google says no to interviews
Google was well known for the trickiest questions you could ever
imagine encountering in a job interview. Such questions aimed to
test how well interviewees could tackle complex problems on the
spot. These included brainteasers such as thinking of ways to find a
needle in a haystack or estimating the number of tennis balls that
can fit into a plane. They were supposed to help Google choose the
brightest people in Silicon Valley.
However, along with a culture of data-driven decision making,
Google has the immense analytics capabilities necessary to
discover the true effectiveness of job interviews. In the summer of
2006, Google did a comprehensive study to determine whether
their current hiring methods could accurately predict actual
performance1.
Google required every employee to fill out a 300-question survey.
They looked at thousands of job interviews over the years,
compared how the employees performed in their interviews and
how they subsequently performed in their jobs. They also assessed
the managers who had done the interviews. Surprisingly, they
found no relationship between interview performance and on-the-
job performance.
In an interview with the New York Times, Laszlo Bock, vice
president of people operations at Google, called that hiring
process “a complete random mess”. He also admitted that
brainteasers are useless and only good for making the
interviewers feel smart.
1 Source: www.theatlantic.com/business/archive/2016/04/the-science-of-smart-hiring/477561
 tr g inter na tiona l.com/ solutions /t a le nt - m a na ge m e nt

“Interviews are a terrible predictor of performance,” Bock said.

That is, they are unreliable in terms of hiring the right people.

Google discovered that none of their managers was good at using

interviews to select the right employees. The only exception was an
expert who interviewed candidates for a highly specialised skill set.

Laszlo’s remark points out one major issue of unstructured job

interviews—they are more about the interviewers than
interviewees. In other words, they are less of a tool to select the
right candidates and more of a way for hiring managers to stay in
their comfort zone and strengthen their biases.

When we mention biases, we do not mean the hiring managers

intentionally favour certain candidates over others. We are talking
about human cognitive biases that hinder the ability to make fair *According to Wolfram|Alpha, approximately 6.7 million tennis
and accurate judgements. balls can fit into a Boeing 747.
 tr g inter na tiona l.com/ solutions /t a le nt - m a na ge m e nt

We are all subject to some

types of cognitive biases
when making decisions.
 tr g inter na tiona l.com/ solutions /t a le nt - m a na ge m e nt
How cognitive biases make
interviews unreliable
Cognitive biases are unintentional flaws
in our ways of processing information
and reaching judgements. We are all
subject to some types of cognitive biases
to a certain extent when making
decisions.
We will further examine what types of
cognitive biases can affect HR
professionals doing job interviews and
how they make unstructured interviews
such an unreliable selection method.
Confirmation bias
This bias is the tendency to look for new information and interpret
it in a way that confirms existing beliefs or conceptions.
If the interviewers hold negative opinions towards a candidate
from the very beginning of the interview, they tend to spend the
rest of the time finding reasons to reject that candidate and
omitting positive aspects. Conversely, interviewers can easily miss
red flags about a candidate if they are impressed by certain
aspects, e.g., spectacular past achievements or being good-
looking.
“What are your strengths and weaknesses?” “What do you do in
your pastime?” Such common questions are practically useless
because they are mainly used to confirm the pre-existing
judgements that the interviewer has towards the interviewee.
 tr g inter na tiona l.com/ solutions/t a le nt - m a na ge m e nt
Anchoring bias
People tend to overly rely on the first piece of information they
receive (called “the anchor”). This anchor subsequently dominates
and skews decision making.
For instance, your friend told you about an outstanding candidate
who would definitely be the best fit for your vacancy. Later you
found out that the candidate did not hold a candle to all the hype,
but you decided to hire the candidate anyway because the initial
impression of your friend’s reference was too dominant.
Recency bias
This is the tendency to make decisions based predominantly on the
most recent data. A typical example is when the stock market is
doing very well and investors think it will always stay that way.
Recency bias also helps explain why the first candidate of the day is
less likely to be recruited than the other candidates.
Contrast bias
This effect involves making inaccurate judgements based on the
benchmark of a preceding observation. It makes assessing several
consecutive interviewees particularly challenging. After a series of
disappointing interviews, the next candidate who is just an average
performer can be falsely viewed as an excellent candidate.
Conversely, a qualified candidate can be rated much less favourably
only because he or she is interviewed after several other outstanding
candidates.
Availability bias
People tend to be over-reliance on the information that is most readily
available for them to recall. This effect is particularly troublesome in
unstructured interviews because the candidates’ responses are
normally not recorded carefully. Consequently, the hiring managers
will need to rely on their memory to make decisions. The candidates’
special or unusual aspects are easier to remember and therefore will
unfairly influence the conclusion. For example, one memorable
limitation could create a highly unfavourable perception towards an
applicant even though in reality that limitation is not significant.
 tr g inter na tiona l.com/ solutions /t a le nt - m a na ge m e nt

Stereotyping bias Halo-effect bias

This bias is the tendency to expect an individual of a group to have
certain characteristics you associate with that group. Some typical
examples of this bias are: 1) Male candidates are better than female
counterparts for technical jobs, 2) women are more suitable for
administrative positions and 3) a creative graphic designer will dress
like a hipster.
This bias happens when one strong point of a candidate dominates
the interviewer’s entire perception about that candidate. For example,
the candidate is highly articulated and answers the first few questions
well, which leads the interviewer to believe the candidate is qualified.
The opposite of this effect is called the pitchfork effect, when one
weak point of the candidate influences the whole interview.
 tr g inter na tiona l.com/ solutions/t a le nt - m a na ge m e nt

Experienced hiring managers

can get too confident in their
ability to judge people and
believe they are always right.
 tr g inter na tiona l.com/ solutions /t a le nt - m a na ge m e nt
Why HR profes s ionals love inter views
Cognitive biases also help explain why HR professionals still cling to
unstructured interviews despite evidence of their limitations.
Choice-supportive bias
People tend to retroactively feel positive about their choices, even if
they are flawed. Hiring managers would like to think they have hired
the best candidates and, therefore, do not see the need to change.
Overconfidence
Overconfidence is also a type of cognitive bias. Experienced hiring
managers can get too confident in their ability to judge people and
believe they are always right. Subsequently, they cling to their much
beloved evaluation method—unstructured interviews—even if concrete
evidence shows they are not nearly as effective as previously believed.
“For whatever reason, we have a deep-seated need to feel that we
can judge character,” says Jason Dana, a visiting assistant professor in
behavioural and decision research at the Yale School of Management.
“The assumption is, if I meet them, I’ll know. People are wildly
overconfident in their ability to do this, from a short meeting.”2
"I like you because I’m like you"
It is no secret that we have a tendency to favour people who are
similar to us. Consequently, interviewers may unconsciously choose the
candidates that have the same characteristics, styles, interests or
backgrounds.
There are, of course, more practical reasons why unstructured
interviews are extremely popular among HR professionals: They are
more convenient for interviewers. There is no need to prepare, test and
finalise a list of questions. Throughout the interviews, interviewers do
not need to carefully record interviewees’ answers.
This is not to say companies should abandon job interviews altogether.
It is understandable that hiring managers want to know their
candidates on a more personal level after CV screening. It is the way
interviews are conducted that must be changed. While much less
popular than unstructured interviews, structured interviews are far more
powerful in terms of predicting actual job performance.
2 Source: bostonglobe.com/ideas/2013/11/24/want-best-person-for-job-don-interview/3LB4rwjf6i88GfaDoRubLN/
story
 tr g inter na tiona l.com/ solutions /t a le nt - m a na ge m e nt

What are the better alternatives?

Structured interviews enable hiring managers to make personal
connections with candidates while minimising the impacts of bias. In
the aforementioned Frank Schmidt and John Hunter analysis, structured
interviews were tied with GMA (general mental abilities) tests as the
second-best predictor of on-the-job performance. They account for 26%
of hired employees’ subsequent performance, as opposed to the 14%
of unstructured interviews.

In comparison to unstructured interviews, structured interviews are far

more demanding and time-consuming in terms of preparation and
implementation. Lists of job-related questions have to be written down,
tested and validated. Weighting is assigned to each question
according to its relative importance. The questions then have to be
delivered to each candidate in a consistent manner, and the
candidates’ responses have to be carefully scored during the
interviews. But the efforts pay off in the end.

8 Source: www.gallup.com/businessjournal/186164/employees-don-know-expected-work.aspx
 tr g inter na tiona l.com/ solutions/t a le nt - m a na ge m e nt

Structured interviews eliminate most biases and subjectivity via

standardisation. All candidates are asked the same questions,
which are job-related and carefully crafted to measure the
candidates’ specific skills, competencies and experiences.
Random, brainteaser questions should be avoided.

Each question is assigned a certain weight, for they are not

equally important. The interviewers should also score each answer
immediately rather than wait until the end of the session to avoid
biases like recency and availability.

Ideally, the list of questions will be reviewed and updated from

time to time to ensure their relevance and effectiveness. It also
helps prevent candidates from obtaining the questions from
previous interviewees and practising their responses in advance.

The underpinning of structured interviews is rather simple—how

candidates behaved in the past is the best predictor of their future
performance. Typically, two kinds of questions appear in a
structured interview—behavioural and situational.
 tr g inter na tiona l.com/ solutions /t a le nt - m a na ge m e nt

Behavioural questions require candidates to describe what they Situational questions require candidates to explain how they would
actually achieved or experienced in their prior jobs. For instance, handle specific, job-related situations. For instance, “What would you
“Tell me about a project that you failed to complete” or “Tell me do if your boss rejected most of your ideas?” or “What would you do
about a time when you had to work with people you dislike.” if you were assigned a task you are not trained to do?”

According to Frank Schmidt and John Hunter’s analysis,

other assessment methods are better than or as good as
structured interviews: work sample tests and GMA tests.

Work sample tests

Work sample tests require candidates to perform tasks or activities
that mirror what they would do in the job. They are, understandably,
more geared towards routine, technical or task-oriented positions,
such as mechanics, graphic designers, clerical staff, and
programmers.
GMA tests
Every single selection method has its own pros and cons, and none
can be claimed to be all-powerful with regard to hiring the right
people. Even the best method in the Frank Schmidt and John Hunter
study, work sample tests, can explain only 29% of employee
performance. Consequently, it is better to use a combination of
assessment tools. That is, structured interviews should be accompanied
by some kind of test to maximise the results.

General mental ability (GMA) tests are tied with structured interviews
as the second-best predictors of actual employee performance. They
assess cognitive abilities such as reasoning, reading comprehension,
logic, mathematical ability and verbal ability. GMA tests can be
administered via paper- or computer-based formats. Cognitive ability
tests have been proven to be exceptionally good at predicting on-the-
job performance.
 tr g inter na tiona l.com/ solutions /t a le nt - m a na ge m e nt
Conclusion
The limitations of traditional, unstructured interviews are well
known and have been mentioned in several distinctive studies. The
development of alternative assessment techniques and tools—
structured interviews, cognitive ability tests, behaviour tests, etc.—
has been the highlight of advancements in industrial and
organisational (I/O) psychology over the past decades. HR
professionals, however, still fail to adopt them and instead rely on
their beloved method, which falls short.
This addictive reliance on unstructured interviews and the false sense
of effectiveness they create can prove costly to organisations. It is,
therefore, imperative that CEOs and HR professionals alike
understand the gravity of the situation and start to take a more
scientific and systematic approach to candidate assessment during
hiring.
For inquiries or a free consultation about employee assessment,
please click the link below.
FREE CONSULTATION
 tr g inter na tiona l.com/ solutions /t a le nt - m a na ge m e nt

Other resources
T RG T AL E N T HE L PS OR GAN I S AT I ON S OF AL L S I ZE S T O B UI L D AN D
T O R E T AI N A HI GH PE R F OR M AN CE W OR KF OR CE .

TRG BLOG | TALENT

hubs.ly/H04Gppz0
You can find out about the newest research and best tactics to ensure your company attracts
and keeps the best talent. Follow our blog for short, easy-to-read posts on what you need to
know and what you can do to assemble the best team to help your business succeed.

TRG LIBRARY | TALENT

hubs.ly/H04Gp8K0
At TRG Talent, we have compiled a wealth of resources based on our extensive expertise in
the workplace field. We offer eBooks, Blooks, Whitepapers and Research Reports to our
audience. We encourage you to explore some of the materials below and please contact
us if you need additional information or have questions.
 TRG TA LE N T H E L PS O R GA N I SAT I O N S O F A L L SI Z E S
TO B U ILD A N D T O R E T A I N A H I GH PE R FO R M A N C E
W O R K FO R C E .

For more information about this solution, please contact us

TALENT@TRGINTERNATIONAL.COM

T R GI N TER NATIONAL.C OM
B L OG. T RGINTER NATIONAL.C OM

https://www https://ww https://t

.facebook.c w.linkedin witter.co
om/TRGInt .com/comp m/TRG_
ernational/ any/53525