Bip 2206-2012

Licensed Copy: Mr.
Universiti Teknologi Malaysia User, Universiti Teknologi Malaysia, 14/08/2012 02:51, Uncontrolled Copy, (c) The British Standards Institution 2012
From Theory to Practice

Quality in Market Research
Licensed Copy: Mr. Universiti Teknologi Malaysia User, Universiti Teknologi Malaysia, 14/08/2012 02:51, Uncontrolled Copy, (c) The British Standards Institution 2012
From Theory to Practice
Debrah Harding and Peter Jackson

This second edition published in the UK in 2012

by
BSI Standards Limited
389 Chiswick High Road
London W4 4AL
© The British Standards Institution 2012
First edition published by Kogan Page in 1997
All rights reserved. Except as permitted under the Copyright, Designs and Patents
Act 1988, no part of this publication may be reproduced, stored in a retrieval
system or transmitted in any form or by any means – electronic, photocopying,
recording or otherwise – without prior permission in writing from the publisher.
Whilst every care has been taken in developing and compiling this publication, BSI
accepts no liability for any loss or damage caused, arising directly or indirectly in
connection with reliance on its contents except to the extent that such liability
may not be excluded in law.
While every effort has been made to trace all copyright holders, anyone claiming
copyright should get in touch with the BSI at the above address.
BSI has no responsibility for the persistence or accuracy of URLs for external or
third-party internet websites referred to in this book, and does not guarantee that
any content on such websites is, or will remain, accurate or appropriate.
Typeset in Frutiger by Letterpart Limited, www.letterpart.com

Printed in Great Britain by Berforts Group, www.berforts.co.uk
British Library Cataloguing in Publication Data
A catalogue record for this book is available from the British Library
ISBN 978-0-580-70782-7
Contents
Foreword vii
1 Introduction 1
Market research today 2
The quality movement 4
Standards 5
Delivering quality 8
Improving quality 10
2 Market research as an industry 11

What is market research? 11
The demand for market research services 13
Research suppliers 16
Market researchers 19
Trends in the market research industry 23
3 Quality and truth 27

Truth, science and research 28
Truth in social sciences 31
Qualitative research and understanding 34
Validity and reliability in market research 35
Science, truth, and managing quality 41
4 Quality and the profession 43

Professions and members 43
Knowledge and professions 49
Professional organizations 51
Codes and standards 52
Key principles 54
Professions and standards 57
Professionals and their clients 58
Is professionalism enough? 60
5 Quality and business process 63

Satisfying clients 63
Conformity to specification 66
Quality and money 67
Delivering market research quality through the business process 71
Quality in Market Research v

6 Quality and legislation 81

Legislation and research 82
Data protection and privacy 82
Freedom of information 93
Other communications 94
Types of research and research environment 95
Is legislation good for research? 96
7 Quality management and BS EN ISO 9001 99

What is BS EN ISO 9001? 100
What BS EN ISO 9001 requires 102
Assessment and certification 112
BS EN ISO 9001: In summary 114
8 Process standards for interviewers 115

Interviewing problems 115
The role of IQCS 123
IQCS minimum standards 124
Assessment and inspection 128
Is it all enough? 128
Standards for the whole research process 129
9 International quality standards for research 131

The BS ISO 20252 standard 132
The BS ISO 26362 standard 144
Are two ISO standards needed? 147
10 People quality – Investors in People 149

What is required? 149
National Occupational Standards 154
Investors in People 159
Assessment to IiP 163
IiP: In summary 164
11 Quality system development, implementation and

assessment 167
Commitment and resources 167
What changes are needed: the initial audit 170
Management system documentation 172
Implementation: getting it all working 179
Assessment and certification 184
Assessment to other standards 187
Specimen procedures 189
Bibliography and sources 207
vi Quality in Market Research

Foreword
Exactly when the first sample survey occurred is a matter on which

opinion is divided. However, it was at least some 250 years ago, and more
large scale census taking can be dated and located much earlier. From
those historic roots market and social research have grown to be the
considerable activity they form today whose results are used by
businesses, governments, pressure groups and citizens around the world.
Whilst quantitative survey research has its theoretical basis in statistics

and related disciplines, the validity of the results – their validity and
reliability – is equally reliant on the rigour and consistency with which
the processes around survey research – the sample selection, the
questionnaire printing/scripting, the interviewer training and respondent
interface, the data preparation and the data analysis – are undertaken.
The importance of the process quality around these components has

been recognized since the earliest surveys. However, the rapid growth,
increased use of technology and international spread of research have
resulted in greater interest in, and need for, more formal approaches to
quality control. BS ISO 20252:2012 is the industry’s response to this need
and is now spreading rapidly around the world, as both providers and
suppliers of market research become aware of the benefits formal quality
standards provide to all stakeholders.
This book draws on the experience and professional expertise that has
grown up around this subject in the UK over the last 35 years. The
authors have been closely involved over many years in the drafting and
implementation of research quality standards and their expertise and
understanding is displayed throughout this book.
It is a timely addition to the professional survey researchers tool box and

I welcome it and recommend it to all those with an interest in survey and
research quality.
Bill Blyth, Global Methods Director, TNS, Chair ISO TC225
Quality in Market Research vii

1 Introduction
In this introduction we discuss the continuing importance of quality issues

to market and social research in the context of ongoing changes in
research methods and the globalization of research services and business
generally. The different senses of the quality concept are also introduced.
Quality standards are well established in market research and they are
put into the wider context of different types of standards including;
ethical, product, service and those of capability or process. How quality
can be achieved, maintained and improved is also introduced, as is
assessment and certification to standards.
Quality has always been a concern of market research,1 although its

nature continues to change. In the 70s and 80s the focus was upon
addressing concerns about the data collection process, particularly the
need for validation of interviewers’ activities. In the 90s quality issues
widened to include the whole research process, driven partly through the
impact of general standards for managing quality. With the emergence
and rapid increase of new methods of data collection, particularly digital
research, in the early years of the new century, quality concerns have
shifted again towards issues such as the reliability of respondent access
panels, the quality of data collected online, the reliability of online
samples, and broader methodological issues such as the diminishing need
for representative research towards ‘good enough research’. Another
trend has been the increasingly international scope and structure of
research with corresponding international quality initiatives. The first
edition of this book2 was published in 1997, soon after a period of
debate about quality in market research and how to achieve it, and the
purpose of the book was to consider the issues then current. In the 15
years since then there have been many changes relevant to the subject
including those mentioned above. An update of the book is, therefore,
timely and to give adequate justice to the changes some of the contents
are new and much of it has been re-written, this time through the work
of two authors.
The research sector has a long history of supporting and devising

standards to improve quality and these have manifested in a number of
ways including codes of conduct, data collection standards (e.g. the
1
Throughout the text we will refer to ‘market research’ as shorthand for market, social and
opinion research unless explicitly stated.
2
Jackson, P (1997) Quality in Market Research, Kogan Page, London
Quality in Market Research 1

1 Introduction
Interviewer Quality Control Scheme in the UK), national and now

international process standards such as ISO (International Organization
for Standardization) 20252:2012 Market, opinion and social research –
Vocabulary and service requirements and professional development
programmes. Among other subjects, the role, impact and merits of these
initiatives are discussed in later chapters of this book, together with some
practical guidance and advice on implementing formal quality schemes.
However, the earlier chapters consider rather more general issues such as
the structure of the market research industry and some of the theoretical
underpinnings of quality in market research.
We hope the book will assist those involved in research to understand

the factors which impact on research quality, the schemes which are there
to assist this, the parameters of the schemes and the holistic approach
that is required to address quality and research excellence. As such this
book is aimed at all those with an interest in or responsibility for
research and research quality: compliance and quality professionals,
research providers, associated suppliers of relevant technology, research
buyers and users and those involved in research assessment and
certification.
Market research today

Research today is a mass of contradictions with a number of push and
pull factors which are stretching research and could irrevocably change its
nature:
• Research is in demand. Since the turn of the 21st century research

has experienced a period of strong global growth and industry
consolidation, only recently stemmed by the global recession of
2008/9. But for some, research has become a commodity and there
are an increasing number of clients3 buying purely on price or doing
it for themselves. DIY digital research tools have made conducting
research easier and some clients no longer need or want a researcher
as the intermediary.
• The emergence and growth of social media has radically opened up
new avenues of opportunity for accessing and learning more about
the populations’ opinions and attitudes. To give some context, the
social media site Facebook had 845 million monthly users at the end
of December 2011 (based on Facebook’s December 2011 Fact Sheet
figures4); if it were a country it would be the world’s third largest
(smaller only than China and India). There are over 181 million blogs,
3
Research clients, buyers and users of research are all terms for an individual, organization,
department or division, internal or external, which requests or commissions research
projects.
4
Facebook’s factsheet, Facebook newsroom,
http://newsroom.fb.com/content/default.aspx?NewsAreaId=22 [accessed 14 May 2012].
2 Quality in Market Research

Market research today
of which 34 per cent post opinions about products and brands.5

Research has never before had such a seemingly readily available and
easily accessible resource. But how best to engage via social media?
How reliable are such sources? What about the established research
models and how do they apply in this environment?
• Social media has led a re-emergence of mass observation on a scale
not seen since the 30s and 40s. Researchers have to listen and
observe more and ask less. But how to do this ethically and legally
when so much information can be so easily accessed with something
as simple as a smart mobile phone?
• Research is widely recognized as an essential business tool, used to
support business decisions. But buyers want these insights quickly
and they want actionable research insight that makes a measurable
difference to their business. Some are of the opinion that there is no
longer always a need for representative results; instead some buyers
are happy with ‘good enough research’, which is timely and based on
what the ‘right’ group of customers are thinking and wanting.
• Response rates are decreasing. Finding respondents agreeing to
participate in research is increasingly difficult, resulting in the need
for different ways to engage respondents, from the growth of
incentivized respondent panels to the favouring of more passive
research techniques (such as online data scraping). What does this
mean for quality? Is the research sector creating enormous groups of
professional respondents through the development of panels? Will
respondents only participate if incentivized? How reliable are
scraping approaches, particularly those trawling online sources?
• Anonymity is no longer an essential requirement, with the focus
being on transparency and confidentiality. Research boundaries are
expanding. More and more non-research exercises are being
undertaken by researchers, for example co-creation exercises where
researcher, respondent and client jointly design and develop products
and services. With the gap narrowing between market research and
other forms of marketing and customer relationship management
what does that mean for research as a profession? If researchers
become the innovators, the creators, how do they maintain their
objectivity to predict, rationalize, persuade and dissuade?
These developments are innovative and exciting, but they can strain the
formal disciplines and in the forthcoming chapters we will consider the
impact of these trends on research standards and the various quality
schemes.
5
Fascinating Social Media Facts of Year 2010 – social media today,
http://socialmediatoday.com/soravjain/237864/fascinating-social-media-facts-year-2010
[accessed 12 April 2012].

1 Introduction
The quality movement

All sorts of reasons can be suggested for why quality has become such a
central concern throughout the economically developed world. One
impetus has been the stimulus of a global economy, the potential to
purchase anything from anywhere via the development of online
markets. These and other aspects of economic growth and expansion
have all stimulated suppliers’ preoccupation with quality issues. However,
downturns have been a factor as well; when demand levels contract the
first need is to hang on to the customers you have and not allow
competitors to acquire the business. Customer satisfaction in a tight
business climate has been very much part of the quality ‘movement’.
However, these concerns are also part of broader issues such as marketing
return on investment.
Quality has two broad and different senses. First there is quality as
excellence; the best; the superb. This usage is probably more common in
normal language, but as a working concept it has been either ignored or
rejected by management thinkers and the quality movement. Quality as
excellence, it is said, is just not a practical business concept. The second,
and the more applied, interpretation is relative quality; relative, that is,
to what is required. In other words good enough for the purpose in
mind: ‘fit for purpose’ and better rather than the best.
Relative quality, then, is a matter of meeting requirements with an

emphasis on customer requirements; providing what is wanted or (and
this may be different) what is needed. A commercial qualification is made
by adding the rider ‘at least cost’ or providing the best value for money.
Practical quality is, therefore, not just a matter of giving the customer
what he or she wants but doing so at least cost, with a measurable
return on investment and most profit to the provider; a focus on
customer wants and needs has a very strong commercial aspect. In a
competitive market this is absolutely necessary; if the customer is not
satisfied they can always go elsewhere. Wants and needs are, however,
themselves problematical in any application of quality and this will be
explored shortly. For the moment though it is worth noting that if
customer requirements are to be met they must be identified and
understood. This is very much part of the subject matter of market
research and therefore relative quality, whatever else, is good business
for market research.
In the literature of quality and in all manner of quality programmes, it is

relative quality which has made the running. But quality as excellence
cannot be dismissed quite so lightly and especially in relation to quality
in market research. In opposition to relative quality, quality as excellence
has the implication of a goal irrespective or even despite what the
customer wants. Also, in an age of greater consumer control, with
respondents actively participating in ‘their brands’ via social networks
and media, customers’ demands may exceed even the expectations,

Standards
aspirations and supply ability of brand owners. Quality as excellence

resonates with concepts such as the Platonic ‘Good’ and ‘Truth’ and the
latter link is of particular interest when exploring quality in market
research. Whatever else, and however it is defined, the output of market
research services is expected to be ‘true’ against some sort of yardstick. In
the end, few market research suppliers can be satisfied merely because
the client likes the results; there is an imperative to provide ‘true’
findings and insight. The practical limitations of timing and budget
constraints are seen as concerning accuracy levels rather than any
fundamental compromise with the concept of delivering the truth. This
subject is explored further in Chapter 3.
Quality as excellence also has a dynamic dimension. Relative quality is all

very well but merely meeting current wants or needs is no stimulus to
improvement. Pursuit of excellence is one way in which businesses
anticipate rather than merely meet consumer demand. Innovation may
also be built into quality programmes based on relative quality along
with the distant drum of excellence acting as a spur.
Professionalism is also built around notions of quality as excellence. The

professional-client relationship assumes less than perfect knowledge by
the latter of his or her needs or wants and it is the job of the
professional to deliver quality based on high standards generated from
within the profession. The issue of quality and professionalism is the
subject of Chapter 4.
Standards
There are different types of standards: product or service standards,
quality process standards, professional ethical standards, business ethical
standards, etc. Whereas a product or service standard establishes the
criteria against which things can be measured in a verifiable and
consistent manner and process standards involve how they are produced,
others such as professional ethical standards are a form of applied ethics,
which concerns the moral issues that arise in a profession. Ethical
standards tend to be assessed by others within the profession (i.e.
self-regulation) as it requires the specialist knowledge of the profession
to effectively judge others’ professional moral conduct and behaviour.
Standards are a well established means of ensuring that relevant needs

are met. Sometimes organizations and practitioners will refer to their
own bespoke internal ‘standards’ but proprietary standards of this type
are, by definition an oxymoron. Standards imply some universality
applied by a range of organizations, not just one, and are defined and
interpreted by some recognized external body such as the British
Standards Institution (for quality standards) or a well regarded trade or
professional association (usually for Codes of Practice or Conduct).

1 Introduction
The point of product standards for technical and manufacturing processes

are relatively easy to understand. If you want to be sure a fire
extinguisher will work if ever you need it you assume or hope that it is
produced to a standard which ensures this is the case. How the appliance
works – the underlying physics – need not concern you if you have faith
in the standard and the supplier’s conformity to that standard. By far the
most common quality standards are the thousands of technical product
specifications issued by the national standards bodies such as the British
Standards Institution (BSI) in the UK, Asociación Española de
Normalización y Certificación (AENOR) in Spain, Deutsches Institut für
Normung (DIN) in Germany, etc. There is a national standard, or British
Standard (BS) as is the case in the UK, for virtually every significant
product with a technical dimension. In business, buyers require their
suppliers to meet relevant product standards as a minimum condition for
being considered at all, although additional and more unique
requirements may of course be specified as well. In applications which
have a critical safety dimension the need for conformity to established
technical standards may also be a legal or quasi-legal requirement. Two
important features of technical product standards is (1) that they concern
the key parameters which determine whether or not the product works
(rather than how it looks) i.e. whether it is ‘fit for purpose’ – and (2) that
conformity to the standard can be determined by objective measures and
tests. If, for example, dimensional tolerance is part of the specification,
whether this is met can be established by appropriate measurements.
While nearly all published technical standards relate to physical products,

there is no reason in principle why the concept cannot be extended to
various services. Again key parameters which determine whether or not
the service meets basic needs can be defined and conformity objectively
measured. At least this is so in principle.
In the UK, market research was one of the first service standards to be
introduced (as BS 7911:1998, Specification for Organizations Conducting
Market Research) and codified the standards that the research sector had
already developed under its own steam, firstly via the introduction of the
Interviewer Quality Control Scheme (IQCS) and later by the activities of
the Market Research Quality Standards Association (MRQSA). Similar
initiatives were undertaken elsewhere e.g. Australia, Germany, France
and Spain. Chapter 8 will look at some of these early schemes,
particularly those standards for interviewers.
The increasing globalization of research, and the need for consistent

quality standards across markets, prompted the research sector to unify
the various standards, culminating in the introduction of an international
standard for market research, BS ISO 20252, Market, opinion and social
research – Vocabulary and service requirements, in 2006. In 2009 this was
supplemented by the introduction of a separate standard for access
panels, BS ISO 26362:2009, Access panels in market, opinion and social

Standards
research – vocabulary and service requirements. Chapter 9 will look at the

implementation, application and assessment of the international
standards for research in more detail.
There are also process or capability standards, developed to improve

management and especially in relation to ensuring adherence to the
delivery of product or service standards. First within the defence industry
and later more generally it was recognized that delivering quality,
whether this is defined in terms of meeting wants and needs or just
conforming to defined product standards, requires a formal
methodology; quality management systems or quality assurance models.
Appropriate standards were, therefore, developed and published such as,
in the UK, BS 5750 which was later superseded by the BS EN ISO 9000
series. As will be discussed in Chapter 7, BS EN ISO 9001:2008, Quality
management systems – Requirements has had some impact within market
research; standards such as BS ISO 20252 in addition to being research
service standards also incorporate the quality management or assurance
models of BS EN ISO 9001.
Once standards are established within a field of activity the question of

assessment arises. Buyers of products or services covered by standards
want assurance that what is delivered is actually in conformity to the
parameters set in the standard. Similarly suppliers wish to be able to
demonstrate that this is the case and thus differentiate themselves from
‘substandard’ competitors. Such assessment can be carried out by a first,
second or third party. First party is simply a matter of the supplier
asserting that the service meets the standard. Obviously there is no harm
in doing this but it may be less than convincing to the buyer (although it
may be possible to seek recompense if non-conformities are subsequently
found). Second-party assessment involves the buyer establishing
conformity through tests in cooperation with the supplier. This may give
the buyer comfort but involves much effort, not to mention
inconvenience to the supplier (each major customer may wish to send in
a team of quality inspectors).
A solution potentially attractive to both sides is, therefore, third-party or

independent assessment. The supplier pays a recognized organization to
carry out the necessary work to determine conformity to the standard
and the resulting certification can then be presented to any potential
buyer requiring conformity to the standard to be demonstrated. Such
third party assessment is quite well established in relation to product
assessment (e.g. the BSI Kite mark scheme) but is even better known for
the quality management standard BS EN ISO 9001. Increasingly buyers
have sought assurance that their suppliers have methodologies in place
to meet requirements and thus (by tautology) deliver quality. Assessment
against BS EN ISO 9001 has come to be regarded as a strong indicator of
this capability. There has, therefore, been some switch from a focus at
the product level (with inspection of incoming material, exhaustive

1 Introduction
testing etc) to what lies behind the attainment of consistent quality

levels. Some disciplines closely associated with research are essential parts
of these management standards, e.g. customer satisfaction monitoring
and measurement can be found in BS EN ISO 9001 and in
ISO/TS 10004:2010, Quality management – Customer satisfaction –
Guidelines for monitoring and measuring.
Market research ethical standards are detailed within the various codes
of conduct, which set out the principles, rules and practices for individual
research practitioners. All the notable research codes are based on
comparable ethical principles. The International Chamber of Commerce
(ICC) and European Society for Opinion and Marketing Research
(ESOMAR) Code on Market and Social Research is the most widely used,
although most developed markets have their own Codes to reflect the
specific needs of their domestic market. The Code requirements are
discussed in Chapter 4.
With an increasing amount of legislation impacting on the practice of

research, codes need to reflect the interpretation of such legislation in
addition to the over-arching ethical principles governing the profession.
Although the sector remains primarily self-regulatory in most markets
(with some exceptions), the practice of research is increasingly being
governed by legal requirements determined in privacy legislation such as
the Data Protection Directive in Europe. In Chapter 6 we will explore in
greater detail the legal issues impacting on research.
Delivering quality
Given that quality has become a central concern to businesses and other
organizations, how is it to be achieved and delivered to the customer?
Three key elements are widely recognized although the emphasis varies
according to the writer, viewpoint and, arguably most important of all,
the nature of the organization seeking to deliver quality. These elements
are purpose, process and people.
Purpose is about commitment to quality within the organization and this

may be expressed in written mission statements, through meetings,
exhortation or other ways, but, however it is done, it must reflect real
belief and commitment and not just pay lip-service. Commitment must
also be achieved throughout the organization starting at the top but
going all the way down. Corporate Social Responsibility (CSR)
programmes tend to be the way in which organizations bring these issues
together and ensure senior corporate buy-in. However, these are
generally top-line policies which match overarching corporate objectives
and goals; Total Quality Management (TQM) programmes place strong

Delivering quality
emphasis on commitment and how to truly achieve it. It is the process

management systems that underpin CSR policies which deliver quality in
practice.
Process concerns systems and methodologies to deliver quality

consistently and constantly and those stressing its importance argue that
commitment, while essential, cannot deliver without the discipline and
rationality of effective systems. Those with a strong emphasis on process
generally favour the adoption of formal systems and see considerable
benefits in underpinning commitment with assessment to BS EN ISO 9001.
Developing formal systems (often seen as largely a matter of writing
procedure manuals) is, however, only one, and the less radical, strand in
the process approach. Another is to fundamentally reappraise all the
processes which deliver products and services; to re-engineer the
organization. Another is to place heavy reliance on automated processes,
removing any human interaction, as human interaction introduces
subjectivity into process. As such, technology plays a vital role in the way
in which processes are organized, and this applies increasingly to modern
market research. At each stage, technology drives the way in which
research processes are structured and carried out.
However, even the strongest advocates of a process approach recognize

that irrespective of the technology, systems either work or fail through
people and quality ultimately depends on the understanding, skills and
motivation of an organization’s staff. There is, therefore, a loop back to
commitment. Functional, management and customer service skill levels
may vary among staff members. Many and various approaches are
offered to develop this human aspect of delivering quality. Changes to
processes themselves of course require changes in how people work and
the skills they need to employ. Similarly radical changes to processes –
re-engineering – call for the same level of rethinking on the human side.
However, re-engineering is linked to de-layering, downsizing and taking
out costs, which often involves shrinking the workforce and this is
particularly the case in hitherto people-intensive businesses such as
market research. Where the immediate impact of re-engineering seems
to be fewer colleagues and less job security, developing any real staff
commitment to quality can be an uphill task.
A human resource focus is always a strong element in TQM programmes

and the training and skills development aspect is covered by schemes
such as Investors in People schemes and occupational standards
initiatives, explored in more detail in Chapter 10.
A final aspect of the people side of quality to mention at this point is

professionalism (the subject of Chapter 4). Professionalism in market
research, as well as in other knowledge businesses, is often seen as a
more appropriate route to quality than an emphasis on process and
systems.

1 Introduction
Improving quality
The quality concept up to this point has been presented as static;
organizations seek it but then just need to keep it to a sufficient degree.
Clearly, however, this is not enough. Organizations need to seek quality
improvement on a continuous basis with the goal of ever moving
forward. The need for improvement is also, arguably, built into both
concepts of quality. Quality as excellence is always unattainable and the
best we can do is move a bit nearer the impossible. Relative quality is
also always dynamic since the requirements to be fulfilled are themselves
fluid and changing; in fact quality improvement is itself part of the
dynamic. Improvements raise expectations.
TQM programmes take improvement as central to the whole quality issue

and generally put emphasis on both the goals sought and people as the
key means to make advances. The goal is often expressed as going
beyond meeting customer requirements to delighting them; finding
something – which may be trivial in itself – to give satisfaction over and
above that which was expected and anticipated. In market research terms
this may include how clients are handled, how information is
communicated but perhaps above all the intangible creative element
which lifts a routine project to something offering the client real added
value, the actionable ‘insight’ so sought and valued by clients.
System approaches such as those based around BS EN ISO 9001 and

service standards such as BS ISO 20252 tend to be seen by TQM
proponents as at best safety checks to quality improvements; the system
stops other aspects deteriorating while specific improvements are made.
At the worst, systems are said to fossilize quality and in fact inhibit
innovation in customer delivery or shift the focus from substance to form.
Criticisms of this sort have been quite commonly expressed in the market
research sector as part of the debate about the worth of BS EN ISO 9001
and indeed the research standards BS ISO 20252 and BS ISO 26362.
However, this is to ignore the dynamic elements which a good quality
system should include and especially the need to seek improvement
through systematic and deep analysis of identified problems.
BS EN ISO 9001 and BS ISO 20252 require this approach. Moreover such
criticisms fail to understand the broader dynamic of quality schemes. No
one quality process standard, ethical code of conduct, or professional
development approach by itself produces quality market research. It is a
combination of all these standards, and only when these standards are in
alignment in a dynamic, fluid, holistic way, will quality in research be
produced.

2 Market research as an industry
The nature and structure of the market research industry has important
consequences for quality. In this chapter we give a broad picture of the
industry including its boundaries and size, both overall and by segment.
The structure of the supply-side of the industry and of its workers is
described together with some specific consequences for quality issues.
How research is bought and used and trends in the industry are also
linked to aspects of quality and quality initiatives.
Having introduced quality and some quality concepts, it is time to turn to

market research more specifically. This chapter describes the market
research ‘industry’, looking at global trends. It may be of more interest to
those not familiar with how this sector is organized and structured (those
who are familiar may want to skip some bits). In describing the research
industry, however, we have a particular interest in showing why specific
aspects of it have, for structural reasons, implications for quality issues.
Incidentally we are using ‘industry’ to describe the practice of commercial

market research because that is how it is commonly referred to. As we
shall discuss in a later chapter, market research can also make reasonable
claims to be a profession but practitioners tend not to describe it as such.
Perhaps there is some significance in this – who has ever heard of the
legal industry?
What is market research?

The ICC/ESOMAR Code of Conduct defines market research as:
the systematic gathering and interpretation of information about

individuals or organizations using the statistical and analytical
methods and techniques of the applied social sciences to gain insight
or support decision making. The identity of respondents will not be
revealed to the user of the information without explicit consent and
no sales approach will be made to them as a direct result of their
having provided information.
In terms of its application, market research is the provision of systematic

information to assist and guide business decisions. Whether a company
should enter a new sector, launch or modify a product, raise prices, foster
a particular brand image or advertise in a specific way are all potentially

decisions for which information will be needed to lessen the risk of

getting it wrong. Even simple and isolated pieces of information may
help in the decision-making process but something rather more formal
and structured is what is usually meant by market research.
Within this overall framework, the specific objectives of market research

projects and services are quite diverse and can be designed to meet
virtually any need. Some examples include:
• market analysis to identify opportunities;

• new product development work;
• consumer and customer attitude research (this includes customer
satisfaction survey work);
• evaluating and predicting the results of changes in the marketing
mix – pricing, sales methods, advertising effectiveness etc.
Media research and audience measurement is virtually an industry in its

own right. There is also the measurement of standards of service delivery,
for which some specific methods have been developed (mystery
shopping), and which along with customer satisfaction research are very
closely linked to quality concerns.
All types of industries and activities are covered by market research

including business-to-business as well as business-to-consumer. Since
marketing is increasingly international, much research is also
multi-country. Market research techniques can also successfully be applied
to public services and other not-for-profit ‘third sector’ activities, as well
as to social research (from which market research largely developed). One
specialized form of market research is opinion polling which, although
accounting for a very small share of all research (around one per cent in
most developed research markets), has become the yardstick against
which the reliability of market research is unfortunately (and
unrealistically) measured.
A lot of market research is ad hoc – specific one-off projects carried out

to meet a business need of a single client. However, it is continuous
surveys, those which are repeated at different points in time, which
provide the crucial trend data that can be most useful in a business
planning context. Often these are large-scale research programmes with
the costs spread between several subscribing clients.
Market research can also be defined in terms of the methodologies used

to collect the data required in marketing decision-making. Primary data
collection techniques are based largely on some form of interview
although forms of passive data collection, such as observation, are also
used (e.g. mystery shopping or ethnographic research). Digital
technological developments plus societal changes have meant that new
forms of passive data collection, driven primarily by the huge
participation in social networks, have emerged. These are best described

The demand for market research services
as ‘e-ethnography’, i.e. digital observation, whether that be via the web,

mobile phones, etc. Figure 16 gives a breakdown of global spend by
research methods used in market research.
Secondary data sources are also used, including ‘traditional’ desk

research, which involves the collection and analysis of existing data rather
than conducting primary research. Continuous research can also use
secondary data in the form of electronic transaction records from
point-of-sale retail transactions. Other electronic forms of data collection
include various audience measurement techniques, behavioural loyalty
card data, and customer database ‘mining’.
So diverse is market research that arguably the only satisfactory way of

identifying what it encompasses is simply to regard it as any service that
recognized market research organizations carry out. However, research
organizations are branching out into activities where they utilize their
research skills and techniques for purposes beyond research, such as the
development of advisory and consultancy services for broader marketing
and business purposes. Other developments include researchers
facilitating the relationship between clients’ staff with consumers in, for
example, co-creation and new product development programmes.
Historically, one important and overriding boundary was that market

research aims to provide aggregated and anonymous data about
consumers and markets. However, this is no longer always the case. In
some countries, for example Germany, research is defined in legislation
(in this instance as ‘scientific research’) which explicitly states that data
must be anonymous. But in most other European countries identifiable
research data can be passed back to clients but only when the
respondents have consented to be identified, consented to the data
transfer and the identifiable data are only used for the purpose for
which it was collected i.e. research. Should businesses wish to identify
and profile individual customers to identify sales prospects for example,
this is a quite legitimate requirement. However this would be an example
of conducting an exercise which is not for research purposes and would
have to be identified as such. These requirements will be explored in
more detail in Chapter 6 when we discuss data protection legislation and
its impact on research practices.

The ESOMAR Global Market Research Report 2011 reported that global
market research revenues reached $31.2 billion in 2010 (representing an
overall growth of 5.2 per cent and 2.8 per cent after inflation
6
This figure is reproduced with permission by ESOMAR from their report.

14
Market research as an industry
market research
Figure 1 – Breakdown of global spend by research methods used in

on 2009)7. Of this, North America represents one-third of all global

market research turnover, with the UK, Germany, France and Japan the
next largest markets. The top five markets represent over $20 billion,
two-thirds of all research turnover. However, these markets are mature
and growth is mixed with some European countries struggling to reverse
negative trends due to economic difficulties. Latin America and Asia
Pacific are more buoyant markets, with countries such as Vietnam and
Brazil reporting growth rates in excess of 20 per cent.
Some would suggest that the size of the research market is significantly
undervalued due to limitations in the approach used for collecting
industry statistics. Many countries’ figures are based on associations’
estimates of their members’ activities as opposed to the market as a
whole, and there is currently no standard approach to determine which
organizations and activities should be included within the statistics for
research. For example, should data analytics activities (i.e. the process of
reviewing data to identify underlying patterns and relationships such as
advanced statistical modelling) or research into market information be
included? If these activities are conducted by organizations traditionally
associated with ‘research’ they tend to be included, but if conducted by
specialists or those not traditionally aligned with the sector they tend to
be excluded. Some countries have undertaken reappraisals of the size of
the domestic market to address some of these concerns and in the case
of the UK this resulted in a significant increase in the estimation of the
industry’s size. With the changing nature of research (to be discussed in
later chapters) there will be even more challenges to accurately measure
the sector size. The vagueness in these estimates is ironic given that
market researchers confidently make market size estimates all the time
for their clients. Despite its undoubted successes however, market
research, whatever its true size, is only a relatively small business sector
or industry, particularly when compared to other forms of marketing
communication such as advertising and direct marketing.
The Annual Survey of the UK Market Research Industry 2010, conducted

by The Market Research Society (MRS)8, reports that the share of UK
research spend by the client sector remained largely unchanged from the
previous years. A third of spend is by the consumer sector, with just
under 17 per cent by the government/public sector and 12 per cent by
IT/Technology. Other services such as media, financial, pharmaceutical are
less than 10 per cent each of the total spend. There are significant
variations between client spend figures internationally. These
7
ESOMAR’s figures include activities conducted by market research companies or institutes,
and excludes work conducted by non-commercial institutes or research performed by
non-research companies that conduct their own research. Work sub-contracted to research
organizations outside the country is excluded in the domestic turnover figures to avoid
double counting.
8
The Market Research Society’s annual survey accounts for circa 40 per cent of the total
industry sales with contributors representing a diversity of company types and sizes, from
£100 million plus organizations to small sole traders.

discrepancies reflect the origins of certain types of research and/or the

structure of domestic economies, e.g. media in the US is 29 per cent of
sales against a weighted global total of 15 per cent and manufacturing is
69 per cent in Germany against a weighted global total of 45 per cent.9
Most research work is carried out in consumer rather than industrial or
business-to-business markets (globally, business-to-business represents
only 4 per cent)10 although pharmaceutical research is also carried out
largely among non-domestic respondents and some of the other
categories have business elements.
The companies and organizations buying research are diverse not only in
terms of their own activities but also in size. However, as in most business
markets, a high proportion of the total research spend is by large
organizations with considerable budgets allocated to marketing activities.
Research suppliers
Some primary research work is carried out ‘in-house’ and in some areas
this is growing – particularly with the advent of easily accessible,
inexpensive research tools such as online survey and software packages
(the impact of which is to be discussed later) – but the largest proportion
of research is still carried out by commercial market research companies
or agencies11. Traditionally research companies were represented by trade
associations and the profession by professional bodies. Invariably one
body tends to be more dominant, sometimes due to historical issues, but
often it will be dictated by who is the regulator. The code-holding
association will tend be the dominant body. More recently there has also
been a movement to create sector bodies which represent both the trade
and profession and in some instances also the client. For example, in
Canada a totally new association was formed, the Market Research and
Intelligence Association, which included the membership of the former
professional, trade and client associations. In other countries such as the
UK, the larger body, in this instance the professional body – MRS – took
over the services formerly provided by the trade association, and changed
its focus to serve both the research business and the research profession.
The research market sector follows the classic Pareto principle.12 Of the
2010 global research revenues, the top 10 research organizations
represented just over 52 per cent of the total research revenues. This sort
of pattern is replicated nationally. Research companies are commercial
and profit-making businesses with varied ownership patterns. Of the
9
Based on figures from the ESOMAR Global Market Research Report 2011.
10
11
The research suppliers are a mix of companies, partnerships and independent consultants
but ‘research company’ will be used throughout for short-hand.
12
Sometimes known as the ‘80:20 rule’ that roughly 80 per cent of the effects come from 20
per cent of the causes.

Research suppliers
global top 25 research companies, two-thirds are publicly listed or have

parent companies that are public companies, and generally tend to be
managed by non-research practitioners. At the lower end of the scale
most of the boutique and small suppliers are privately owned and are
more likely to be managed by research practitioners and operate in
single markets. The diversity between large and small companies, the
differences in ownership and leadership, creates different cultures and
foci within research companies, not least in terms of approaches to
standards. At the top end of the pyramid it is quite common for research
companies to be owned by larger groups with more ambitious
commercial targets and diverse commercial interests (many of the large
research suppliers are subsidiaries of advertising and media groups) and
may be ultimately answerable to owners unconnected with the
profession or its quality or ethical concerns. This is of course not to
suggest that the research companies concerned are in any way unethical
or any less committed to high standards, simply that the overall business
focus may be different to that of a company that solely supplies research
services.
A wide range of market research services are available, but some research
companies will specialize in some way or other. One division is between
those offering ‘full service’ and those restricting operations to just part of
the research process (data collection and data processing being the most
common). Full service work includes the quality critical process of
research design – identifying client requirements, defining the research
problem and developing a suitable research programme using a range of
techniques and approaches to meet these objectives. On the other hand,
services such as ‘field and tab’ (these are organizations that just offer
data collection services i.e. go out in the ‘field’, collect data and tabulate
– ‘tab’ – the results) are limited to the more routine parts of the process.
The two types of company often inter-trade with ‘full service’ companies
(some of which have no internal data collection or analysis resources)
providing much of the turnover of ‘field and tab’ operations. Equally
some of the larger suppliers which have retained field and tab divisions
will offer data collection services to their competitors.
Research companies also specialize in both the markets and products

covered and the techniques used. Few research companies except the
very largest take on any and every type of client, although most work in
quite a wide range of market sectors. There are two reasons for this.
First, it is a matter of commercial advantage; generally the opportunities
are greater with less sector specialization except in sectors where a small
number of research organizations dominate the supply of specialist
services, such as in media measurement. Second, the principles of market
research are in fact much the same regardless of the product or sector
covered. However, in fully understanding clients’ needs and providing
relevant interpretation of the data, prior experience of the product or

sector has clear benefits and quality implications. Certainly, when selling
to clients, research companies will always emphasize such knowledge and
buyers often put a premium on it.
The most common methodological specialization was traditionally

between quantitative and qualitative research. Put simply, the former is
concerned with key marketing variables while the latter is more focused
on understanding why things are as they are. Typical techniques are
structured interviews with a larger and statistically significant sample of
respondents for quantitative research and group discussions or a small
number of depth interviews13 for qualitative research. The growth of
‘new technology’ methodologies, including online, are blurring the
quantitative/qualitative split. Moreover, many of the newer research
companies veer away from the quantitative/qualitative specialization
approach, preferring to emphasize the consultancy nature of their
businesses rather than the techniques they deploy to address clients’
issues.
Reasons why research companies traditionally specialized in one or the

other of these approaches (some offer both) relate to people and
resources. Professional-level staff are, by training and inclination,
oriented to one or the other and this may include research company
principals and founders. The resources that are relevant include those for
data collection and analysis. The scale of operation required in these
areas is generally much greater for quantitative rather than qualitative
research (although shortfalls can be met by subcontracting and
buying-in). As a consequence most qualitative specialists tend to be small
research companies down to one-man bands – economies of scale are less
apparent in this type of work. Other specializations based on research
techniques include companies (generally the largest) oriented to
continuous research, of which a distinct speciality is retail auditing and
those that have developed their own proprietary research techniques. In
the latter case, companies seek a competitive advantage by offering
specialized techniques and models which they develop and keep
confidential and which may not as a consequence be accessible for open
debate on their intellectual and practical validity. This is increasingly
becoming the case with the advent of technology-driven research
solutions, particularly for online research. This lack of openness is itself a
quality issue.
Shortly, the people working in research will be considered, but a point to

make here is that while the senior staff have and certainly claim
professional status, the companies in many countries are detached from
the recognized organization of the profession. This has been addressed in
some countries, such as the UK, where its association, MRS, has
introduced a corporate scheme, MRS Company Partner Service, to create
13
Depth interview is an individual one-to-one research interview with a single respondent.
Depth interviews can also be done in pairs (i.e. a paired depth interview).

Market researchers
a formal contractual link between research organizations and the MRS

Code of Conduct. Arguably (and this is discussed later) professionalism
and having a code of practice or conduct in place is some guarantee of
quality in research. However, the detachment of research companies that
provide the service may in practice limit the impact of professional
regulation – professional bodies, trade associations and code-holders
ultimately have very limited means of checking conformity, and sanctions
are invariably limited to membership status. Often the biggest deterrent
that regulators have is the publicity that results from issuing a sanction as
opposed to the sanction itself (although discipline of senior members of
staff is not a trivial concern to the companies). Furthermore, companies
are commercial undertakings with a potential for conflict between profit
goals and professional conduct. The reality is of professionals working in
operations (whether as suppliers or research users) which have aims and
goals outside any professional structure or concerns.
Market researchers
Market research is often referred to as a people business and this is true
in quite a few ways. Good research has depended on flair and
intellectual ability rather than technology, although this may be changing
to some extent with ‘intelligent’ IT tools. Placing business with a research
company is often about confidence in the individual selling the project,
in personal chemistry. Also – at least outside the largest agencies or
specialist data houses – the level of capital investment required in
research has been relatively low and a far more crucial issue for agency
management is recruiting and keeping good staff. However, while such
talented executive staff is vital, most people actually working in market
research are interviewers who are non-professional, relatively untrained
and are usually not employees of the research organization (most are
either workers or freelancers retained for specific projects). This particular
structural feature gives rise to some critical quality issues.
Entry-level positions
We shall leave a discussion of whether, or the extent to, which market
research is a profession until a later chapter but it is reasonable for now
at least to refer to professional market researchers. Typically recruits at
this level join a research company or in-house research department with
a first degree from university. Rather fewer have postgraduate degrees.
The disciplines from which they are drawn are very diverse; indeed this is
often the preference at recruitment (there is certainly a good sprinkling
from the social sciences, but also many with arts or natural science
degrees). Professional recruits also come from business courses but
generally the would-be researcher is not expected to have specifically
relevant research knowledge. This is usually acquired in the first two or

so years through a combination of on-the-job training and short courses.

Many of the large research companies have formal graduate programmes
which include vocational research qualifications supplied by professional
bodies (which is the case in countries such as the UK, Australia and
Canada). Some of the larger research companies invest in developing
formal graduate schemes so they can compete with other professions
with track records in offering vocational development (e.g. accountancy
and law) to attract the best graduates.
New recruits at a fairly early stage gravitate to either qualitative or

quantitative research; this partly reflects many corporate structures which
tend to prefer separate qualitative and quantitative divisions, but also
reflects researcher preference for either one or the other discipline. One
of the features of research in recent years, for some countries, has been
the loss of qualified researchers just five years or so into their careers.
Anecdotally this appears to be partly due to the temptations of an early
career break for travelling but for some it is the attraction of other more
lucrative professions in which they can utilize their research skills14. The
2008–09 recession and its after effects has somewhat halted this
movement but this is likely to only be temporary due to current
economic uncertainty.
Professional-level positions
The archetypical job for professional-level staff in research companies is
‘research executive’, although different terms may be used depending on
the company and the level of seniority achieved. However, regardless of
title, the work undertaken includes many of the ‘thinking’ parts of the
research process:
• taking a client’s brief and identifying the requirements to be met,

• preparing a research design,
• developing questionnaires and a sampling plan,
• outlining the requirements from the data processing stage,
• preparing a report from the data including an exposition of the
findings,
• some interpretation and possibly conclusions drawn or inferred from
the results.
The research executive also has a project management role to see that
the whole programme of work is carried out to specification and to
schedule. This side of the work will involve liaison with the ‘service’
departments of the agency – data collection and data processing – and
briefing staff within those areas on the specifics of the research. Finally
and at least as important as the other roles, the research executive also
14
This is based on in-house research with lapsed and former members undertaken by the
membership department at The Market Research Society based in the UK.

Market researchers
has commercial responsibilities: to sell work to potential clients, liaise

with them throughout the project, present the final results and control
project costs. This brief description of research executive responsibilities
applies to ad hoc research and the agencies carrying out this sort of
work. In research companies with a high level of continuous work, similar
roles are carried out but often with a greater division of staff resource; a
feature of ad hoc research is the wide range of tasks undertaken and low
level of division of staff resource. Researchers were exemplars of ‘flexible
labour’ before the term gained common currency.
As well as research executives, professional-level staff in research

companies includes some specialists and managers of service functions.
The specialists are particularly involved in data analysis and IT generally,
from fairly routine programming work to the development of statistical
models and other sophisticated techniques including digital data
collection. In these areas there is potentially a significant impact on the
quality and validity of the final output. Internal service function
managers are much more concerned about ’people handling’ than
structuring research activities. The principal area of internal servicing is
data collection (e.g. field or telephone units), which can include directors,
manager and assistants (depending on the size of the operation) plus
staff in subsidiary departments such as data coding and entry. While most
of the supervised tasks require limited formal qualifications or skills, the
output of work in these areas is the foundation of the finished product,
potentially prone to error and variance and for this reason the quality
issues involved are both critical and problematic.
Client-side researchers
Professional-level staff working in research companies are within
communities that inevitably attach a high level of importance to
professional research values and implicitly to quality issues peculiar to
market research. People with similar training, background and
professional commitment to research also work for companies buying
and using the research output of agencies, i.e. client-side researchers,
who are often transitioning their careers between research companies
and clients. However, not all those buying research are necessarily
researchers. Many client-side research departments have been
rationalized or merged into broader marketing communications
functions, resulting in marketers having responsibility for market research
without necessarily having any real expertise or specialist training. In
addition, individuals in broader business functions will commission
research without any research knowledge at all. In conjunction with this
is the involvement of procurement as a specialist function, which has
meant that an increasing amount of research is being purchased by
procurement professionals. Many of these individuals have broad
responsibilities across a number of services and products, and therefore

are less likely to gain the depth of knowledge required to understand

how to buy good research and to ensure that it is carried out properly,
and to judge the value of the results or the quality issues which arise.
Whether of professional status or not, buyers and users of research do

not work within communities where the values of research predominate.
This can lead to tensions which could potentially affect and diminish the
quality of work supplied and used. As non-researchers, the ultimate users
and buyers of research findings may not understand the issues that affect
the quality of output and may for various reasons pressure research
companies to compromise on aspects of the research design or process
that (the research companies consider) are critical to producing work of
adequate quality. This may take the form of contracting timetables to
‘impossibly’ short deadlines – beyond some point speed is at the expense
of quality – or limiting budgets to levels which are too low to finance
adequate research. There is also the policy in some organizations of
buying research through competitive tendering – effectively on price
alone.
Non-professional researcher
Professional market researchers are in the minority of those employed in
research; the majority are various other ‘non-professional’ staff. Market
research companies of course employ non-researchers for non-research
functions (e.g. finance staff), but the groups we really have in mind are
those involved in the primary data preparation and collection process,
particularly interviewers. The latter in turn are split between ‘field’
interviewers and phone unit staff. In the UK, circa 6,000 researchers are
members of the professional body but of these only approximately 200
are interviewers. This is in sharp contrast to the estimated total of 25,000
interviewers, of which around 5,000 carry out interviewing on a full-time
basis. The status and position of interviewers raise some particular quality
issues.
In Chapter 8 the quality issues arising from field and phone interviewing
are discussed in some detail. For the moment it is enough to mention
that field interviewers are recruited without any formal qualifications
(except basic qualifications such as literacy), receive only limited initial
training, work remotely from the designers of the research, cannot be
directly supervised for more than a very small part of the time and, as
essentially casual workers, commonly work for a number of research
companies (sometimes even in the same week). They also have limited, if
any, promotion prospects and the rates of pay offered are certainly not
high by most standards. Given this situation, company loyalty is unlikely
to be high. Even the local supervision of field interviewers may be by
staff who themselves often are working on a casual and freelance (if
better paid) basis for more than one research company.

Trends in the market research industry
This structure of non-professional employment grew up in the ‘cottage

industry’ phase of the market research industry, and by minimizing
overheads has probably contributed significantly to the successful and
long term growth of market research.15 Legislation – for example the
Working Time Directive in Europe and comparable legislation in other
parts of the world – has contributed to greater rights for interviewers,
for example the requirement that they be paid holiday pay, but this has
not necessarily meant that the relationship between interviewers and
research companies is any stronger. It is not hard to imagine that this
structure creates some quality issues for the raw data on which any claim
to research validity must rest. Problems include both variance in the
administration of data collection (e.g. misunderstood instructions, wrong
respondents, inadequate response recording) and some level of
malpractice; in the extreme form, completely fabricated interviews. To
attempt to address these problems the research industry responded with
the introduction of a number of data collection quality process standards
and this, as well as the underlying problems, is discussed in Chapter 8.
This approach has also been extended out to other areas of the research
process and this is discussed in Chapter 9. Another way these issues have
been addressed (if indirectly) is through the increasing use of
self-completion data collection and especially via online research but this
leads to other quality issues.

Although well established, market research continues to be very
methodologically dynamic. Along with an overall increase in size, the
industry has expanded in the range and sophistication of services offered.
The reasons for this are threefold: the industry and profession developing
new research tools particularly as a result of new technology, the need
for businesses generally to address new types of marketing research
problems, and client demands for non-research services, such as
consultancy services from research organizations. There is also a new
emphasis on service response – itself part of the general quality
movement. This has seen research organizations become more business
focussed and less ‘academic’. However, this trend has its own quality
tensions; service quality is a fine thing but there is a continuing need to
offer research integrity and validity whether or not clients are explicitly
concerned with such issues (they ‘need’ it whether or not they ‘want’ it).
15
The revenue growth rates in market research were until the 2008-09 global recession the
envy of many other business sectors.

Trends in research structure

An obvious structural trend of the industry is the continued consolidation
and globalization of research organizations within larger media groups,
resulting in continued growth of the larger companies and increasing
scale and complexity of their operations. One facet of this is the now
quite rapid escalation of technology-based research. Although the split
between quantitative and qualitative research methods has been fairly
consistent (see Figure 1), within this there are some significant
technology-related changes. The most dramatic is the rise of online
research, which in 201016 accounted for 22 per cent of total research
spend, and other automated digital/electronic forms of research (e.g.
retail audits) accounting for 14 per cent. Such technology offers
advantages in speed and, to a large extent, cutting out the role of the
interviewer.
However, other aspects of new technology present huge quality issues

which are dominating many industry conference discussions and
initiatives. The more traditional forms of data collection – telephone and
face-to-face interviews – have been partly supplanted by the growth of
online data collection (telephone and face to face interviews were only
15 per cent and 12 per cent of research spend in 2010)17, although these
forms of data collection will still continue to be used as a complement to
online research approaches. The growth of large agencies may also
impact on the working conditions and professional role of internal staff
with a diminished or fragmented role for the research executive and a
greater division of labour. Any decline in the numbers of ‘all–round’
researchers will have an effect on research as a profession.
It has been argued that the growth of the largest agencies has been or
will be at the expense of the medium-sized company – the organizational
type which is responsible for many of the established structural forms of
market research. However, medium-sized agencies have not been
squeezed out. What have emerged are medium-sized groups, created
through the acquisition of small and medium enterprises; these groups
are smaller than the large research groups but are of sufficient size to
compete, although they tend to specialize in specific sectors and/or
countries. At the other end of the scale new, small agencies continue to
be formed, particularly as a result of the 2008/9 recession and downturn.
As in the past, most of these will remain small ‘boutiques’ but others will
move up the scale.
16
17

Trends in client needs

In parallel with changes in suppliers, client needs are also changing.
Clients want research to deliver more, to deliver ‘insight’. To achieve this
clients want to be closer to the research, particularly to the outputs.
Clients want to talk directly with their customers; they want to hear
first-hand their customers’ views and they are no longer necessarily
interested in receiving this via the filter of a third party. It is increasingly
unlikely that the whole research process will be conducted within the
confines of a research supplier. Suppliers may provide frameworks to
manage client relationships with customers, building routes of access to
respondents via online customer panels and so on, but clients want to be
directing the discussion with their customers. Parts of these discussions
are research discussions, but increasingly platforms such as customer
engagement panels and social media sites are being used for
conversations and purposes beyond the parameters of research.
Customers are now seen as an integral part of the innovation and
development process, to re-shape brands to their own needs and to
provide input into the creation process. This has led to the emergence of
activities such as co-creation where market research, marketing and
branding disciplines are being combined to create new forms of
engagement with customers. These developments are not restricted to
commercial market research. Within the public sector, comparable citizen
engagement programmes are being undertaken for consulting and
engaging the public in public policy decision making, and results are
being used as part of behavioural economic models.
Almost inevitably, with increasing numbers of clients wanting to talk

directly with their customers, and buying research becoming more
difficult due to procurement concerns, more non-research practitioners
are doing it for themselves. The creation of research tools, particularly a
number of online survey software and questionnaire packages, has
assisted this process. On the surface the results can look as valuable as
more expensive, well-conducted research. But much literature has been
produced on questionnaire design, and the impact wording can have on
research results (Ian Brace (2008) is one example from the literature). The
amount of research being conducted by unskilled non-practitioners is an
increasingly significant factor in research quality.
Finally, it should be considered that not all important trends affecting

market research will arise within an enclosed industry. Market research is
part of a wider information and data service industry and competes with
other types of data and information providers that operate with different
conceptions of quality and within different ethical codes. Also, market
research interacts with the whole of society in the sense that it is entirely
dependent on public cooperation for its ‘free’ input of raw data. But
such cooperation is declining; research response rates are falling,
researchers are increasingly reliant on panels to access willing
respondents and respondents are increasingly aware of the value of their

opinions and expect researchers to compensate them accordingly. In a

way, research is a victim of its own success – so much research is carried
out that those willing to be respondents are over-researched and the
approach of the industry to falling response rates, particularly the
creation of respondent access panels, has created new quality issues. The
quality of the research experience often leaves much to be desired from
the respondent’s perspective and this is another quality issue that
professional researchers need to consider.

3 Quality and truth
In this chapter we expand the concept of quality, including contrasting

‘quality as excellence’ and ‘relative quality’ (‘fit for purpose’). The
purpose of market research is generally to not only be thought to be
useful to decision makers, but to be actually so by providing ‘true’
information. We explore the concept of truth in research including its
link to the methods of natural and social science and the limitations of
this paradigm, such as including the need for ‘understanding’ versus
‘explanation’. The concepts of validity and reliability of research methods
are particularly highlighted. Whilst some of these subjects may appear
rather theoretical and distant from the day-to-day concerns of market
research, they imply problematical issues that are relevant to practical
steps in achieving quality.
In everyday life quality is thought of as a form of excellence built into

things. Rolex watches, Ferrari cars and Stradivarius violins are all believed
to have it; they are amongst the best in their fields. To some extent this
quality is defined against tangible yardsticks. The watch keeps good time,
the car is very fast and the violin plays accurate notes. But the quality in
them is more than just utilitarian; possessors and envious would-be
possessors have a strong intuition of these objects’ excellence and
quality.18 This view of quality cannot be dismissed but it is hardly a
practical tool to improve market research practice or business generally.
Few would say that an economy Multi Purpose Vehicle (MPV) has the
quality, in the above sense, of a Ferrari, but if you want to carry your
children about or pick up compost from the garden centre it actually
better meets your needs. It matches a practical specification; it is ‘fit for
purpose’. Quality, in this sense, is relative to the requirements we have.
The MPV is much cheaper; quality as excellence has a cost which may be
too high for the intended purpose. At least in business decisions, we
cannot be carried away by intuitions of excellence and we need to take
account of what we spend on inputs with an effect on the end profit.
In market research what, in general, are the relative quality requirements

that need to be met? Whatever else, market research is a practical
activity, hopefully providing useful input into the business
decision-making process. Effective research produces information and
18
To pursue this aspect of quality further, Zen and the Art of Motorcycle Maintenance (Pirsig,
1999) is recommended.

3 Quality and truth
insight which should contribute to better marketing strategies and

tactics. Identifying opportunities by understanding how consumers feel,
think and behave, predicting consequences of different marketing choices
and testing new product ideas are examples of practical requirements
met, and of market research being useful. Of course carrying out research
has a cost and this may limit the depth of the research carried out. The
methodology may produce data of potentially low accuracy and statistical
robustness, but it may still be adequate enough to be useful; it may be
‘fit for purpose’. The research needed for high accuracy may be
unaffordable compared to the investment risk.
So quality in market research is tied up with the output being useful. But
is being useful enough? We need to distinguish between outputs which
are thought to be useful and outputs which are actually useful. Good
decision making may be based on information but the decision still has
to be made; we cannot prevaricate forever. Market research can provide
a crutch so that some decision is made and is, therefore, thought to be
useful. It is a catalyst to a decision being made. But to be genuinely
useful the information produced through research also needs to be true
in some sense. In designing a research project we need to ensure that
data sought is relevant to specific marketing decision-making
requirements – to be potentially useful – but we also need to ensure the
methods are capable of getting at the truth. Quality in market research
is, therefore, bound up with truth and we next discuss the basis of this
truth.
Before moving on, it is worth making a couple of other observations. The

first is that it may be easier to see that a research project is not useful
and relevant than judge whether or not the outputs are true. Much
criticism of research providers by research users is in this vein; researchers
are not telling us what we need to know to improve our competitive
position and increase our profits. Indeed researchers are accused of being
wrapped up in methodologies rather than meeting the clients’ useful
needs although ‘getting it wrong’ – not providing the truth – is not
wanted either. Second, it is all very well uncovering the truth, but it
needs to be communicated effectively and this is another area of
common complaint; researchers too often do not present findings
effectively and, therefore, usefully.
Truth, science and research

The methods and tools of market research are drawn from eclectic
sources (see Dexter, 2007) but generally from the social sciences and this
is as much the case in qualitative research as quantitative. Nearly all the
fundamental methods of collecting, analysing and interpreting data were
originally developed in one or other of the social sciences. In turn the
social sciences are at least linked to the natural sciences and have to

Truth, science and research
some extent adapted the broad purposes and logic of the latter. It is,
therefore, worth thinking about how science seeks the truth and initially
we focus on the natural sciences.
The basic methodology of the natural sciences, sometimes referred to as

‘positivism’19, is uniform regardless of the subject of investigation
although physics is generally regarded as the exemplar. Explanatory
theories are the desired result of science and the more general it is – the
wider the range of phenomena explained – the better the theory. Most
of physics is now covered either by general theories of relativity or of
quantum mechanics (the Holy Grail being, of course, a unifying ‘theory of
everything’, which as yet hasn’t happened). Such scientific theories are
developed by the process described as ‘hypothetico-deductive logic’,
illustrated in Figure 2. A hypothesis – a tentative theory – is posited
which is capable of deducing a prediction of observable phenomena. A
method is then devised to test whether the prediction is true or, as per
Popper (Popper, 1959), a test is devised that is capable of falsifying the
prediction. If the outcome of the test matches the prediction (or is not
falsified) the theory is regarded as provisionally proven (although it may
be later shown not to be true in some situations or only a special case of
a later developed more general theory).
In some parts of natural sciences the observations testing a hypothesis

take the form of a controlled experiment, where we can be confident of
the chain of cause and effect by controlling all variables except those of
interest. However, even in the natural sciences, the observations used to
test the hypothesis may have to be taken from nature rather than in a
controlled, artificially closed laboratory system. Much of cosmology is a
case in point as well as climate change science. Although each branch of
science has developed specific methodologies for hypothesis testing,
progress also involves devising new techniques and experimental design;
methodology evolves as much as substantive knowledge. Good science
entails good methods.
One important point from Figure 2 is that the logic of the scientific
method is not based on induction – generalizing from a number of
observations to form a theory. Hypotheses to be tested may well be
developed in this way – there is no ‘correct’ way of thinking them up –
but testing proceeds by deduction from the theory to a specific
prediction that can be tested through observation. A hypothesis that is
incapable of leading to a testable predication is not valid science.20 Other
general observations about the scientific method are that much scientific
effort has gone and still goes into description rather than explanatory
theory and that conceptualization is very much part of the scientific
19
Natural scientists probably rarely refer to ‘positivism’; more often it is a term of abuse by
some schools of social science.
20
Cutting-edge physics may be the exception with nine dimensions and parallel universes but
is criticized by some on just these grounds.

3 Quality and truth
Figure 2 – Hypothetico-deductive logic
enterprise. A lot of geology, for example is descriptive rather than

explanatory; the classifications of various forms of rocks and how and
where they lie are important parts of this branch of science alongside the
causal or historical explanations of how things came to be as they are.
Similarly much of biology is still descriptive. The point about
conceptualization is that much of it is beyond that which is apparent to
‘common sense’. The concepts of physics and many other areas of science
are conceptualized mathematically, and translation into words may be at
best imprecise or at worst impossible. Much of science is beyond or
contrary to common sense.
Finally in this brief discussion of truth through science is the question of

communication. Established science is communicated to its students
through textbooks, lectures and practical demonstrations in the
laboratory. New science – discoveries – may be unveiled at conferences
and other meetings but is formally communicated through peer reviewed
papers published in academic journals. Two key and linked aspects of
these processes are transparency and replicability. A paper not only shows
what has been found (the results) but, in detail, how the work has been
done (the full methodology). Others in the field can then replicate the

Truth in social sciences
work to test its validity for themselves as well as review the logic of the
chain from hypothesis, through observations made (the experimental
design) to the conclusions drawn; this process of testing is why
transparency is needed. Replication implies that the same results should
follow from the same methods regardless of who does the work; in this
respect at least the experimenter does not affect the results and science
in this sense is value free21.
The above is clearly a very sketchy and inadequate discussion of the

method of the natural sciences but it is sufficient for the purposes of
talking about truth in market research. Some themes we return to are
the hypothetico-deductive logic versus induction, explanation and
description, observation and experiment, transparency and replication
and the neutral, value-free researcher.

Social sciences originate from the attempt to apply the methodology of
natural science to the social phenomena. This is commonly referred to as
the positivist approach and has been pursued by some schools from
Auguste Comte in the early 19th century up to the present. Quantitative
market researchers are heirs of Comte whether they know it or not.
Social science was, in this paradigm, to seek general theories, tested by
rigorous observations if not by experiment, with the social world a
continuum of the natural.
Positivism in social science is not and never has been without its critics.
Whether or not positivism works in natural sciences (clearly whatever the
method used, hard science does work, as we all know in our daily life) it
is inappropriate to all or many of the tasks undertaken in social science.
The criticisms are many and varied. For example, it is claimed that the
objects of social science are just not comparable to phenomena in the
natural world and that value judgements (a no-no in natural science),
rather than being outside true knowledge, are largely the subject matter
of social science. Who does the work in much of social science is
inevitably relevant and this limits replicability. It is argued that the social
world can only be known through inevitably ‘slippery’ concepts, e.g.
‘market leader’, ‘brand share’, ‘consumer attitudes’ – and how these are
chosen and elaborated determines all else. The findings are limited to
what can be fitted within the conceptual framework chosen (though this
can be argued for the natural sciences as well). Additionally, social science
concepts have ‘meanings’; meanings to the observer who selects them
but also very often meanings to the observed, and both draw these
meanings from the social life they share. Even the language used to
define the concepts is based on social life. Value-free social science is a
21
Perhaps not wholly true in quantum mechanics – i.e. Schrödinger’s cat.

3 Quality and truth
non-starter. Also, the idea of phenomena awaiting discovery in the social

context is to put the reality upside down; social phenomena are
constructed, not found, and are intrinsically linked to the theory used to
‘explain’ them.
Theory is also more complex than positivists allow or than is used in

natural science practice since, as well as explanatory hypotheses, theory
includes conceptualization. This covers most ‘models’ and ‘paradigms’
although these terms are also applied to hypothesis type theory –
‘predictive’ models invariably imply some sort of causal relationship
between variables. Theory also covers various sorts of ordering
frameworks such as many descriptive schemata used in social sciences.
Conceptualization is in fact argued, by some, to be the main theoretical
output of social science rather than true explanatory hypotheses which, it
is said, are in fact few and far between compared with the position in
the hard sciences. Some claim that the only reliable general theories
about society are no more than trivial commonplaces which we do not
need an elaborate methodology to discover. Others (e.g. economists)
disagree.
Even if general explanatory hypotheses can be developed in the social

sciences, testing them is a rather different matter than in the natural
sciences. Experimentation is generally not possible (though it is not
possible in some natural sciences as well) and various sorts of survey are
not an equal substitute. Which variables are held constant and which are
changed is nearly always problematic in any survey work. This is why
reliable prediction in the social sciences is so difficult (but so it is in
weather forecasting or climate change). More radically, some critics of
positivism would claim that explanation as per the natural sciences
should not be the aim of social science. What should be sought instead is
‘inside’ understanding based on meanings attached by the subjects of
study: people. After Hammersley (Hammersley, 1993), the two approaches
to truth in science – explanation and understanding – can be represented
diagrammatically as in Figure 3, which includes some terms commonly
used in each approach but which we will not explore any further. Social
scientists may straddle both arms of truth but usually go down one
branch or the other. Finally to be noted is that induction is common in
social science and is used to justify claims to truth – a range of data are
examined (data mining) – and generalizations are made.
Market research – at least quantitative market research – historically has

a strong positivist strain whether or not this has been consciously
recognized. One of the most uncompromising advocates of this approach
was Mick Alt (Alt and Brighton, 1981). He argued strongly for the
application of a strict hypothetico-deductive methodology to product
testing or in any comparable works and castigated inductive methods –
such as ‘data-dredging’ (looking for relationships in the data after the
event or survey) – which were said to be a comforting approach but

Figure 3 – Understanding and explanation
lacking in plausible logic. Even within the quantitative school of research,

however, inductive logic has its defenders. If only at the level of practice
– what quantitative researchers actually do – the strong positivist claims
have to be regarded with some doubt.
Most market research carried out does not even approximate to any
hypothetico-deductive methodology and induction largely rules.
Data-dredging approaches are, for example, how many researchers pass
much of their time and the output is prized by users. However, there are
other criticisms of the positivist position to consider. One is that most
market research, including quantitative, arguably does not even seek
‘explanation’. Description is considered quite sufficient. Projects
concerned with market structure analysis, user and attitude studies, and
customer satisfaction surveys, for example, are largely limited to
describing how things are rather than attempting to explain why they
are so. Even predictive modelling is mainly ‘empirical’ rather than truly
explanatory.
Most market research is more akin to natural history than modern

biology, but the output of such work – whether or not it ‘explains’ – is
widely considered to be practically useful and adequately true. Another
point against positivism is that market research has not built up any
substantial corpus of general and explanatory theory; such theory as
exists tends to be at the level of conceptualization. Seldom is the stock in
trade on offer – from market research service providers – theories of
consumer or market behaviour that can be applied to a practical

3 Quality and truth
marketing problem. If, therefore, the positivist project – restricting the

truth to that which fits the paradigm of natural science – is the right one
for market research, the discipline has very weak foundations. Finally,
even if quantitative research in some respects approximates to the
positivist paradigm, qualitative research self-evidently does not and
instead substitutes a quite different perspective to acquiring marketing
knowledge.
Qualitative research and understanding

It is perhaps significant that much of the discussion of the
epistemological basis of market research has taken place within the
context of qualitative rather than quantitative research even though
qualitative research accounts for a much smaller proportion of global
research spend than does quantitative research (because qualitative
research methods are so labour intensive it accounts for a rather higher
proportion of professional time). Is it that qualitative researchers are
more inquisitive and sensitive compared to number-minded and
unreflective quantitative researchers? Qualitative research has often been
subject to implicit attacks in respect of its claim to validity, especially over
the ability to generalize its findings, and so needs some theoretical
defence.
Almost by definition, qualitative research has been concerned with

explanation at some deeper level and this is often characterized as
‘understanding’ with a focus on meanings attached by consumers to their
market experiences. Rather than ask how much more beer than wine is
consumed, or even at a simple level why one is preferred over the other,
the qualitative researcher may wish to understand the meanings
consumers attach to beer versus wine and on this basis produce useful
marketing knowledge – e.g. to plan advertising approaches.
Rather than positivism, qualitative researchers often subscribe to a more

‘humanistic’ approach with considerable emphasis on the researcher’s
softer and hands-on skills in conducting the research face to face with
the subject of study. There are various schools within this general
humanistic approach including hermeneutics, semiotics, historical analysis,
and even postmodern market research. A common thread is what is
described as the ‘linguistic turn’:
The study of language tells 20th century humans who they are ... the
disciplines dealing with language and other aspects of
communication hold the key to understanding human reality ...
marketing research must move towards an ever greater participation
in this linguistic turn.
O’Shaughnessy and Holbrook, 1988

Validity and reliability in market research
All this is a bit rarefied and it would be wrong to suggest that the
average qualitative researcher spends his or her time worrying about
such matters. In fact most will have never heard of them at all. However,
that is not to say such theoretical concerns have no impact. They do, if
only through a slow trickle down. Researchers may well use, for example,
a semiotic framework without even knowing the term. Also, most users
of qualitative research will judge the outcomes in only the fuzziest way
and rely on their confidence in the practitioner. Good qualitative research
is widely regarded to be that done by good qualitative researchers; it is
inevitably subjective and you either accept this and its consequences or
you reject such research. One consequence is that two first-rate
qualitative researchers working to the same brief may well come up with
two quite different sets of findings (this is also an argument for having at
least two researchers on a project). However, quantitative research also
has an arguably subjective element (including in questionnaire design)
and, therefore, it is not so simple a matter as good and valid quantitative
research versus unreliable and dubious qualitative work.
Not so long ago, qualitative research was almost synonymous with focus
groups (group discussions) supplemented with occasional depth
interviews with small samples (rather more costly than groups). More
recently, a wider range of techniques are in vogue including groups and
interviews online rather than face to face, ethnographic studies (really a
revived old methodology, originating in social anthropology), and
monitoring and ‘data scraping’ online social media (e.g. Facebook). None
of these, however, change the more fundamental questions of qualitative
research and how the truth is found.

In both qualitative and quantitative market research two critical aspects
of theoretical justification and truth are validity and reliability. Figure 4
illustrates the links and is discussed below.
Addressing validity
Validity relates to the subjects of any research study and the fundamental
question here is whether what we think we know about them truly
corresponds to how they really are, act or think. It would be a simple
matter to establish validity if we could in some way lay down the
observations made or measurements taken of our subjects alongside
them but we cannot do this. We can only know the subjects in terms of
such observations and measurements and in an absolute sense, what is
true, over and above any observations, is unknowable. The best we can
do is to increase validity through using effective tools (effective on the

3 Quality and truth
Figure 4 – Truth, validity and reliability
basis of both theory and practice) in the belief that good work comes
from the successful application of these tools.
Observation and measurement tools (e.g. questionnaires)
In most quantitative work the key tool is the questionnaire and

maximizing the design of an effective questionnaire is one of the first
steps to a quality project. Questionnaires and the questions included have
to provide measures or observations relevant to the purpose of the
research and capture true data from the respondents. Both requirements
of questionnaires and questions are difficult to get right, and particularly
the latter. The equivalent in qualitative research is the discussion guide
and stimulus material – although in some ways the detail is less vital
because there is no intention to use these in any standard or fixed way as
there is with a quantitative questionnaire. While it is often said that
questionnaire design is as much an art form as a science, perhaps it is the
artwork element which is often the problem. In ad hoc research,
questions are typically either recycled from other research projects or
made up afresh (maybe by quite junior researchers) without any attempt
to establish whether they are capable of capturing the measurement or
observation sought. Basic training may help the designer to avoid the
obvious pitfalls of ambiguous wording, two or more questions in one,
leading respondents etc. But even when these basics are covered, there is
still often uncertainty as to whether the data sought can be captured by
the question or questions.
No questionnaire, however perfectly designed, can capture every opinion

respondents may have on a topic. Piloting, which can increase
confidence, is either not done (for cost reasons) or done so superficially
as to be worthless. Only where the complexity of questioning is
acknowledged, is piloting generally regarded as important. Designing

questions and questionnaires is, therefore, neither easy nor an area

where the profession can believe that anywhere near the optimum is
commonly attained. Bad questionnaire design may also have a longer
term insidious effect; boring, repetitive questions put respondents off
and make them less likely to cooperate in future research which in turn
lowers response rates.
Definition of subjects
While much has been written on the measurement and observation tools
of research, there is surprisingly less apparent concern about the need to
use these among relevant research subjects. It is no use having the
perfect questionnaire if it is asked of respondents who have no
involvement in the decisions being studied or whose attitudes or beliefs
are quite irrelevant to the objectives of the research. Ask the right
questions certainly, but ask them of the right people. The right
respondents may be clear-cut but in some projects insufficient thought is
given to the problem; pilot research may be needed to provide a valid
solution before the main stage of fieldwork begins and this is seldom
undertaken. Business-to-business research is an area where respondent
qualification is often a special concern. In qualitative research the
question of whether the respondents are ‘right’ is usually transparent in
groups or depth interviews but in larger scale quantitative surveys the
extent of any problem may remain hidden.
On top of this there is also the perennial problem of respondents not

always being the best judge of their own actions and opinions, or they
may be unreliable witnesses (see Kearon and Earls, 2009). Newer online
self-completion techniques also raise related issues of the ‘right’
respondent: can we be sure the respondent is who they claim to be?
There are also problems of ‘true self’ in areas such as social media
research; the respondent may be role-playing, as in other online sites,
rather than giving ‘true’ responses. Also, the samples have an inevitable
bias to the online population and such issues need recognizing and
addressing to produce true research findings.
Minimizing variability of observations and measurements (e.g.

interviewer bias)
The issue of minimizing variability between sets of observations –

consistency in the administration of questionnaires, etc – is also regarded
as a more relevant issue in quantitative than in qualitative research
(where variability is minimized through having a single
moderator/interviewer both seek and interpret responses). The main
extraneous variable in this case is the interviewing team; however well
trained and briefed, they are people and not machines whose variance
can be strictly controlled. Apart from training and briefing, validation
techniques can be used to at least attempt to measure levels of variance
and interviewer bias although in practice these tend to be used more to

3 Quality and truth
address a quite different problem: interviewer cheating. The use of IT

arguably also has a positive effect on variance; interviewers using
computer aided personal interviews (CAPI) or computer aided telephone
interviews (CATI) will be more likely to administer the questionnaire
consistently, and the use of monitoring, particularly in CATI, can also help
to identify any potential interviewer impact. Online interviewing also
raises comparable if different issues.
Internal consistency of data
A final aspect of validity is to look for internal consistency within the

data from the research subjects. A simple example is that frequency of
purchase, size of orders placed and annual consumption (in business
research) must all be compatible and various checks can be built into
data editing work to identify consistency problems. However, the
question is what is done once or if problems are found. All too often the
answer in quantitative surveys is very little, not even qualifications made
in the presentation of the results. Clearly any commitment to quality
demands that action should be taken. In qualitative research internal
consistency can be achieved by an iterative process where the moderator
or interviewer resolves inconsistencies in the responses during the course
of the interview or group.
Addressing reliability
While validity is ensured at the level of the research subjects, reliability
addresses the other major problem in arriving at the truth: on what basis
can we generalize the results to a wider population, which raises issues
of inference. In quantitative research the key to this is sampling and rules
of statistical inference, but there is no such obvious solution for
qualitative researchers. Few clients really want to know the opinion of 20
respondents, no matter how skilfully their deepest feelings have been
probed, unless they can be taken as representative of some wider group
such as all consumers of a product group, or those from a particular
social class.22 Even if the researcher does not formally extrapolate to a
wider group, this is likely to be done by others including the client.
Alternative bases for inference in qualitative research usually rely on
some loose assumptions that, even though the sample is small, it can be
taken as representative, providing the measures or observations extended
are not claimed to have any precision. These problems have always been
the bugbear of qualitative research and, to some, imply that such
methods should never or hardly ever stand alone.
22
In the UK, research has its own social class grading system which is primarily based on
occupational grouping; often in media articles you will find references to ‘ABC1’s’ to
describe individuals that are from the upper end of the social demographic scale with C2,
D and E at the other end of the spectrum.

Sampling
We will not enter into any exposition of either the theory or problems of
sampling within this chapter or even book. Sampling is the basis of
inference in quantitative research and the validity of any conclusions
drawn from the sample data to the general population (from which the
sample has been drawn) depends on the rigour of the sampling
techniques used. Sampling is based on statistical probability theory and
the necessary range of sampling error in any research has to be both
recognized and taken into account in drawing research conclusions.
Sampling error itself does not, however, compromise the pursuit of truth
and quality. It does, however, qualify it. The effective sample and sample
size is of course the achieved sample. Non-response is always a potential
problem and, with falling respondent cooperation levels, of increasing
concern (particularly as those that do respond generally are those with
more time e.g. the unemployed, young mums, retired, etc.). The
self-selection bias of self-completion research – whether by mail or online
– has always been seen as a limitation of that sort of research. However,
with phone interview response rates down to around 10 per cent of
those initially contacted, self-selection bias also affects administered
interviews.
Repeatability
Another aspect of inference is repeatability and replicability, which is tied

up with transparency. If findings from research are ‘true’ we would
expect to come up with the same results should the research be
repeated, even by other researchers, providing conditions are held
constant, which in practice may be difficult to achieve. One condition
that clearly cannot be held constant is time – a repeat by definition is
carried out at a different time – and at least some types of measures or
observations can be expected to change over even a short time, e.g.
public attitudes, opinion, etc. The mirror of this is that in continuous
research or projects done in waves, we hope that differences found are
the result of time and external factors rather than research variability.
That is why, in these sorts of projects, a premium is put on such things as
constant question wording even where doubts arise as to whether the
form in use is the optimum. However, this sort of research apart, the
problems of consistency of this type very seldom arise because while the
research may be repeatable in principle, in practice this is rarely done. In
comparison to hard sciences or even academic social science, such testing
in market research is generally low. This may be a particular problem
where results are generated by induction; the researcher examines tables
and discovers an apparent relationship between variables (e.g.
consumption levels and educational attainment) and builds firm
conclusions on this basis (e.g. about effective promotion tools). However,

3 Quality and truth
the relationship may be completely spurious and ought to be tested by

separate research, which in practice is not undertaken for reasons of
budget and schedule.
Qualitative research is frequently criticized because repeatability is not

possible, even in theory. Different researchers, with different perspectives,
are very likely to produce findings that are substantially different.
However, since repeat research is not in fact a feature of ad hoc
quantitative research, there is no practical difference in this respect
between the two forms of research.
External consistency
Reliability of research output can also be tested through consistency with

data from other and independent sources. Factual survey data, for
example, can be compared either directly with published information or
at least compatibility sought with linked data (e.g. the market size of a
niche estimated from a survey can be compared with published estimates
of a larger sector of which the niche is a part). Similarly, opinions and
attitudes from research data can be compared with behavioural data
(e.g. the views of respondents about their eating habits can be compared
with their actual food buying through the use of loyalty card data).
Quality of research results would be improved if this was attempted (if
indeed it is possible) more often and with electronic access of published
data it is increasingly easy to do this extra work. Consistency can also be
sought where other primary research results are available. This is clearly a
possibility in continuous research where big fluctuations in a reading
should alert the researcher to potential research problems, but is also
possible where there are both quantitative and qualitative stages.
Consistency in results at least confirms the ‘robustness’ of the data.
Reliability from outcomes
A final approach to reliability is to consider whether the research results

appear valid in light of the outcome of any decisions they have guided:
proof through action. Since market research must be a practical
discipline, validation through end results is a beguiling prospect but one
that is seldom followed. One set of problems is structural: the divorce of
research users/decision makers and research providers. In much ad hoc
research the research provider simply does not know what happens next
(unless it is very dramatic). Another practical point is that research
sometimes produces negative findings leading to inaction rather than
action: the product is not launched, the advertising treatment not used
or the market not entered. In this situation there is usually no yardstick
against which to test the research because nothing is done. The
exception is the spectacularly wrong research result where someone else
enters the ‘no-hope’ market and makes a killing. Proprietary models –
those developed as branded products by agencies – are often claimed to

Science, truth, and managing quality
be validated through outcomes. However, the ‘proofs’ offered cannot

necessarily be tested and may be more rhetoric than science.
Even where the research is followed by positive action, linking findings to

subsequent events is often not easy. The research data may have been
only one part of a jigsaw and perhaps only tangentially relate to the
decisions taken (e.g. a market structure survey may have suggested
possible options but these were not specifically researched before a
go-decision was made). Also there is the problem of the gap between the
plan and execution. The new product concept may have been valid and
the basic plan sound, but the way it was launched and sold guaranteed a
flop. Of course, if the new product is a success, the researcher will be
pleased to bask in the glory, but it may all be a case of the self-fulfilling
prophecy. Out of a number of alternative designs the research indicated a
clear potential winner. However, its success may be no more than the
marketing effort that was put behind this particular design; despite the
research, others may have potentially been even more successful but we
will never know.
Science, truth, and managing quality

So, where has this ramble through the epistemology of science and
market research taken us in a book claiming to offer guidance to quality
in market research? There are some useful conclusions to draw.
How market research fits into the wider methods of social science and
the scientific method more generally is a debate which will not end, at
least not in the foreseeable future. There will continue to be divergent
views. Much of market research is not ‘scientific’23 in the more rigorous
use of the term. Despite this, professional researchers need to think
about the issues and have a vision of how truth is reached. Good
research depends on good methodology and whatever research users
may say, the need to develop effective methods and critically debate
them has to be a continuing professional concern. Market research
projects need to be fit for purpose and, although this has to be linked to
costs and whilst high accuracy levels may not always be required, below
some level methods may be simply inadequate to produce usefully true
results. However, as in natural and social sciences, there is no final and
fixed list of valid methods. Many become well established and some
principles cannot be breached lightly but methods and techniques evolve
as part of a progressive discipline. In selecting methods the researchers
need to know what has been shown to work where and what has been
proven invalid – this is part of professional education and development.
23
In Germany market research is required to be carried out to ‘scientific principles’ but this
has a wider meaning than is common in English usage.

3 Quality and truth
However, specifying research design as a set of fixed standards, except in

very well defined fields (e.g. how readership should be measured) is
neither viable nor desirable.
One vital aspect of science that is directly applicable is the need for
transparency and replicability. Except rarely, the output of commercial
research tends not to be published in journals. However, the need for
users of the research to understand how the findings have been
produced and to be able to later replicate as well as evaluate them is still
vital. Even where a research user or client disdains to understand the
mechanics, professional research practice requires methodological details
to be documented and at least available on request.24
24
Proprietary research techniques and commercial advantage may sometimes raise issues
about limiting transparency.

4 Quality and the profession
Quality in market research depends on people; the practitioners carrying

out the work. On a continuum of occupations based on the skills
required, market research is at the professional end of the spectrum. This
chapter examines market research as a profession including forces
favouring or inhibiting the impact of professionalization on services and
quality. The structure and role of professional organizations in market
research is also covered including in relation to development of the
knowledge and skills of practitioners and the impact of this on quality.
Professional, ethical codes, with particular reference to that of the UK
MRS, are discussed as is the boundary and difference between such codes
and quality standards.
Professionalism in market research is often seen as the main

underpinning of quality. Some have argued, when the need for other
mechanisms has been debated, that professionalism is the only important
means of achieving quality. Our view is that the professional dimension is
certainly important, but that it primarily relates to quality as excellence
rather than relative quality, and that even in relation to excellence there
is a need for other approaches as well. However, the professional aspect
of market research and its links to quality are sufficiently important to
justify a separate chapter.
There is a temptation to start the chapter with a lengthy discussion of

whether or not market research is a profession in any true sense of the
term but this is all rather academic. It is more useful to see occupations
lying on a continuum, at one end of which lies the long-established,
recognized and regulated professions and unskilled casual occupations at
the other. Market research lies towards the professional end of the line,
although within this there is some variation due to the diverse nature of
market research functions and roles.
Professions and members

Professions are made up of individual practitioners and industries are
made up of the corporate suppliers that provide services. Traditionally a
professional body would represent the practitioners and trade
associations would represent the corporate bodies. The dynamic within
professional organizations is about the pursuit of practitioner excellence,

whereas trade association focus is about creating the most profitable

commercial environment for the suppliers within the industry.
One of the problems with this model for the market research sector is
generally that the professional bodies and not the trade associations are
the code-holders and regulators. The result is that the professional bodies
can exercise no direct control over research suppliers, and there is
inevitably overlap in both interests and activities and some potential for
conflict.
Professional bodies in the 21st century

Within market research the separation of the functions is no longer
necessarily the only model being followed. This is partly due to financial
necessity – the sector is not economically large enough to always support
more than one organization – but is also due to the changing role and
nature of professions within society as a whole. Research is not a licensed
profession in many countries, including the UK and the US; anyone in
theory could set themselves up as a researcher without necessarily being
associated with either a professional or trade association. More and more
individuals are entering the market research sector from non-traditional
routes, particularly online research where many panel owners and
managers are more likely to feel aligned to the IT sector than research.
Not only that, but the concept of a professional body is difficult to
reconcile with the aspirations of 21st century researchers; indeed this is
not unique to research and similar attitudes can be seen across many
other sectors. Why join a professional body for networking, training and
qualifications when all of these needs can be catered for elsewhere? The
‘competition’ has increased significantly through the emergence of
successful online business and social networks. Professional bodies need
to significantly adjust their business models to be able to actively
compete and give individuals something more to make participation in
the formal frameworks for the profession worthwhile.
The challenges facing professional associations are severe and in the

past decade they have, if anything, become more so. In the
non-statutory regulated sector, members have become more
demanding, less willing to accept subscription increases without a
clear idea of what they are getting and more demanding of services
tailored to their needs. Relations with members have become an
issue for strategic thinking.
Friedman and Mason
The Professionalisation of UK Professional Associations:
Governance, Management and Member Relations, 2004
There are numerous examples of market research professional bodies

which have combined with the research trade association (e.g. in the

Netherlands, UK, Canada and South Africa) or where the professional

body and trade association share costs of facilities or services (e.g. the
Australian associations). Throughout this chapter we will use the UK
situation as an example to highlight and discuss the dynamics that are
driving these changes.
Although professions are individually based and the older professions

developed on an implicit and assumed commercial structure characterized
by sole practitioners, most professionals, in all fields, increasingly work in
larger professional corporate practices or as specialists in organizations
whose core activities are unrelated to the particular profession. The latter
structure in particular can give rise to conflicts between the requirements
of the profession and the commercial ends of the organization. This may
be in relation to ethics or standards of work and quality of output. In
terms of structure and the position of professional staff, market research
is quite varied. Many research professionals still work as sole practitioners
or in very small research companies (in numbers the large majority of
research companies are small). However, more are employees of the top
20 or so research companies and groups or work as ‘client-side’ research
professionals. In either case, the potential for conflicts of ethics and
quality exists.
Characteristics of professionals in market research

Professions tend to seek inclusivity. They seek to ensure that all
practitioners carrying out professional-level work are brought into the
professional fold. Once well-established, professional bodies may seek to
define professionals in their own field as exclusive members and label
others as non-professionals without the necessary competence and skills.
Problems inevitably occur, however, where the performance of the

professional service depends substantially on workers that are outside of
the professional fold. Means of overcoming these structural problems
include having the workers’ output controlled and reviewed by full
professionals (e.g. in legal practices) or by the professionalization of
‘ancillary’ workers (e.g. legal executives and paralegals in relation to
solicitors). This problem certainly arises in market research with large
numbers of workers responsible for key activities (especially data
collection), but who are generally not fully supervised by professional
workers. Neither are they organized as an ancillary profession. Amongst
other issues this has some important quality aspects (see Chapter 8).
Another difficulty for professional inclusivity is specialization. While the

professional momentum is generally an outward movement from a core
of professional knowledge, the growth of specialized branches of
knowledge tends to lead in the opposite direction, to a fragmentation of
practitioners pursuing very different interests and activities and possibly
unable to communicate to each other in any meaningful way.

Specialization certainly exists in market research – qualitative research,

quantitative ad hoc and continuous research, suppliers and users, and
new and emerging techniques versus traditional approaches – but most
specializations have something meaningful and useful to communicate to
and learn from each other. There are also skilled specialists working in
market research but who are not fully part of the profession (they may
consider themselves to be members of other professions); statisticians and
technology specialists are prime examples.
Open professions, closed professions, and regulation

Linked to inclusivity is the extent to which a profession is open or closed.
Closed professions have rigid and demanding admittance qualifications
and commonly seek to prevent non-members from offering the service.
The very name of the occupation (e.g. solicitor, doctor, etc.) may be
restricted or sought to be restricted to its members. Such exclusivity may
even be state-backed and enforced via legislation. Achievement of closed
status, however, requires a claim to be pressed and accepted by the client
base and wider public, that the profession has some special knowledge
which is necessary to deliver the service to an adequate level of quality.
Open professions seek inclusivity by making entry relatively easy, based

on interest and some sort of commitment rather than formal
qualifications. However, the effect of this may be to reduce the standing
of the profession; if anyone can join, what is membership worth? The
historical trend is often to start as an open body but, once most
practitioners are included, to move towards a more closed status.
One strategy used to develop closed status is to develop various grades

of membership which reflect the different levels of experience and
competence of the members. Grade structures such as full and associate
are fairly common, with only full members given recognition as
professionals and to restrict voting rights in the organization accordingly.
MRS in the UK partially adopted this approach, with only MRS Full and
Associate Members plus honorary grades eligible to vote (i.e. not Affiliate
members), have the right to stand for Council elections, etc. The MRS’
decision to allow Associate Members greater status was primarily to
encourage younger members (who are more likely to be Associate
Members) into greater involvement with the Society and to participate
more in the Society’s governance. As a relatively recent experiment
completed in 2002–03, it is still too early to determine how successful this
approach has been and whether it addresses the wider issue of getting
21st century practitioners interested in belonging to and developing their
professional body (which by their very structure and nature feel more
19th century than 21st). However, with professional bodies having mixed

fortunes in today’s society, with some struggling to retain relevance25,

professional associations are increasingly likely to become more and not
less open, for example by offering some form of affiliate membership
status which is open to the non-professional with an interest in the
profession. This makes it increasingly difficult for professional associations
to marginalize the non-member as ‘unprofessional’, without appearing to
be hypocritical. Market research is no exception to these trends.
While closed professions may welcome some aspects of state

involvement, especially where it effectively grants a monopoly of practice
to members of the profession, regulation of how the professionals
perform their work is invariably strongly resisted. Indeed self-regulation
by committed members is usually a cornerstone of professional status and
imposed outside control is commonly seen as a failure of the professional
institutions (e.g. the legal profession in the UK lost its self-regulatory
status due to perceived failures in the effectiveness of its self-regulatory
system). State regulation tends to be most evident where the risks to the
general public are at their highest (e.g. in the medical and financial
fields). In the UK, there is no direct state regulation or involvement in the
market research profession although there are a number of pieces of
legislation that directly impinge on the practice of research (to be
discussed in Chapter 6). There are also some countries where legislation
directly regulates research. For example many countries (such as Italy)
already regulate how and when political opinion research can be
undertaken and other countries (such as Germany) regulate scientific
research. In the UK there has been a limited amount of interest in
introducing legislation to directly regulate research, although the sector
cannot be complacent on this issue. Research has become a commodity,
something to be used (and sometimes abused) for furthering business or
policy goals, such as the increasing use of opinion research in public
policy debates about the contentious issues of the day.
This change in how research is being used and why it is commissioned

has resulted in greater scrutiny being given to the quality of such
research. In 2002 this culminated in parliamentarians asking questions
about the robustness and coverage of self-regulation to control such
activities. This event was one of the factors that compelled MRS to take
more active steps towards creating a corporate self-regulatory structure.
The argument the sector was facing is similar to that which the financial
sector is currently battling, nicely summed up by Willem Buiter in his
article about financial regulation: ‘Ultimately, self-regulation stands in
relation to regulation the same way self-importance stands in relation to
importance and self-righteousness to righteousness. It just isn’t the same
thing.’ (FT.com, 10 April 2008.)
25
Financial & Operating Ratios for Professional Bodies: 2009/2010, Professional Association
Research Network (PARN) reported that in 2010 57% of professional bodies increased their
membership base whilst 24% saw a decrease and 19% no change.

The regulatory issue is not currently pressing in the UK but this could
change without any apparent cause. Online research, for example, could
trigger new concerns arising from more general issues about privacy in
the online age.
Professional ethics
Being a professional is supposed to involve more than merely following
an occupation or earning an income; a profession is supposed to be a
vocation or a calling. This implies that individuals would want to pursue
the activity regardless of reward. This is clearly not the case today; in
general, people become professionals to enhance their income, not
restrict it. Market research is also unlikely to be carried out purely out of
a love of the subject or for humanity’s sake. However, the concept of a
vocation or a calling still has some force and implies that even though
members of the profession may earn lucrative salaries, reward is in the
context of commitment to norms and standards set by the profession. A
professional is expected not to be motivated solely by money, even if it is
allowed within the framework of the general law. These restrictions are
supposed to protect clients and provide some guarantee of quality;
professionals are expected to deliver the best and the truth, or at least
strive to do so. This sense of calling is implicitly recognized within some
sectors of market research, particularly those where ‘craft’ still forms its
core. However, many professions, and research is no exception, are
experiencing a movement away from craft towards industrialization and
mass market production, from bespoke craftsmen making bespoke
products to a ‘flat pack’ approach by non-specialist, cheaper and less
experienced labour. In research terms this can be the difference between
some ‘cheap and dirty’ online data collection, and the large scale
longitudinal social research project, with the vast majority somewhere in
between this spectrum.
The ethics and standards of a profession relate to dealings with each

other as much as the outside world. Members are expected to deal with
each other fairly and with consideration even when they are
commercially in competition. However, it is all very well for members of a
profession to respect each other but are the interests of their clients as
well-served? Professions may stand in the way of client service and
become (or have always been) self-serving conspiracies, although within
market research there has been little reported evidence of such a
self-serving attitude. Arguably, a major factor limiting any such tendency
within market research is that it is still an open profession subject to
competition from those outside of the formal structure, including clients
doing research for themselves assisted by new online technologies. With
these factors in play it would be self-defeating for research to be too
restrictive. In the end it would be the researchers and their employers
who lose out, not the clients. Also it may be over-cynical to regard all

Knowledge and professions
professions as inevitably self-serving; Koehn (1994) argues that a true

professional will act in a way that serves their client and society as a
whole even at the expense of self-interest.
Knowledge and professions

Professions are all knowledge based; what the professional has to offer is
expertise in utilizing that knowledge to create the best solution. Nor is
this knowledge a mere collection of practical techniques; it cannot be set
out recipe-style in a cookbook. On the contrary, ‘full’ professional
knowledge, while obviously applied, is underpinned by a theoretical body
of knowledge and usually intertwined with academic fields of study –
some professions explicitly claim to be ‘learned’. Possibly not all
professional practitioners are as well versed as they might be in the full
depth of knowledge, but a profession as a whole has roots which can be
tapped as required and which have a constant interchange with practical
applications. In part this deeper knowledge results in an insight of what
is ‘true’ and so defines what quality is in a professional service.
We have, in the previous chapter, explored some of the theoretical roots

of market research in social science and shown that they clearly exist
even if they are contentious in part. There is also a distinct academic and
practical literature on market research through academic journals and
other publications worldwide. The market research professional bodies
also provide forums for the exchange and advancement of knowledge
and some long-term projects. As in other professions, there is a
commitment to not only have a basis in knowledge but also to advance
knowledge. There is an assumption of improvement and, therefore, a
striving for quality. Professions are more than learned societies, they are
vocational, rooted in practice, but they have this learned aspect to their
institutional structures and in this way a link into the wider intellectual
world.
As knowledge is at the core of professions, professional development (i.e.

qualifications and training), is an important aspect to ensuring
professionalism. Most research companies have recognized the need to
formalize their professional development, particularly if they wish to
attract the best candidates, and as such most will have some formal
structures in place, particularly for graduates. This ensures new entrants
to the profession are trained and that practitioners keep abreast of
developments or extend their range of skills. The attainment of some skill
level may be a condition of entry to the profession and for most research
companies this will be an undergraduate degree from a ‘good’ university.
Once within the profession, graduates will be expected to gain some
form of additional vocational qualification to prove they are competent
in their elected profession.

The most widely recognized qualifications are those of MRS, which

exports their key qualification, MRS Advanced Certificate in Market and
Social Research Practice, to over 40 countries. Most of the large research
companies recognize the suitability of MRS qualifications for their
business needs and have incorporated MRS qualifications into their
formal graduate professional development programmes. The benefit for
MRS, aside from increasing the uptake of its qualifications, is the access
to new, young researchers at the beginning of their careers, attracting
them into membership and the benefits of belonging to a professional
body. MRS, like most professional bodies, will recognize qualifications as
part of its grading structure, i.e. attaining recognized qualifications will
allow access to the higher grades of membership. This approach by MRS
is not unique; most associations will have some form of competence
framework which underpins their grading structure.
Of necessity, professions define a corpus of basic knowledge which

members should have. There are two more general points to be made
about the nature of knowledge within research. First, the intellectual
roots of market research are diverse and include most of the social
sciences as well as statistical and other techniques borrowed from both
the natural and the social sciences. These specializations can be deployed
practically in market research with only a limited additional knowledge
of the practices and techniques which are particular to market research.
As a consequence, insistence on studying market research in any depth as
a specific subject may deter entrants from other fields who can make a
substantial contribution on the basis of knowledge from outside the
profession.
The second general point about market research knowledge is that

however deep it may be, it tends to be underrated by the business world
in general and clients in particular. This point can be illustrated in
relation to questionnaire design. Getting a questionnaire right and
ensuring it is an effective tool is not an easy matter and calls for
considerable skill and expertise. But because they are ‘just questions’ and
questions are intrinsic to ordinary life, everybody is an expert, or at least
thinks that they are. The emergence of DIY research rather underlines
this attitude. In other areas too, market research is not seen as offering
much over and above ‘common sense’ or the sort of general abilities
expected from university graduates. This is not the situation in other
professional business services; e.g. business owners will not normally
argue with their accountant about how financial statements should be
put together. This situation inevitably affects the general standing of
researchers; very few large businesses have researchers on the Board (or
indeed marketers for that matter). Arguably it can also get in the way of
quality; researchers are not always trusted to use techniques that, while
effective, are not easily understood. Also the researcher’s diagnosis of
client needs may not be believed and instead work is geared to only
superficial wants. Although clients are demanding insight from

Professional organizations
researchers, they are not always open to what is required to achieve this,
nor for the control to be relinquished to enable it to be gathered.
Professional organizations
Professions are invariably organized as formal bodies with constitutions,
codes, management, and services to members and the public, and they
may own substantial assets. In the UK some bodies have submitted
themselves to greater scrutiny to obtain Royal Charters e.g. The
Chartered Institute of Marketing and The Chartered Institute of
Professional Development. However, organization and structure alone do
not define professions and some bodies cannot claim such a status. Some
professions have a number of competing bodies but in the UK MRS is the
only substantial body covering the market research profession. That is not
to say there are no other research bodies in the UK – there are numerous
bodies covering client-side researchers, social research, qualitative
research, business to business research, independent consultants, account
planners, etc – but MRS is the body which covers the broad church of all
those involved in research from whatever background or interest.
MRS was originally a professional body for individual research

practitioners. Established in 1946 with 23 founding members and with a
Code of Conduct since 1954, it was and remains the largest professional
body for researchers. In 2006, MRS integrated the services of the UK’s
research trade association, the British Market Research Association
(BMRA), which subsequently ceased to exist as an association, and MRS
became the business and trade association in addition to its existing role
as professional body and Code-holder. Today MRS is run by a team of
circa 35 professional staff – with support from volunteers in key areas
such as self-regulation – and has over 400 corporate members and 6,000
individual members. There are some significant benefits that MRS has
achieved through the integration of both professional and trade
functions. In addition to the economic efficiencies of having one body
rather than two, MRS has a much stronger voice in terms of media and
public affairs representation for research, and can directly regulate both
research suppliers and research practitioners, resulting in a significantly
more effective and robust self-regulatory system for the sector.
MRS is also a good example of a professional body that has developed its
business model to be less reliant on membership (currently membership
subscriptions represent less than 20 per cent of its total turnover) by
cultivating its commercial activities e.g. trade publications such as
Research magazine and the trade directory, Research Buyer’s Guide, and
their associated websites, www.research-live.com and
www.researchbuyersguide.com. This has the advantage of diversifying the
business base with the disadvantage that the commercial activities tend
to be discretionary (sponsorship, advertising, training and conference

attendance, and so on) and thus more responsive to general economic

fluctuations (professional membership tends to be more resilient during
economic downturns).
...Together these changes may be thought of as adding up to a

‘modernization’ or even a ‘professionalisation’ of professional
associations, they indicate a trend away from professional
associations as an informal club dependent primarily on member
subscriptions and the voluntary effort of members, and towards
associations that are strategic and business oriented. They also
indicate a trend towards associations that have a relationship with
their members that is more like a relationship with customers, at
once less personal, also more structured, subject to reflective
procedures and more concerned with the value they provide
members, or at least how the services they provide are valued by
members.
Friedman and Mason, The Professionalisation of UK Professional Associations:
Governance, Management and Member Relations, 2004
Another role for market research professional bodies is to represent the

sector, to be the ‘voice of research’, via public affairs and media
representation. Those professional and trade bodies that are effective in
this arena (in terms of promoting and protecting a sector) can most
clearly make an argument for the need for their continued existence.
Whatever doubts some may have about the need for professional bodies
– particularly when so many of the services can be attained from
elsewhere – this is clearly not the case in the area of representation.
Government and legislators want to talk to representative bodies that
can speak for the whole sector, rather than speaking with suppliers or
practitioners who can only speak for themselves. Strong and effective
representation and protection of a sector is an essential element to
success for any professional or trade body. Indeed in a changing world
where ‘the whole notion of professionalism has lost much of its authority
that it enjoyed in the mid-twentieth century’ (Friedman and Mason,
2004), the need has never been greater for professions and their
professional bodies to protect their sectors.
Codes and standards

One aspect of professional bodies in general is the role of codes of
conduct and practice, and the extent to which these differ from quality
standards. These have important consequences for quality. The following
discussion of codes is illustrated by reference to MRS Code of Conduct
although codes of other professional research bodies, in other countries,
are very similar in at least general principles.


standards
Figure 5 – Differences between codes, constitutions and quality
Codes and standards
53
Professional bodies usually have formal codes of conduct which set the
ethical principles and rules for individual practitioners; and trade
associations have codes of practice which set corporate rules. Such codes
may be referred to as professional standards but there is a difference
between codes of practice and standards. Codes are also different to the
constitutions of the professional bodies. See Figure 5 for the differences
between these three types of document.
Codes typically relate to how members of a professional body should

conduct themselves in their relations with each other, their clients and
with the wider world. They may also define some general principles
about the nature of professional work and its boundaries; in effect what
activities fall outside professional services – these may be forbidden to
members or it may be enough for members to make clear to clients
where the boundaries lie. Codes may include principles which will affect
how work is carried out and therefore define quality in some way. Codes
may go further and contain detailed rules which define how work or
specific aspects of it should be carried out, but in such cases codes are
incorporating some element of standards.
Key principles
The current MRS Code of Conduct (2010) contains 10 overarching
principles which are applied and interpreted via 74 rules split between
the responsibilities of the researcher and the research process. A
one-page summary of the MRS Code of Conduct key principles is
reproduced below.
The principles of the Code
These are the principles of the MRS Code of Conduct:
1. Researchers shall ensure that participation in their activities is

based on voluntary informed consent.
2. Researchers shall be straightforward and honest in all their
professional and business relationships.
3. Researchers shall be transparent as to the subject and purpose
of data collection.
4. Researchers shall respect the confidentiality of information
collected in their professional activities.
5. Researchers shall respect the rights and well-being of all
individuals.
6. Researchers shall ensure that respondents are not harmed or
adversely affected by their professional activities.
7. Researchers shall balance the needs of individuals, clients and
their professional activities.

Key principles
8. Researchers shall exercise independent professional judgement

in the design, conduct and reporting of their professional
activities.
9. Researchers shall ensure that their professional activities are
conducted by persons with appropriate training, qualifications
and experience.
10. Researchers shall protect the reputation and integrity of the
profession.
Source: The Market Research Society
The MRS Code in practice

A substantial part of the MRS Code relates to responsibilities to
respondents – people who are interviewed or otherwise contribute to
primary market research. Market research is based on confidential
responses which are used for research purposes only and not for other
activities such as sales contact. Traditionally, this meant that individual
responses were aggregated before being passed to the client and were
not identifiable as relating to a particular respondent. Many projects are
still conducted on this basis but the MRS Code also allows identifiable
research to be conducted, whereby respondents have explicitly consented
to be identified and the identifiable research results are passed to the
client. The basis for this is ‘informed consent’; respondents know exactly
what is going to happen to their data (i.e. who will see it, what it will be
used for) and have consented to participate on this basis. In most cases
there is in fact little point in reporting responses on an individual basis
since those interviewed are only representative of some wider population
which is the subject of interest, and the point of collecting data from
them is to provide a basis for generalization. However, in some cases,
such as customer satisfaction research or business-to-business research,
knowing individual comments can be useful. The Code’s requirements in
relation to respondent confidentiality have legal force through data
protection legislation (to which the Code specifically refers and will be
discussed in more detail in Chapter 6).
The MRS Code is also unusual in that it explicitly recognizes that

researchers may be asked to utilize their research skills for purposes other
than research, and MRS has separate regulations which stipulate on what
basis such exercises must be undertaken (MRS Regulations for Using
Research Techniques for Non-Research Purposes). These rules would apply,
for example, if a researcher is approached to conduct a mystery shopping
exercise for the purpose of gathering identifiable details about
regulatory adherence for the purpose of taking direct action, or if a
client asked for data to be collected for direct marketing purposes in
addition to research purposes. MRS Non-Research Regulations mirror the

ethical rules of the MRS Code of Conduct, and provide a framework for
all professional activities that a researcher may be engaged in as opposed
to just exercises for research purposes.
The key principles of the MRS Code of Conduct reflect the broadening of
researchers’ activities and apply to when researchers are conducting
research and non-research activities. In relation to clients, the MRS Code
generally avoids impinging on commercial considerations, although it
does stipulate the need for contracts between all parties involved in a
research project (research supplier, subcontractors, clients, etc.) as
misunderstandings in this area often give rise to complaints. Mirroring
respondent confidentiality, however, are requirements for keeping
information about or supplied by the client confidential and this includes
keeping even the identity of the client confidential (e.g. from
respondents); the fact that a particular company has an interest at all in a
market or product may be commercially sensitive. For various practical
reasons, quite apart from the client’s interest, researchers may prefer not
to identify clients. However, respondent data protection rights may
require client disclosure in some circumstances e.g. when respondents are
contacted from the client’s own database.
Standards and enforcement of the MRS Code

The MRS Code, in relation to clients, also contains elements that are in
effect standards and which lay down in some detail how some aspects of
the work should be carried out. The rules are structured around the
standard research process from research planning and design, data
collection, research reporting, etc. In particular, there is a requirement for
sufficient technical information to be made available to support the
validity of the results and any supporting interpretation that is made.
This is in recognition that the validity and ‘weight’ of research findings
can only be evaluated if it is known in some detail how the data were
obtained. MRS Code reporting requirements are, therefore, a standard
for the presentation of research findings and as such are concerned with
the quality of work. There are also fairly detailed requirements for the
retention and security of records from research (e.g. the original
questionnaires) and again these constitute a standard affecting the
quality of work and service. These comments highlight some aspects of
the MRS Code which are relevant to the subject of this book, but readers
should not take it as a summary of the MRS Code.
As with any such code, however, merely having it in place is no

guarantee that practitioners will abide by it. MRS has procedures for
enforcement of its Code up to tribunal hearings which can lead to the
expulsion of erring members. However, it is fair to say that only those
issues which are drawn to the attention of MRS are investigated and
there is no ‘policing’ in place to establish levels of compliance. This is the

Professions and standards
same with all other major research code holders, including international
bodies such as ESOMAR. Insofar as research codes and associated
disciplinary procedures impinge on quality of research, it is not a very
effective vehicle for giving clients assurance about quality levels and, to
be fair, this is not a primary purpose of ethical codes.
A further limitation of codes relates to knowledge of its provisions.

Whether or not they are enforced, research codes, like any other sector’s
code, can only be effective if practitioners and members are aware of
their provisions and even long-standing members may have less than
adequate knowledge. In the UK, MRS has actively attempted to rectify
this with its qualifications programme which covers the MRS Code and
standards knowledge as key parts of the syllabi.
Professions and standards

As discussed above, professional codes of conduct are not, as such,
statements of standards, although particular codes may well include
elements which are in effect standards. There is of course no reason why
a professional body should not develop formal standards which specify
the contents of the service or professional ‘product’ and, therefore, in
effect define key aspects of quality. Rather than develop such standards
within its professional body, the market research industry has chosen to
build them in other ways which will be discussed in later chapters.
However, it is useful to briefly consider the development of standards in
a different profession: accountancy.
Case study: Accountancy standards

Up to 30 years ago, how accounts should be prepared and presented was
largely guided by the general principle that they should provide a true
and fair view of a business situation. How this general principle should
be applied in a particular case and in a particular business was largely a
matter for the professional judgement of an individual accountant,
although this is not to say that every individual’s approach was
automatically accepted as valid. Inevitably, this led to marked variation in
the basis of accounting statements and to problems of comparability
between companies. Serious issues arose in particular from takeovers,
where the acquirer discovered that the interpretation of true and fair
applied by the acquired company was rather different to that used by
their own accountants. Faced with pressure from both outside and inside
the profession, and with some possibility of legislation being imposed, it
was agreed that the profession, through its various bodies, should
develop full and detailed standards of financial reporting. This decision
was, however, not without its critics within the profession, who
considered that their professional integrity and skills would be

compromised by having to work to a standard which might not always fit

the needs of particular businesses. The work in preparing the standards
was considerable and was guided by both the ‘best’ professional opinion
and a search for ‘true’ principles to incorporate. Inevitably, however,
there were elements of compromise and arbitrariness in reaching final
drafts of the standards.
Today, a comprehensive range of financial reporting standards is in place,

many of which have international foundations in addition to domestic
concerns. They are issued and revised by a body set up for the purpose –
The Accounting Standards Board – which has replaced earlier
arrangements within the UK. Compliance with these standards is
compulsory within the profession and to all intents and purposes must be
complied with to meet the Companies Acts. Arguably, the effect of this
move to standards within this profession has been to raise the quality of
financial reports to the benefit of the business community at large.
However, the standards intentionally constrain how professional
accountants will work for their clients and undoubtedly there are many
cases where the latter’s ‘wants’, if not ‘needs’, are not met through strict
adherence to the standards. The introduction of standards considerably
reduced the freedom of financial reporting and, in some cases, give a less
true and fair view of a business than would have been the case under
the previous, laxer regime.
An important point about accountancy standards is that they specify the

nature of outputs produced and lay down minimum quality levels for
these. Within the market research industry there are some parallels but
only to a limited degree. The standards which have been developed by
IQCS, MRQSA and ISO (see Chapters 7, 8 and 9) contain some output
requirements but, in general, are far more concerned with how the work
should be organized and managed than with what is produced. There
are various reasons for this and not least of which is a lack of consensus
about what constitutes a ‘good’ output. This relates to the problems of
‘truth’ as discussed in the previous chapter.
Professionals and their clients

The discussion of professions so far in this chapter has been focused on
the professionals themselves. However, a justification for a professional
organization and status is to safeguard the interests of clients. The basis
for this is the assumption that the professional has some privileged and
expert skills and knowledge to which clients do not have access. From
this principle, it follows that clients often lack the skill or knowledge to
recognize their own needs (rather than ‘wants’) in relation to the
professional service and it is part of the professional role both to identify
and diagnose these needs and to design a service which meets these
‘true’ needs. Professional work therefore always involves some element

Professionals and their clients
of design, using this term in a broad sense (e.g. a solicitor planning the
conduct of a case is carrying out ‘design’ work).
Whether or not a profession is generally successful in working this way

depends on various factors. One is that the relationship between the
professional and client and the context in which this takes place
facilitates the identification of ‘true’ needs. As a minimum, this implies an
ongoing link and opportunities for the professional to be fully aware of
the client’s situation. This also assumes that the client knows to some
degree what they want and are able to summarize and communicate
these needs through the research brief and subsequent discussions. In
much ad hoc market research this is not the case; the practitioner has
very limited opportunity to diagnose needs and instead largely responds
to a brief prepared by a non-expert client (although this is clearly not the
case with briefs prepared by client-side researchers), and in many cases
this is sufficient because the needs are transparent and easily identified
by the non-professional client. Equally, however, this can produce
research of limited or negative value and possibly without the problem
even being recognized. One response to this, on the client side, has been
to set up in-house ‘insight’ teams integrating data from various sources
including ad hoc research, continuous projects, customer database mining
and ‘blue sky’ thinking. This may make professional research input more
effective or reduce it to just one input in the quest to understand a
market and its consumers.
The second factor which affects the success of the professional–client

relationship is the status and standing of the professional. If the client
does not believe the professional is an expert who can provide valuable
skills and knowledge, then the practitioner is unlikely to be trusted to
identify true needs. To a considerable extent this problem exists in
market research. Often market research does not look too difficult and
appears largely to be based on common sense and general business skills.
This is evidential in the increasing amount of DIY research that is being
undertaken, whether it is designing questionnaires or conducting focus
groups. Clients may also bypass professional research entirely through
direct interface with customers, e.g. open innovation and co-creation
approaches whereby companies and consumers work together to create
and innovate ideas, exemplified by projects such as Procter and Gamble’s
‘Connect and Develop’ open innovation project. This may partly be a
result of the market researcher’s own presentational skills making the
work done transparent and, therefore, seem relatively easy. Although
other factors, such as the consumer’s access to information, global social
media networks and consumer empowerment, may mean that the
traditional top-down marketing approach would have been less effective.
As discussed above, professional service is characterized by a focus on

needs rather than just wants. However, this can lead to problems and
conflicts. What happens if the client insists on some wants being met

which in the professional’s judgement do not correspond to true needs?

Codes of conduct and other elements of professional ethics may limit the
grossest mismatch in this respect by forbidding the satisfaction of some
sorts of wants, e.g. supporting a claim by spurious research results.
However, often it is a matter of commercial decision how far the
practitioner will satisfy less than adequate requirements. In the real
world, professionals will often deliver less than a quality solution in order
to meet expressed requirements, if only on the principle that even if they
do walk away from the project others may not.
Finally, in relation to professionals and clients, it should be noted that

professions exist to tell clients what they both want and shall be allowed
to have and that, despite all the fine sentiments, professions are in fact
conspiracies to serve the interests of practitioners rather than clients. One
does not have to take on board the strong expression of this position to
recognize that professions can be self-serving and that, whatever the
intentions, some of the practices in place are more in the interests of
members than their clients. The extent to which professions, left to their
own devices, can have this effect is very much linked to the extent to
which they are closed rather than open and can approach a monopoly
position. Market research doesn’t approach either condition and has little
opportunity (quite apart from any intent) to be unresponsive to clients.
Is professionalism enough?
Market research by most criteria is a profession, even if the wider world
does not always recognize it to be one. It has a well-established
professional body and high-level of intellectual content. Within the
profession there is some consensus about good practice and therefore
about what makes for quality, or at least quality as excellence. This
consensus is also enough to provide a body of knowledge which it is
agreed should be acquired by new entrants. However, there is far from
unanimity on what constitutes good practice, not only in relation to
specifics and details but even the basis by which the ‘good’ can be
determined. Furthermore, the ability of the profession to ensure that
quality, whatever its makeup, is delivered in practice is limited for
structural reasons. Market research is very much an open profession and
not all practitioners are within the professional body (nor is it reasonable
to assume that those outside are necessarily inferior in skills). Even full
membership in a recognized professional research body does not
guarantee any particular level of competence or ability to deliver quality
(although in future this might change). There is also the problem that
the level of control exercised by codes of conduct and enforcement is
quite limited. Similarly, research services may be delivered by research
suppliers who are not members of code-holding bodies (this will vary
between countries) and supplier organizations may not be managed and
overseen by those from within the research profession. Increasing

Is professionalism enough?
accountability to shareholders and the like means that large,

publicly-listed research organizations are much more likely to be run by
experienced business and financial professionals than by researchers.
Even in relation to quality as excellence, therefore, professionalism in

market research has a limited ability to give assurance to the buyers and
users about the quality of its output. It may be that choosing the
practitioner is the most important consideration of all in seeking quality,
but the institutions within market research are of only limited help in this
respect. Other approaches, however, support and supplement the
professional basis of quality and are discussed in later chapters. There is
also the other concept of quality to consider; relative quality or satisfying
clients’ requirements. In this area professionalism has some implications
but other approaches have more potential impact. The achievement of
this kind of quality is covered in some detail in the next chapter.

5 Quality and business process
Whether or not it is a profession, market research is a business. We

introduce how both excellence and relative quality is linked to and
achieved through business processes. Aspects of efficiency and its relation
to quality are considered. We also describe the main business quality
models and their relevance to market research. Various types of standards
linked to these models are introduced prior to more detailed coverage in
later chapters.
The last chapter discussed how market research is organized to provide

quality on an individual professional basis, with a bias towards quality as
excellence based on true and valid research. However, research is not for
the most part carried out by sole practitioners but by market research
companies and the people (professional and non-professional) who make
up these companies. Quality in market research is, therefore, largely
delivered through a business process and understanding what this
involves is the subject of this chapter.
Satisfying clients
That market research companies – or any business – must over a period
of time satisfy clients or customers is almost a truism. No client is bound
forever to any research company and if seriously dissatisfied the client
will sooner or later take their business elsewhere. Clients expect value for
money, research which meets their purposes and which can have a
demonstrable and measurable return on investment. If the research
company concerned consistently performs poorly and fails to meet clients’
internal requirements it will soon be out of business. Like any other
business, therefore, relative quality – satisfying clients – has to be a major
objective whether or not it is set out in, for example, a ‘quality policy’.
However, the other sense of quality – quality as excellence both in terms

of the creative thinking that goes into designing the research and
excellence in process – cannot be ignored either; some balance needs to
be kept between the two types of quality. A sole pursuit of relative
quality with a primary focus on the client’s wants to the possible neglect
of their needs will, in the long run, be self-defeating; research output can
only be useful for so long. In other words, if there is a gross mismatch
between satisfying wants (relative quality) and needs (quality as
excellence), sooner or later the deficiencies will become apparent and the

client will recognize the underlying need for ‘true’ research. On the other
hand, a pursuit of excellence, irrespective of client wants, has its own
problems. For one thing, if the satisfaction of manifest wants is ignored
there is a good chance that business will not be placed; the proposal will
simply be lost if it inadequately addresses stated wants, even though the
research proposed may objectively be the perfect solution. Also – and
many research companies have taken a long time to realize this – clients’
wants go beyond areas that correspond to any objective research needs.
Confidence in the people involved in the research work and whoever sells
it, clarity in all forms of communication relating to the work, and high
quality reporting are all likely to be client wants that have to be satisfied.
Commercially, it is such considerations that often decide who will have
the business. Finally, wants must be balanced with needs within a budget;
the cost of ‘best’ research may be unrealistically high in relation to the
client’s available budget or in relation to the value of the information
provided.26 It may not even be possible to meet the client’s needs at all
within the budget proposed; the affordable research methodology may
just be grossly inadequate. In most cases, however, some compromise or
trade-off can resolve the dilemma.
Four steps to balancing needs and wants

Successful commercial market research is a matter of balancing the two
senses of quality and we would suggest there are four linked tasks to
address in pursuit of quality in market research: the identification of
client wants; deciding the extent to which these can be met through
‘true’ research; the use of effective rhetoric to persuade clients that the
satisfaction of such wants involves meeting needs; and finally, delivering
the service in a way that enhances satisfaction over and above meeting
‘objective’ needs and ensuring ‘non-objective’ wants are met as well.
Identification of wants and needs – of requirements – clearly has a very

important place in providing a quality market research service. This is
hardly a surprise, given that one of the major functions of market
research is to identify the wants and needs of consumers and potential
customers for a product or service. As a business activity, market research
needs to take its own medicine and identify the wants and needs of its
own clients. This process works on two levels. First, there is a strategic
necessity to identify general classes of wants and needs that the research
company will be able to satisfy effectively, bearing in mind the resources
available now or in the future. This part of the process may lead a
research company to seek certain types of research work (e.g. qualitative
rather than quantitative, ad hoc rather than continuous), or specialize in
a market sector. There is also the linked question of how the research
company will be able to develop some form of competitive advantage in
26
As an extreme example, there is clearly no point in spending more on research than will be
spent on any resulting business decisions based on the research.

Satisfying clients
satisfying these identified requirements. What will, or does, make a

research company special? As much as being part of need identification,
these areas are the core of business planning.
The second level of identification of requirements involves the particular;

the specific, i.e. in relation to an individual client. In ad hoc research this
translates to taking a brief from a client – understanding the background
to and need for the research and identifying other requirements.
According to McGivern (2009) detailed briefs usually include: title,
definition of the problem (often the most difficult thing), background to
the problem, why the research is necessary, statement of research
objectives, use of information, target population, suggested approach,
analysis required, outputs, liaison arrangements, timings, budget, form or
proposal and selection criteria. Whether classed as wants or needs, these
requirements will also include some element of the budget and timetable
available.
Whether at the strategic or specific level, assumptions are usually made

(they may even be researched) about clients’ general requirements
irrespective of the particular project. What engages a client? Possible
categories include ‘true’ research, the ability to meet promises and
objectives, generating real business insights, creativity, professional
working methods, honouring contract terms, timeliness, clarity in
communication, adding value to facts and many other things. Some of
these – those that most buyers want and suppliers are willing to provide
– can be incorporated into industry-wide standards (more on this later).
Identifying client requirements

As set out above, the identification of requirements in market research
all sounds rather impersonal and bloodless. In practice, however,
identifying requirements at this particular level has a very strong personal
element. For one thing it involves individual skills to have the necessary
rapport with clients to understand the requirements fully. Paradoxically,
in the process of identifying requirements, some of these requirements
are delivered; this process should give confidence to the client. A
universal client requirement is to have confidence in the service and
output provided by the research company. Without that confidence the
results cannot be used to guide decisions, which is the whole point of
buying research. There are various ways in which a research company can
offer that confidence: past performance, reputation and image in the
market, and independent quality assurance (to be discussed in later
chapters) are all important, but nearly always confidence is bound up
with individuals. The ability of the agency to deliver depends on faith
and trust in whoever fronts the service.
As an industry, market research has long recognized the need for

building confidence in the ability to identify and deliver client

requirements. However, there are also some business problems related to

this. In a very small research company a service based on the qualities of
the particular individual is realistic, but is less so as you move to where a
larger team will be involved. The individual taking a brief may not write
the proposal, let alone manage the project, and by far the majority of
research is carried out by operations (some of which are automated and
technology driven) well above the level where one individual largely
determines the quality of output. In these situations, confidence needs to
be based on something more than personal chemistry even if buyers do
not always recognize this. And on the supplier side, a predominant
emphasis on the personal also has its problems – the business becomes
over-dependent on the reputation of individuals and this can seriously
inhibit growth. There is a limit to the number of clients the ‘star’
performer can even contact, let alone handle. The problems of
confidence building and how it is achieved – whether by an emphasis on
the personal or otherwise – have a close link with the identification of
requirements, but it is also a much wider issue to which we shall return.
Conformity to specification
Once identified, client requirements at the individual project level can be
set out as part of a specification. This will also include some statement of
the outputs which will be delivered to meet these requirements. The
most common tangible form of such a specification is a research proposal
(at least it is in ad hoc research). However, the ‘full’ specification of what
is required and will be delivered is always greater than what is normally
covered in a proposal document. For one thing there may be an implicit
assumption of professional standards of practice beyond what is formally
set down, and this may include that any methods used will conform to
generally recognized and valid practice. There are also the requirements
that relate to service outside the area of objective research practice –
good communication, clarity in reporting, timeliness, levels of client
contact, and involvement. Some of these may be covered explicitly in a
proposal while others may be set out in promotional literature or
standard operating procedures or as part of a formal quality policy.
Agreed and recognized industry or quality standards may also apply.
However set out, and even if part of it is implicit, a specification for the
research project will exist and should be recognized by the supplier and,
as far as practical, explicitly agreed with the client. Difficulties arise and
serious quality problems result where this specification leaves out some
important requirements, since there is no possibility that the research
company can satisfy the client if they are unaware of what is required. If,
however, a more or less complete specification is achieved, quality
becomes a matter of meeting that specification: meet it in all important
particulars and quality is achieved; fail to do so in one or more areas and

Quality and money
the service provided is less than high quality. Here, then, is another sense
of quality – conformity to specification.
Day to day, conformity to specification is an effective way of

operationalizing quality in a business process and it will be a theme
throughout the rest of this book. This sense of quality bridges both
relative quality and quality as excellence since both should be balanced in
the specification. However, achieving quality through conformity to
specification does require a thorough understanding of requirements and
a full specification. It is emphatically not just a matter of technically
‘good’ research with a disregard for the less objective client requirements
that may also need articulating formally, whether as part of a research
proposal or as general statements about what a research company
delivers to its clients. The specification also needs to be fully understood
by the team involved and this cannot just be left to chance.
Although this pre-empts some later comments, it is worth noting here

that the concept of conformity to specification can be criticized for its
static nature. Client requirements change and are changed by research
companies, and specifications once set may miss this point. There is also
the issue of whether the aim should be mere satisfaction rather than
something more; delighting, for example. There will be more on this
later.
Quality and money

The issue of money has already been mentioned in the context of clients’
wants and needs, but money also has another dimension in quality.
Research companies are for the most part private sector undertakings
which are bound to have commercial objectives of some sort. In effect
they need to attain an adequate level of performance on one or more
financial performance measures. These may include profit, cash flow,
shareholder value, dividends and net worth or just the income of the
principals (at least in smaller companies, making a ‘good’ living may be
considered a quite adequate financial performance). What is adequate
will depend on various circumstances and in some cases, particularly
companies that are not publicly listed and remain in private hands,
performance may be below any optimum – some market research
companies can be ‘non-commercial’ in this sense. However, financial
performance cannot be ignored or ignored indefinitely and has to be
considered in relation to quality.
Bringing money into quality is usually by some caveat of ‘least cost’. For
example, customer requirements will be met at least cost, or conformity
to specification will be at least cost. Least cost is regarded as the mirror
of most profit although this congruence is not, strictly speaking, logically
watertight. In one sense, least cost can compromise what is delivered to

the customer; a budget airline for a long haul flight may be the least cost
option but for a comfortable flight, convenience and all the added
extras, business class might be more appropriate. Similarly an online
sample of 500 may be cheaper than a random face-to-face sample of
1,000 but may be inadequate to meet the research objective and
purpose. However, least cost is a valid concept if considered
post-specification. Once what will be delivered is defined and agreed,
least cost becomes a matter of efficiency.
Prior to the advent of online research, efficiency as an aspect of market

research tended to be disregarded. However, efficiency is one of the
major attributes lauded by online research specialists. Historically, at least
in some areas, research has been a fortunate trade with high enough
margins to make efficiency a not obviously pressing matter. However, this
has changed with the advent of online research where traditional
methodologies have to compete with a methodology which can claim
results in hours, as opposed to days and weeks. However, speed does not
necessarily equate to efficiency if robustness, reliability, etc. are sacrificed
as a result. In all purchasing decisions there are trade-offs; purchasing
purely on speed of results will often be a bad purchase.
Achieving efficiency
There are three main areas to seek efficiency in market research: in
methods of working, in productivity, and in minimization of errors.
Working methods
Methods of working include internal organization. The classic means to

efficiency in the industrial age was division of labour – Adam Smith’s pin
factory27. In market research the level of division of labour is largely a
function of research company size; only at certain levels of output can
staff be dedicated to a narrow range of tasks. Working against division
of labour is the need for flexibility to deliver the service. Dedicated staff
need to be brought into other tasks if the need is there. Division of
labour also stands in the way of communication and therefore often of
service delivery. When you have a chain of staff, each with their own
specific task and, in the case of research operating in geographically
dispersed locations, miscommunications and misunderstandings are
bound to arise. For example, the research executive may have trouble
communicating field-related issues to the client as they are not directly
involved in this activity in the way that a Field Manager or Interviewer
would be. Similarly, poor communication reduces understanding of the
specification to be met and thus the chances of delivering quality. Of
27
Adam Smith in An Inquiry into the Nature and Causes of the Wealth of Nations (1776),
described the production of a pin divided into eighteen distinct operations and used this
to describe how efficiency could be increased through the division of labour.

Quality and money
course these problems can be overcome in various ways. One way is to

have an effective quality system; in fact to a large extent the need for
this can be regarded as a consequence of the division of labour. Another
approach is a team focus – again, the need for this is also partly a result
of the division of labour.
It may appear from the above that we are suggesting that various forms
of non-specialist flexibility are always better than a pronounced division
of labour in market research, but this is not our intention. Division of
labour is always a means to be considered in the pursuit of efficiency
providing that any problems it creates do not outweigh the realizable
benefits. Some research companies may paradoxically have too much and
too little division of labour, with tasks that need to be integrated divided
and tasks that would be better divided, integrated. The changing nature
of research, particularly how data are gathered – less face to face and
more indirect via digital techniques28 – means that the ‘old’ divisions
between field, operations, head office, etc. may result in entirely
different divisions of labour.
The question of getting the balance right in relation to division of labour

is of course not the only aspect of working methods which affect
efficiency. Others include the degree of pyramid versus flat-structure
organization, the effectiveness of teams working as a cure for
communication problems arising from a division of labour, and the whole
issue of skills building. All these can aid efficiency and, therefore, deliver
quality at least cost. However, there is another aspect of efficient
working methods: capital investment. In market research, and
increasingly elsewhere, capital investment is now largely based around
technological development.
Technology has made considerable inroads in market research vertically

(reaching into all sizes of operations) and horizontally (involving more
functions and processes). Originally technology was introduced to
improve processing, at first through analysis packages and then
automated data entry systems. Next, technology came to dominate
research reporting and client communication (word processing,
presentation tools etc.) and data collection with computer-aided
interviewing by phone (CATI), face-to-face (CAPI), and web-assisted
(CAWI) based research. Now technology has moved from being a useful
tool in support of research to a methodology in its own right. Some of
the early online research took the traditional face-to-face and telephone
techniques and transferred them to the online environment. The newer
techniques, based around web 2.0 (and indeed 3.0), take a completely
different approach, being heavily based upon observation and
ethnography techniques which have more in common with mass
observation of the 1930s than research at the end of the 20th century.
The researcher is a listener and watcher of behaviour and attitudes, no
28

longer asking specific questions but, by using the right techniques,

drawing conclusions from discussions that are already taking place.
Waiting to come into the mainstream are increasing amounts of
behavioural and tracking data e.g. Radio Frequency Identification (RFID)
tags, GPS mobile data, eye tracking, neuroscience, etc. This is not to say
that the more established question and response techniques have been
replaced wholesale; they still have a key role within and outside of online
research.
These and other facets of technology have had a major impact on

efficiency and the potential to deliver quality research at least cost
(although technology-based research creates its own quality dilemmas).
One other aspect of this process to be noted is a certain bias towards
larger companies since it is they who can more readily afford the
investment level involved.
Productivity
Addressing or monitoring individual productivity historically was less of

an issue in research than efficiency more generally. This is strange in that
staff costs in market research predominate and, one way or another, any
attempt to improve efficiency must consider the question of staff
productivity. One problem is the difficulty of measuring output, which is
essential to any serious attempt to define and improve productivity. In
the long run, project norms can be established for staff productivity in
data collection and processing, but this is much harder in short-run ad
hoc work; by the stage at which output norms can be defined, the work
is half finished.
Even more problematic is the issue of research executive or professional

productivity – how much time should be devoted to a questionnaire, a
proposal or report? One executive may produce a ‘standard’ proposal in
half a day and another might take a whole day over the same job. Is the
first more efficient? Only if they win the business; a half day spent on a
lost proposal is inefficient by any measure. What we are alluding to is
that superficial productivity may be at the expense of quality – of
meeting the specification – but that the differences may be very difficult
to measure or even recognize. The lost proposal is clear-cut; the problems
of the hastily drafted questionnaire or report may not be so transparent.
In any sort of non-routine work – and much of market research contains
at least some of this – there is likely to be some trade-off between time
spent and quality of output, although the law of diminishing returns also
applies.
Minimization of errors
The last aspect of efficiency to consider is error reduction, or more

precisely errors identified in the research process. In the manufacturing
industry, errors or defects in the process result in rejects and scrap, and

Delivering market research quality through the business process
the benefits of a quality approach are often discussed in the context of

reducing these levels – do it right and do it right first time. Research
output is, however, seldom scrapped in the manufacturing sense and
instead identified errors are more likely to be reworked; the booked
interviewers are put on hold, poor interviews are repeated, etc. So even
though there is little in the way of visible scrap levels, there is nearly
always potential to tighten up on the process of error reduction in
research and so reduce costs.
Even more important, error reduction improves quality in the sense of

matching requirements. This may be through avoiding delays and so
meeting timing requirements but also because some level of detected
error is in practice often not reworked; the error is not ‘big enough’. Also
some sorts of error are difficult to identify at all or it is a matter of
judgement whether it is an error. Is an uninspired report an error? Finally,
it should be recognized that in market research (and other services) error
reduction in the process has a special importance because many cannot
be detected in the final report. Bad data collection does not look any
different to good data collection when the results are set out in the
report.
This brings us to the more general point that changes introduced to

improve process efficiency may do more for quality than achieve it at
least cost, since the new practices make the output better in some sense.
This is very clear in relation to technology. CATI and CAPI, for example,
reduce or obviate errors from interviewer routing or allow the
questionnaire to be administered in a way that might be just too
unwieldy by traditional methods. Similarly, logic checks are built into data
processing that identify (even if they do not always solve) errors in data
collection or coding. Quality of output may also be improved by other
sorts of changes primarily introduced to improve efficiency; team work
may for example bring new creativity to reporting and a formal quality
system should at least improve the consistency of the service provided, as
well as making the process more efficient.
Delivering market research quality through the

business process
Figure 6 summarizes the various elements of quality we have discussed in
this chapter. Conformity to specification should take account of both
quality as excellence and relative quality, and quality has to be delivered
within a commercial environment – at least cost.
Whatever is delivered can be anything across the quality spectrum; at one

end poor quality, poor value for money, not meeting client needs and at
the other excellence, meeting client needs, to specification and at
minimal cost. In other words quality can be treated as a variable and

Figure 6 – Elements of quality
measured. All sorts of measures are possible within the processes (e.g.
error rates, instances of non-conformity to specification, production costs
and productivity) or of the outputs (e.g. customer complaint levels,
customer satisfaction, market performance, financial performance); some
of these measures are more direct than others, which may reflect
‘outside’ factors independent of quality. Market research techniques
themselves can be a quality measurement tool, particularly with customer
satisfaction research.
The measurement of quality is often hard to quantify in market research,

but for now it can be accepted that research companies, like other
businesses, should have the optimization of quality as an important goal.
The question is, how can this be achieved? There are many approaches to
quality optimization within business with different ones emphasized or
dominating at different times (e.g. quality control, Total Quality
Management, Six Sigma, Zero Defects, etc.) and in different business
areas (and more or less appropriate depending on the application). They
can all, however, be reasonably summarized as within four classifications
or business quality models: professional responsibility; quality control;
quality assurance; and Total Quality Management (TQM). Each model is
discussed in this chapter in relation to their effectiveness within three key
elements of a business (mentioned in Chapter 1): commitment, process,
and people.
To recap, commitment to quality from top to bottom within an

organization is always essential. Process is how the business is organized
and works to deliver quality to the client and clearly nothing can be

done at all without people, and particularly so in businesses such as

market research. Although it is useful, analytically, to separate these
three elements they clearly do not in any sense exist independently; they
are all aspects or perspectives of unity of a business. Figure 7 links the
business quality models with the three elements.
Figure 7 – Four business quality models
Professional responsibility and process

Professionalism has been discussed at some length in the previous
chapter. The issue here, however, is how effective it is as a quality model
within a business rather than at the individual practitioner level. As
already indicated, problems and tensions arise in any link-up of
professionalism. The business process and the difficulties encountered can
be discussed within the context of the effectiveness of the professional
responsibility model, including the business elements of commitment,
process and people. Professional responsibility is very effective in building
a commitment to quality amongst members of a profession, including
market research. This is achieved through values (a commitment to
excellence, truth etc.), professional institutions and the way new
members of the profession are inducted. All of these carry across to a
commitment to quality within an organization. However, this needs to be
understood and promoted by the senior executives of an organization,
and one of the difficulties for research is the increasing number of
research companies led by non-researchers who may not necessarily
appreciate the need for aspects of quality over cost. Professional

responsibility also works very well at the people level through the strong
emphasis on professional development, both initial and continuing, and
the advancement of professional knowledge (professional bodies in their
role as learned societies).
Yet in Figure 7 the effectiveness of professional responsibility as a

business quality model is only marked as medium for both commitment
and people. Why? Within a market research business, not all or even
necessarily the majority of staff are of professional status; there are also
the various support staff and the ratio of these tends to be higher in the
larger companies. Even more important are the legions of interviewers
working in the field or telephone centres. Professional responsibility
tends in practice to bypass these staff and whatever commitment to
quality they may have. This is not entirely surprising considering the way
in which interviewers are retained (workers not employees, no formal
training or qualifications generally made available, etc.). The
professional/non-professional gulf can emphasize the division and
difference. Even ‘near’ professionals such as supervisors/field controllers
can consider themselves to be second class.
Perhaps a greater limitation of professional responsibility, though, lies in

relation to process. With professionalism focused at the individual level it
has little relevance to how a business process should be worked. At best,
the model is of the senior professional controlling the process through
taking responsibility for the output, coaching and mentoring juniors, and
supervising non-professionals. In relation to process, professional
responsibility is amateur. But until recently this was the implicit model for
quality within market research.
The limitations of professional responsibility as a business quality model

are linked to size. Its limitations in relation to process only really become
apparent above a certain organizational size and division of resource. In
a research company with a couple of experienced practitioners and a few
assistants (and many still are of this size) process problems are either less
evident or can be solved in the informal manner possible within the
professional responsibility model. Even in a medium-sized company,
however, real thought and effort have to be put into managing the
processes and the increase of interest in other models may, in part, reflect
the growth of company size.
Quality control and process

The other models for delivering quality are all rooted in the
manufacturing industry and have been imported and adapted to service
industries such as market research. Quality control is the oldest of them
and has long been the backbone of product quality within industry; at its
crudest, quality control works by having a quality ‘policeman’ inspect all
(or a sample of) the products that are awaiting shipment. The inspection

will be based on a template of what is required and may involve testing

physical properties or functions. Products that match up to pre-set criteria
go off to the customer while those that do not go into the scrap heap or
back for reworking. Process is, therefore, controlled through output only.
This model was (and still is) widely used in manufacturing and, while it
may be better than nothing, by design it does not address commitment
or people at all, except negatively. Rejection, as the only output of
quality, is hardly much of a motivator.
Quality control also has some fairly obvious limitations even in relation to
process. For one thing, scrap represents waste and unnecessary costs but
within the model nothing is done to address why scrap arises or why the
rates go up or down. The output of quality control of this sort is
inefficiency and with the underlying problems not considered, no
remedies can be sought or found. Also, who is to say the ‘policeman’ is
up to the job or that the criteria by which products are judged
correspond to requirements. In market research the application of the
quality control model in this form is clearly of little value since the
quality of research can scarcely, or only very partially, be inspected in any
final output. A check of a written report may pick up processing errors,
poor layout or even an inadequate coverage of the research objectives,
but even if it passes those checks it still does not say the work is any
good; the research design may be poor, or badly executed, the data
collection haphazard or full of errors and the analysis may have made
everything worse.
Quality control can be made more effective in relation to process if it is

brought into the workplace. In this variant, the total process is broken
down and inspection carried out on the output of each critical
sub-process (which is of course the input of the next sub-process). This is
far more effective. At least problems are picked up earlier so that later
processes are not wasted working on already defective products. Also,
when errors are found, we know at which part of the process to look if
we are going to find ways of solving the problems. However, even in this
variant, the quality control model is negative rather than positive in
outlook and there is nothing within it to find why things go wrong or
put them right. Also it does not address commitment or up-skilling
people; in fact with the ‘policeman’ in the factory, morale is probably
worse.
However, such intermediate quality control has more of a potential role

in market research than final inspection and it is, in practice, used
effectively. Research designs, questionnaires and reports may be checked
by more senior staff and the output from data collection – and possibly
data processing – is often checked or verified in ways not unlike the
methods used in manufacturing. The UK’s Interviewer Quality Control
Scheme (IQCS) prescribes such methods in data collection (and this
standard has been adopted and adapted in other countries such as

Australia) and the international standard for market research,

BS ISO 20252, Market, opinion and social research – Vocabulary and
service requirements does so across a wider spectrum of the market
research process. (BS ISO 20252 is discussed below and in Chapter 9.)
Quality assurance and process

Quality assurance or quality management,29 the third business quality
model, is often misunderstood. Either the difference between quality
control and quality assurance is not grasped or it is assumed, wrongly, to
be about the setting of product or service standards (which are, not
surprisingly, then found to be wanting). The focus of quality assurance is
on how things are produced (or the service carried out) rather than on
the characteristics of what is produced, i.e. finding effective or best ways
of managing the process to minimize defects and errors. A key objective
of this model is the systematic investigation and solving of problems.
Methods include using tools such as a formal quality system, internal
auditing, and management involvement; quality assurance is a model for
the active management of quality and, therefore, involvement. Quality
assurance also builds in steps to ensure requirements are defined and
met.
Inspection and testing are not in principle intrinsic to quality assurance

and ideally the objective may be to obviate the necessity for much of this
‘wasted’ effort; solve the underlying causes of error and there will be no
need for inspection and testing. In practice though, quality assurance
invariably incorporates some quality control elements (with good effect)
and may be introduced as a next step to a quality control system.
However, the point may be partly missed if quality assurance is just
crudely superimposed; the limitations of quality control are then not
surpassed.
Quality assurance clearly aims to address process in a far-reaching way. It

also involves commitment to some degree; it will certainly only work if
management believe in it and believe that managing quality is a high
priority. Other staff also have to work at quality and in this sense
commitment to some level is needed for the approach to work. However,
commitment is not engendered by quality assurance as such; it needs to
be inspired from ‘outside’. Quality assurance is also scarcely
people-oriented; they at best only appear as role players and then
one-dimensionally; as ‘operators’, ‘investigators’ or ‘representatives’.
As with many other sectors, quality assurance has made an impact on and
inroads into market research, with companies implementing the most
widely adopted standard for quality assurance – the BS EN ISO 9000 series
29
We use the two terms – quality assurance and quality management – as synonymous
although arguably quality management is the means of achieving quality assurance.

– particularly BS EN ISO 9001, Quality management systems –

Requirements for quality managements systems and BS ISO 20252,
Market, opinion and social research – Vocabulary and service
requirements, which places strong emphasis on quality assurance. The
interviewer scheme, IQCS, also implicitly if not explicitly, incorporates
quality assurance thinking. Because of its widely accepted importance,
quality assurance is featured heavily in the remaining chapters, including
Chapter 7 (which covers BS EN ISO 9001). How it can be made to work in
market research is also discussed in some detail in Chapters 9 and 11.
A point to note about both quality control and quality assurance is that
the specification of what should be attained in quality terms comes from
outside and particularly from customers and clients and their
requirements. Quality control takes this as given while quality assurance
should incorporate methods for establishing or checking what these are.
However, quality assurance has no built-in standards of product or service
quality. It is a method for ensuring that such standards, whatever these
are, are identified and met constantly or consistently, but it is not a
method of defining them. As an aid to both customers and suppliers, it
may be found useful to define requirements and the specification wholly
or partly in terms of some recognized and independent standard. Quality,
or at least minimum quality, then becomes a matter of conformity to the
standard. In manufacturing, quality standards, particularly BS and ISO
standards are very well established. Service standards have also become
increasingly prevalent in recent years and in the UK, market research led
the way in this respect with the first true BS service standard
(BS 7911:2003, Specification for organizations conducting market
research). In turn UK standards for market research influenced the
development of comparable published standards in other countries and
eventually led to the international research standard ISO 20252, first
developed in 2006 and subsequently updated in 2012, and its
accompanying standard for access panels: BS ISO 26362:2009, Access
panels in market, opinion and social research – Vocabulary and service
requirements (see Chapter 9).
Although product or service standards are not defined within quality

assurance models, there is a link. Quality assurance provides a method of
ensuring that output is consistently up to the minimum standards. BSI,
for example, offers its Kitemark to give third party assurance that a
business’s product, service or process conforms to defined BS (or other)
standard. However, a prior condition of awarding the Kitemark is that
the business has effective quality assurance management system in place
through BS EN ISO 9001 to ensure its capability.

Total Quality Management and process

The final model of business quality to consider is Total Quality
Management (TQM). We could have considered a number of the models
out there, particularly Six Sigma (the business management strategy
approach originally developed by Motorola), but the difference between
the models is fundamentally about the approach as opposed to the
content. For example, while Six Sigma focuses on improving quality by
reducing the number of defects, TQM tries to improve quality by
ensuring conformance to internal requirements. We are going to look
more at TQM as this is a more suitable model for market research.
From the TQM perspective, quality assurance is about ensuring that a

given level of quality is met (i.e. a product standard). This is all very well,
but is not going to ensure long term competitive advantage. Instead,
quality should be viewed dynamically, existing standards exceeded and
the achievements of other organizations ‘benchmarked’. Rather than
merely satisfying the customer, the aim should be to surprise and delight,
to create loyalty to the supplier by giving ‘extras’ that they were not
even aware of. Quality itself, therefore, becomes a process and one with
an ideal but not an attainable end. The pursuit of quality is forever. This
all may sound very exaggerated but then that is the nature of TQM.
Some critics argue that excited disciples and jargon are the more
characteristic outputs than any real improvements.
Some would also contend that TQM sets up artificial limitations to quality
assurance so that the new methods can claim to surpass them. Also,
effective quality assurance programmes recognize that customers’ wants
and needs change and shift with time – partly because of the efforts of
suppliers to give new experiences – and that quality, therefore, is not and
cannot be static. However, TQM cannot be dismissed, if only because of
its level of uptake. Also it is not necessarily a competitor of quality
assurance; it can be an extension. The standard BS EN ISO 900430 is
intended to provide a bridge between the quality assurance approach of
BS EN ISO 9001 and a more TQM approach. Also some regard quality
assurance as necessary underpinning on which further progress can be
built. Others, however, are more sceptical; at least to quality assurance
built around standards such as BS EN ISO 9001, which some perceive as
not necessarily suitable for their business needs.
It is not easy to characterize what TQM involves in practice, since it

encompasses a wide range of methods and techniques. In all versions,
however, there is a very strong emphasis on people and their
commitment to the process of quality improvement. Much is made of
30
Although BS EN ISO 9004 is classed as a standard it is better considered as a guidance
document on quality management and it is explicitly stated by ISO that BS EN ISO 9004 ‘is
not intended for certification, regulatory or contractual use’; organizations are not
assessed externally against this standard.

realizing the potential of staff to enable them to deliver quality to their

customers and clients. Staff members with no access to ‘real’ customers
are encouraged to regard other staff as internal customers. TQM is soft
and fuzzy, yet is often accompanied with messianic fervour and a
hearts-and-minds outlook. Empowerment, skill enhancement, team
building, managers as facilitators, quality circles and open learning
organizations are all alternative or overlapping techniques within a TQM
approach. A framework around which to implement the staff
development techniques within TQM is the Investor in People (IiP)
standard (although this can be taken up without a conscious subscription
to TQM). IiP is discussed in Chapter 10.
Process is certainly not ignored in TQM, although, in some versions,

planning changes to process comes out of and follows changing people;
right ways come from right people. However, other approaches involve a
planned review and change of processes right from the start of the
programme. The British Quality Foundation, for example, promotes a
European model – the EFQM Excellence Model – which allows
managers/leaders to understand the cause and effect relationships
between what their organization does and the results it achieves.
Business re-structuring and re-engineering, involving a radical change of
business processes to achieve some substantial measurable goal (e.g.
delivery time, cost reduction, productivity etc.) can also be linked to or be
a variant of TQM. Obviously, process is the focus, although the need for
commitment and the need for people to change the way they work are
also essential in re-engineering programmes.
Business re-engineering is clearly not at the cuddly end of TQM and the
reality may often involve acute pain for the participants, with
down-sizing (i.e. fewer staff undertaking more of the processes and
having to take greater responsibility for activities) and de-layering (i.e.
making management staff redundant). These aspects may clearly work
against commitment and be seen as anti-people, although they may well
improve shareholder value regardless of staff morale (which is not, after
all, an end of capitalist enterprise). Moreover, with developments in
technology – and more processes being driven by technology rather than
people – the cycle of re-engineering, re-structuring, resource rationing,
etc. is now a permanent feature of business life including market
research. Although the sector has experienced an unprecedented period
of growth (only recently stemmed by the global recession) there are
generally fewer traditional researchers employed within the sector who
are generating more research than ever before.
TQM as a self-conscious movement has made little impact in market

research (although many companies have unconsciously adopted
elements of it). However, quality as an explicit concept is still relatively
new in market research and is currently focused around the growth of
the international standard for research, ISO 20252, rather than more

formal business quality standards such as TQM. In practice, TQM may well
take the form of a linking and amalgamation of professionalism, quality
assurance, and the adoption of industry minimum standards such as ISO
and IQCS together with softer, people programmes (several research
companies have taken up IiP). As an industry, little effort is needed to
persuade market researchers of the importance of people to achieve
quality. Techniques such as empowerment, facilitating management and
the learning organization are already often in place in market research
companies and were so before the terminology was invented.
Because of its vagueness, TQM will not be explicitly discussed elsewhere

in this book but most of the elements are covered one way or another,
particularly in the various quality standards which apply the TQM
principles in practice (see Chapters 7, 8 and 9).

6 Quality and legislation
Legislation of many sorts constrains how market research can be carried

out and impacts on working practice and quality. In this chapter we
consider legislation which has a particular impact on market research.
Legislation on data protection is the most obvious case and the general
principles, definitions and application in market research of these legal
requirements are covered, particularly those based on EU directives. Such
legislation constrains how research methodologies are used and guidance
for compliance is given. Other types of legislation are also touched on
including those that may limit access to or communication with research
respondents and those which protect respondents, such as health and
safety regulations applicable to product testing. The positive value of
legislation in shaping codes and standards is also discussed.31
In the previous chapters we discussed the methods organizations can

adopt to improve research quality. But there are also mandatory legal
requirements that underpin research, and in this chapter we will explore
the practice and process of research in the context of legislation that
influences what can and cannot be done. Research is, as other companies
are, subject to many legal requirements including areas such as
employment, general health and safety, company law etc. but in this
chapter we are concerned with legislation having a special impact on
market and similar research.
However much research pushes its self-regulatory status, legislation

cannot be ignored and is becoming increasingly influential in the way
that research is designed and carried out. Legislation affects the way in
which the various codes of conduct and practice and quality standards
are developed and amended; more and more ethical principles and
process rules reflect legislative obligations as opposed to just research
good practice. The legislative impact can be found not just in the way
that researchers treat respondents, (e.g. data privacy concerns) but also
the technology that researchers use to reach respondents, the products
and services which are the subject of research and the environments
where research is conducted.
31
It should be noted that the interpretation of legislation and its implications for research
detailed in this chapter is not, and should not be regarded, as a legal document.

Legislation and research

In most countries there are few specific pieces of legislation that directly
regulate research (with the exception of opinion research, particularly in
relation to political elections). However, there are various strands of
legislation regulating data or data collection that directly impact on how
research is undertaken and what happens to research results. These
pieces of legislation often mirror requirements already established in the
research codes of conduct and quality standards, but provide a stronger
framework in which researchers have to set the boundaries for what they
do in the name of research. Legal requirements also have to be met
regardless of whether the researcher subscribes to ‘voluntary’ codes of
conduct.
Data protection and privacy

Data privacy is a growing global issue. In today’s world an increasing
number of day-to-day activities require individual citizens to part with
personal data about themselves. The data industry has been described as
the new gold rush; the race is on for grabbing the best, most accurate
and most insightful data about customers to maximize financial returns.
The Internet has revolutionized this (and is becoming the equivalent of a
gold mine for the immense qualities of exploitable data currently
available) not only in the way that information is collected but also as
the avenue for enabling individuals to communicate personal information
about themselves to others (via social networks and so on). Developing in
tandem with this has been a huge sea of change in the way such data
are held and distributed. Data are no longer necessarily held within the
confines of any specific organization’s mainframe or countries’ borders,
as the ‘data cloud’ is the new reality for many online forms of data
collection, processing and transfer.
In response to these trends, and the continuing concerns about the

security of personal data and how it will be used,32 an increasing number
of countries have enacted data protection legislation, and more are
considering introducing it, as citizens become more aware of the dangers
as well as the opportunities of 21st century technology.
Five general principles can be found as the foundations of most data

privacy legislation:
• Notice: informing individuals that their personal data are being

collected; describing how it will be used; who it will be disclosed to
and who will have access to it.
32
Special Eurobarometer 359 Report – Attitudes on Data Protection and Electronic Identity in
the European Union (June 2011).

• Consent: obtaining the agreement of individuals for the purposes of

data collection and for certain types of processing, providing
individuals with the opportunity to prevent data being used for
purposes to which they may object (primarily direct marketing – this
is usually achieved via an opt-in/out).
• Access: enabling individuals to see personal information held about
them and having the right to amend any inaccuracies.
• Security: data owners taking all reasonable steps to ensure that
personal data are not lost, damaged or misused.
• Enforcement: ensuring that effective controls are in place to enforce
the other principles.
The legal basis for the protection of personal data originates in the
Universal Declaration of Human Rights of 1948 in which Article 12 states:
No one shall be subjected to arbitrary interference with his privacy,

family, home or correspondence, nor to attacks upon his honour or
his reputation. Everyone has the right to the protection of the law
against such interference and attacks.
This was reinforced with the Council of Europe’s 1950 Convention for the
Protection of Human Rights and Fundamental Freedoms in which Article
8 provides for the right to respect for private and family life.
Europe has led the way in developing data protection legislation. In 1981
the Council of Europe adopted the Convention of the Protection of
Individuals with regard to Automatic Processing of Personal Data (known
as Convention 108) and this lead to the first comprehensive, national
data protection laws. Convention 108 was built on initial legislation
which had developed in countries such as France and Germany during the
70s. The national laws were originally divergent and to address this, the
European Union (EU) introduced the Data Protection Directive (95/46/EC),
with the aim of harmonizing European data protection legislation. As
membership of the EU has grown so have the number of countries that
have developed privacy legislation (a requirement of European
membership is to adopt Directives) and increasing numbers of
non-European countries have taken a similar path (Australia, Canada,
Japan, Philippines, South Africa, Israel, etc.), as have significant economic
regions such as California in the US.
Some market research results are collected and reported anonymously

and anonymous data are not subject to data protection legislative
requirements. However, collecting personal details from respondents is
usually an essential part of the research process irrespective of whether
respondents are identified in the research results. Quality standards such
as BS ISO 20252 (see Chapter 9 for more detail on the standard) require
personal data on respondents for the purpose of quality control processes
such as back checks. Due to the nature of research, with personal data
collection being an intrinsic part of the process, data protection

legislation is the single most important piece of legislation that directly

impacts on market research. As Europe led the way in introducing data
protection legislation most non-European data protection legislation has,
to a degree, been modelled on the European approach, and we’ll spend
a bit of time outlining how the European Data Protection Directive
(95/46/EC) works as an illustration of how privacy legislation impacts on
research and research quality.
Key definitions within the Directive:
• Personal data are what the Directive protects, defined as any

information relating to indentified or identifiable natural persons
(‘data subjects’). Therefore the focus of the Directive is to protect
personal data about people, not companies, although
unincorporated bodies such as limited liability partnerships and
charities are also covered.
• Processing of data is when the Directive applies and this means any
operation or set of operations which are performed upon personal
data including collecting, recording, organizing, storing, adapting,
altering, retrieving, dissemination, etc. If anything is done to or with
personal data the Directive will apply.
• Controller is the person, authority, agency or any other body which
alone or jointly determine the purposes and means of the processing
of personal data. The controller is the body that will be legally
responsible for the way in which personal data are processed.
Research companies offering ‘full service’ will normally be data
controllers; this is discussed in detail later.
• Processor is the person, authority, agency or any other body which
processes personal data on behalf of the controller (research
subcontractors may be in this category; again see below). As
processors work on behalf of controllers they are not legally
responsible for data processing they carry out on behalf of others
(although the way the Directive has been applied in some European
member states means that this is not always the case).
The ‘rules’ for data protection are defined in Article 6 of the Directive.
The key points from Article 6 are:
1. Member states shall provide that personal data must be:

a) processed fairly and lawfully
b) collected for a specified, explicit and legitimate purpose
c) adequate, relevant and not excessive
d) accurate and where necessary, kept up to date
e) kept in a form which permits identification of data subjects for
no longer than is necessary
2. It shall be for the controller to ensure paragraph 1 is complied with.

Fair and lawful

What these mean in practice for research is that in order for data to
meet the first requirement – for data to be collected fairly and lawfully –
the data subject (or the respondent/participant in research language)
must have unambiguously given their consent for their data to be
processed. In most research cases this will be a matter of asking
respondents to take part in research, to answer a series of research
questions or to agree to have their responses recorded in a group
environment, for example. If they agree and participate, they have
unambiguously consented. This can be more difficult when using passive
data collection methods. If researchers want to collect identifiable data in
this way they have to design the research to obtain consent. In some
cases this can be relatively straightforward, for example collecting CCTV
recorded data from a retail environment would require additional
signage in the stores clearly stating that the CCTV was being used for
security and research purposes, with contact information for the research
organization etc. However, this is not necessarily so easy in all research
projects; conducting mystery shopping research, for example, where
covert recordings take place. The whole point of mystery shopping
research is for interviewers to pose as members of the public and to have
a real life customer experience, so telling someone you are recording
them for a mystery shopping exercise would somewhat defeat the
purpose! To resolve this, most codes, reflecting the legal requirement,
restrict recording of mystery shopping exercises to environments where
staff have been made aware that this might take place at some point in
the future (via terms in staff employment contracts, handbooks and so
on) thus restricting recording to mystery shopping of a client’s own staff
(to measure performance) as opposed to competitor mystery shopping (to
provide comparisons of performance, service, etc.).
The issue of consent is even less clear cut in some online e-ethnography
projects where researchers observe behaviour of individuals in social
media networks and forums. The ‘observer effect’ whereby subjects
modify their behaviour in response to the fact that they are being
studied, is an important research design consideration. Nevertheless the
researcher cannot ignore the requirements of the legislation, however
tempting it might be for researchers to remain unobserved; ‘lurking’ on
such interactions. Legally, if personal data are to be collected for research
purposes, data protection legislation must be adhered to, and researchers
would need to identify themselves and state the purpose of their
observation including what they intend to do with any data collected.
Although this is a legal requirement when collecting personal data, it
would still apply irrespective of whether personal data are collected due
to ethical code requirements. The main research codes all contain
principles based on the concept of informed consent, respecting
respondents and avoiding respondent harm. To lurk in an online social
forum, without correctly identifying yourself as a researcher, would be in

breach of all these principles. To take a comparison in the ‘physical’

world: often ethnographic/observation studies are undertaken in peoples’
homes, where researchers will observe everyday interactions. These
clearly are always undertaken with consent; without consent this would
be home invasion, trespass or worse. The same is true online. The fact
that technology has enabled researchers to do things like observe
intimate personal discussions online without identifying themselves does
not mean they should do it; the legal boundaries are well understood in
the real world, the same level of restraint is required in the virtual one.
Similarly, the sheer volume of personal comments that are available on

‘open’ online forms are very tempting for research purposes. For example
if you were working for a car manufacturer, and you want to show your
client a snapshot of public opinion, a seemingly great way to do this is to
visit an online car forum and grab various random, unattributed
comments from the sites. However, this is not without its problems. First,
most sites have terms and conditions of use, and using information from
participants for research, as just suggested, would in most cases be in
breach of those terms. The second problem is the sophistication of online
search engines. These are so powerful that it is possible to identify an
individual merely by putting in an unattributed but direct quote from a
social networking site word for word in a search engine (even ‘cloaking’
a quote does not guarantee that it cannot be identified). There is a
growing view that ‘anonymization’ of data is increasingly an impossible
thing to guarantee.33 From a data protection perspective if you are
collecting personal data, you must obtain an individual’s consent. The
safest thing to assume when data has been collected from an online
source is that it is identifiable and as such can only be collected with
consent.
This also applies when using more traditional methods. If anonymization

cannot be guaranteed then assurances of this kind must not be made.
The emphasis should instead be of the confidentiality of the data
collected e.g. that only those identified and for whom consent has been
given will have access to the raw research data; others without such
consent will only see the data in aggregated format.
Specified, explicit and legitimate purpose

To meet the requirements of the second data protection principle – to
ensure that data are collected for a specified, explicit and legitimate
purpose – researchers must clearly state the purpose of their data
collection (e.g. ‘for market research purposes’) in research preambles,
recruitment documentation, etc. This is an important and generally
positive legal requirement which has some knock-on benefits for
33
Ohm, P (2010), Broken Promises of Privacy: responding to the Surprising Failure of
Anonymisation.

research. Research techniques have long been used by other sectors

(particularly direct marketing) conducting exercises such as the ubiquitous
‘lifestyle survey’ to enhance prospecting and profiling, or the use of
‘sugging’ (selling under the guise of research) techniques to obtain sales
leads. Now, if research techniques are used to tempt individuals into
taking part in an activity which is actually for another purpose (e.g.
questionnaires being used for direct marketing) they must be fully
informed about the purpose so they can make an informed decision
whether to take part. In addition to data protection legislation in
Europe, there is also legislation regulating unfair commercial practices
and distance selling, which means that those engaged in commercial
practices such as selling and direct marketing, would be in breach of
these regulations if they were to misrepresent their activities as being for
‘research’.
The Directive also gives rights to data subjects, one of which is the right
to object to direct marketing. The Privacy & Electronic Communications
Regulations 2003 enables individuals to exercise this right via telephone
direct marketing. Do not call/preference lists operate in most European
countries where individuals can register their personal (and in some
countries business/corporate) telephone numbers, with the purpose of
opting out of being contacted for direct marketing purposes. Although
market research is exempt from having to screen against most of these
lists (e.g. the Telephone Preference Scheme in the UK, the Do not Call list
in the US, etc), as individuals are only opting out of direct marketing,
some research organizations and clients screen research samples against
them. This is partly in the belief that subscribers to such services are less
likely to agree to participate in research projects (although this is not
always the case) and to reduce the likelihood of respondent complaints
and dissatisfaction. However, such decisions introduce a bias (or another
bias) to the sampling methodology by excluding potential respondents;
subscribers to opt out lists, for example, tend to be biased towards the
higher end of the social demographic.
In other environments this right is enacted via the use of so-called ‘Fair
Processing Notices’ where a data subject can opt in or opt out of direct
marketing (the ubiquitous tick boxes found at the end of any paper or
online application forms). The emergence of this concept is noteworthy
for research in several respects. Some of these statements can include
opting in or out of research in addition to direct marketing, although
there is no legal obligation to do so. This is sometimes because the data
controller has elected to be more stringent than the legal requirements,
but often it is due to poorly worded notices, together with a weak
understanding of the legal requirements. Either way, if research has been
included in the wording of fair processing notices, the wishes of the data
subject must be respected, and any respondents which have elected not
to be contacted for research purposes must be screened out of research
samples.

Within the research community the requirements of the two first

principles of Article 6 have been summarized and interpreted into two
practical concepts: transparency by ensuring individuals have a clear and
unambiguous understanding of the purpose(s) for collecting their
personal data and how it will be used; and consent by ensuring it is given
at the time that the data are collected and giving respondents the
mechanism to opt out of any subsequent uses of the data to which they
object.34
Adequate, relevant and not excessive

The third data protection principle of only collecting adequate, relevant
and not excessive data is an interesting issue for research. On the positive
side this argument could be used by researchers to restrict clients from
asking pointless and unnecessary research questions, but on the other
side how can a researcher reasonably know whether a question will be
relevant until the data are gathered and insight identified? Practically
researchers can apply this in general terms, e.g. if the aim of a research
project is to find out opinions on the latest mobile phone technology
there is no need to ask for a respondent’s inside leg measurement.
The same can apply with passive data collection techniques, online
technology may enable researchers to gather large amounts of personal
data relatively easily (by using data scraping techniques which are
software programmes which enable vast amounts of personal data to be
‘scraped’ off internet such as social media sites), but personal data should
only be collected if it is to be actively used for research (and again this
should only be done with the consent of the individuals whose personal
data are gathered). If the personal data are not going to be used for a
legitimate research purpose, it should not be gathered as to do so would
be excessive and not relevant, and hence in breach of data protection
legislation. Ultimately research projects should always be designed with
the minimum amount of demands being placed on respondents and so
any legislative requirement which stifles boring, repetitive and
unnecessary research questioning should be welcomed.
Accurate and up-to-date

The fourth and fifth principles of keeping data accurate and where
necessary up to date, and for no longer than is necessary, can also be
challenging for research. Accuracy is an interesting concept in terms of
opinion and attitude (see Chapter 3 for more about this) and
respondents are often not the best witnesses to their own lives (Kearon &
Earls, 2009). Ultimately researchers have to rely on what the respondents
34
The Data Protection Act 1998 & Market Research: Guidance for MRS Members (September
2003) first introduced this idea and this has been adopted elsewhere.

tell them. The key requirement is that researchers accurately record what
respondents tell them and do not inaccurately attribute information to
the wrong respondents. The requirement for keeping data accurate and
up to date also has practical implications in the management of research
panels, where processes need to be in place to ensure that panellists are
frequently contacted (to make sure they are still active) and to ensure
any recorded information about the panellists is accurate and frequently
updated. For simple ad hoc projects the requirement for accuracy applies
but there is not a practical need to keep the data up to date. As for data
retention this will usually be determined by the requirements of the
research e.g. a longitudinal study will usually require longer retention
periods, ad hoc much less although retention for some period for quality
(and accuracy) checks is needed. In the absence of specific contractual
requirements, BS ISO 20252 sets out practical periods for data retention
(one year for primary data, such as questionnaires, and two years for
secondary data such as research reports).35
Controllers and processors

In addition to the principles, one of the other big issues concerning the
application of data protection is defining the controller and processor in
a research project, particularly remembering that the majority of the
burden for data protection rests with the controller. In most cases, where
researchers build sampling databases (such as access panels), acquire or
purchase lists for sampling purposes, or collect research results in their
own or company name they will become controllers of the data and must
ensure that the principles detailed in Article 6 must be adhered to.
Researchers and research companies will tend to be only processors if
they undertake research wholly in the name of the client, some ‘field and
tab’ exercises which involve the collection and tabulation of research
data with analysis and reporting done elsewhere, and where researchers
are sub-contractors who might only undertake part of a research exercise
on behalf of another research supplier and/or a client. Researchers can
often be joint data controllers with clients or other researchers
depending on the circumstances. In these cases the burden for meeting
data protection requirements would be shared among all parties.
Data security
All controllers must implement appropriate technical and organizational
measures to protect personal data against accidental or unlawful
destruction or loss, alteration or unauthorized disclosure or access. In
today’s technological environment this is one of the toughest challenges.
35
There are also some limited exemptions within the Directive for following requirements
such as data retention for data collected for statistical and historical purposes but this has
limited coverage to only some statistical quantitative research.

Within Europe, the Directive has endeavoured to control this by limiting

personal data transfers to within the European Economic Area (the EU
plus Norway, Iceland and Liechtenstein) or to countries which are deemed
to have adequate levels of data protection (currently a very short list of
Switzerland, Guernsey, Argentina, Isle of Man, Faroe Islands and Jersey,
with Canada having been approved for certain types of personal data
and two specific arrangements with the US; the ‘safe harbor’ for US
companies, which enables US organizations to have their processes
evaluated for compliance with EU data protection requirements and the
Passenger Name Records from airline passenger data for fighting
terrorism and transnational crime). If data are to be transferred to
outside the EEA or countries not on the approved list it must be done
either with a specialized data processing agreement which includes
adequate levels of data protection, the data subjects’ consent or binding
corporate rules for movements within a global organization.
With so much research data flying around some is bound to get lost, and
the more handovers of data between parties there are, the bigger the
risks. A very simple research project would involve at least three of four
data handovers between client, researcher, sub-contractors and possible
interviewers (see Figure 8 for a ‘typical’ very basic research data flow).
The client’s security is likely to be good, so are most researchers, but
what about an interviewer? Someone who is not an employee of a
company, probably works from home and is not likely to have Fort
Knox-type security arrangements – how do you keep the data secure
there? Or outsourcing arrangements where research data processing and
collection is conducted in another country where direct and frequent
monitoring of arrangements might be difficult? This is without even
considering multiple sub-contractors, multi-country projects where the
risks can rise ten-fold. Reports of general data loss incidences are
increasing. The British government seem particularly prone to losing vast
quantities of their citizen’s data; two CDs containing 25 million personal
records of families of child benefits claimants went missing in the post in
2007, a hard disk with 3 million details of candidates for the driving
theory test was also lost in 2007 by US subcontractors working on behalf
of the Driving Standards Agency and multiple smaller losses have
subsequently been reported, some of which included research data being
lost (mercifully these are currently few and far between).
However, ensuring adequate technical and organizational measures for

ensuring the security of research data will continue to be a challenge for
the sector. To meet such legal requirements, ISOs have a role to play.
BS ISO/IEC 27001:2005, Information technology – Security techniques –
Information security management systems – Requirements was launched
in 2005 with the aim ‘to provide a model for establishing, implementing,
operating, monitoring, reviewing, maintaining and improving an
Information Security Management System (ISMS)’. BS ISO/IEC 27001 is one
way in which research companies have endeavoured to address the need

Figure 8 – The possible data risks for a simple research project
for ‘appropriate technical and organizational measures’ as defined in the

European Data Protection Directive. BS ISO/IEC 27001 requirements are
generic and are intended to be applicable to any type or size of
organization; it requires research organizations to adopt the
‘Plan-Do-Check-Act’ (PDCA) model to ISMSs. One of the main benefits of
BS ISO/IEC 27001 for the research sector is that the standard provides

some evidence that a company’s ISMS is sufficiently robust to keep

research data secure. Some clients are now insisting that research
companies comply with this standard before they allow their data to be
shared; ironically one of the biggest clients to insist on this measure is
the UK Government and its departments (as a consequence of their data
losses as opposed to being a pre-loss requirement!).
Specific types of data

The Data Protection Directive specifies ‘special categories of data’ for
which more stringent requirements apply. The types of data this covers
are race, ethnicity, political opinions, religious or philosophical beliefs,
trade union membership, and the processing of data concerning health
or sexual life. When these types of data are collected, explicit consent is
required as opposed to unambiguous consent. The vast majority of
research projects will collect at least one type of sensitive data (ethnicity,
for example, for demographic sampling purposes). For most research
projects this means that at the commencement of a data collection
exercise respondents clearly understand, before answering any questions,
what their responses will be used for (e.g. research purposes), that they
can withdraw at any time, refuse any question, etc. This will usually be
achieved by a sufficiently detailed preamble (although this is not always
sufficient, in Italy, for example, pre-obtained written consent is required
before sensitive data can be collected).
Other examples of special requirements for specific types of data are

more prevalent outside of Europe. For example, in the US the Privacy
Rule of the Health Insurance Portability and Accountability Act of 1996
(HIPPA) establishes minimum Federal standards for protecting the privacy
of individually identifiable health information and requires healthcare
providers to establish conditions under which those covered can provide
researchers with access to and use of personal data. Also, in the US, there
are regulations protecting the rights of children using the Internet. The
Children’s Online Privacy Protection Act (COPPA) places parental control
over information collected from children online. The Act requires
commercial websites to post privacy policies, provide notices, obtain
verifiable parent’s consent before collecting personal information from
children under 13, and provide parental access to the information
collected from their children – with the right to amend the data and
opt-out of future data collection. These requirements are not dissimilar
to the rights of European data subjects via the Data Protection Directive,
which incidentally does not currently specify any specific or additional
rights for children. Many of the issues regarding children and vulnerable
adults have been dealt with under wider legal provisions aimed at
providing protections beyond just personal data e.g. Scotland’s Protection
of Vulnerable Groups Act 2007, Safeguarding Vulnerable Groups Act 2006
in England and Wales, etc.

Freedom of information
Online e-privacy
Other pieces of legislation have been introduced with the purpose of
addressing data protection and privacy issues but in specific contexts. For
example the European Directive (2002/58/EC) ‘concerning the processing
of personal data and the protection of privacy in the electronic
communications sector’ (Directive on privacy and electronic
communications) – called the ‘e-Privacy Directive’ for short – aims to
address data collected by invisible processing. There are a number of
techniques used by researchers which employ invisible processing
techniques – the main ones are cookies and digital fingerprinting.
A cookie, of the non-food variety, is a piece of text that a web server can
store on a user’s hard drive with the purpose of storing information and
later retrieving it. Researchers use cookies for a number of reasons: to
ensure that respondents only participate once in any given research
project; to protect against unauthorized intrusion into protected pages
and/or sites; to protect against fraud, etc. Similarly, digital fingerprinting
is a form of technology which invisibly and automatically collects
individual’s computer browser and hardware settings for the purposes of
producing a unique ID for a computer. The benefit for research of such
technology is that it helps to identify potentially fraudulent respondents,
particularly panellists who try to join research panels using multiple
identities through the same computer.
These are clear research quality control reasons for taking advantage of
these technologies, but it must be done in adherence with the
requirements of the e-Privacy Directive. The main requirement of the
Directive is that such technology cannot be deployed without obtaining
the consent of the user, having provided them with clear and
comprehensive information about the purposes of the storage or access
to the information that is being collected. This could be achieved by
providing, within research preambles, some top line cookie information
and why they are needed, providing a link through to more detailed,
technical information about specific types of cookies used, how
information is stored, who would have access to the information, how to
disable them, etc. Similarly when using digital fingerprinting technology
in the management of a research panel for example, to explain within
the terms and conditions for the panel that such technology is used, for
what purpose, etc.
Freedom of information
It is also worth mentioning the other side of the data equation –
freedom of information. Whereas the focus of privacy legislation is to
protect personal data and the rights of the individual to have a private
life, freedom of information is all about promoting openness by enabling

free access to information (whilst still protecting personal data which is

generally exempt from release). Most countries have enacted some form
of legislation to enable its citizens access to public and/or government
information. Freedom of information legislation can have an impact on
government or publicly funded research. Whereas most research codes
place an obligation on researchers to release technical information to
support research results that are released into the public domain (if
research results are used in press releases, or marketing campaigns for
example); freedom of information legislation can force all research results
– including those never meant for public release – into the open. On the
up-side this forces researchers to ensure that all public research – not just
those results meant for publication – are of the highest quality and able
to withstand public scrutiny. The downside is that rather than run the risk
of unfavourable, and potentially problematic research results being
released, some publicly funded bodies/government departments just do
not ask the difficult research questions any more. This is less a problem
for research and more a problem of a free society, where freedom of
information rights usually come in tandem with freedom of expression;
these two concepts are often difficult to balance.
Other communications
So when it comes to legislation should researchers only be worried about
data legislation? Well, no. There are various strands of legislation which
impact on the technology that research uses and/or the environment in
which research tends to be undertaken.
There are various pieces of legislation around the world designed to

regulate unsolicited commercial emails (so-called ‘spam’) and text
messages (SMS) to individuals. In Europe unsolicited spam and SMS can
only be sent to individuals who have ‘opted in’ or given their ‘active
consent’ to receive such commercial communications unless in an existing
customer relationship. In most European countries, market research
emails and SMS fall outside the legislation as they are not defined as
commercial communications.
There are also restrictions on the use of telephone diallers. Often

telephone research units will use automated diallers which can be
programmed to automatically initiate telephone calls based on
assumptions on the number of telephone interviewers, length of call,
frequency of calls, etc. The benefits to the research company are in terms
of cost-saving and labour efficiency. However, the problem with this
approach is that such algorithms are not fool proof, and telephone
numbers can be called when no interviewers are available and a so called
‘silent’ or ‘abandoned’ call results (i.e. someone picks up the ringing
telephone only to discover silence at the other end of the call). The UK
regulates this through the Communications Act 2003, but similar

Types of research and research environment
legislation can be found elsewhere in Europe and the US. The primary
focus of the UK legislation is to regulate the medium e.g. automated
diallers; whereas other countries such as the US have focused on both the
medium and the purpose e.g. diallers used for telemarketing, plus
restrictions on the use of auto-diallers when calling mobile phone
numbers. The difficulty with legislation in this area is that the traditional
argument that market research is not a commercial activity, and hence
should be given special treatment, is not necessarily sufficient (not in the
UK context for example). Many consumers who receive such calls perceive
them as a nuisance irrespective of whether they are generated by market
researchers or direct marketers. The legislation therefore places the same
restrictions on all users of such equipment irrespective of intended
purpose.
It is not just the telephone and online research environment where legal
restrictions can be encountered. On the ground, conducting research in
shopping areas or in-street can be just as problematic. Local laws
empowering communities to create ‘No Cold Calling Zones’, closed and
gated communities, or commercial shopping districts restricting any
non-retail related activities (such as street interviewing or fund raising)
can all make face-to-face research interviewing difficult. Often the focus
of these restrictions is not to stop research interviewing, nor are they
necessarily underpinned by sufficient legislation to enable residents or
town centre managers to stop interviewers from doing their job, but the
end result is that these restrictions curtail what researchers can practically
do.
Types of research and research environment

The other area where legislation impacts upon research is where research
is conducted and the products and services being researched. If research
is being conducted in a hall test, a viewing facility or some other location
face-to-face, the researcher needs to be aware of health and safety, and
disability legislation so that they ensure respondents are kept safe, that
they avoid potential harm and are ensured equal access. Similarly, if
products such as food are being tested they need to be aware of relevant
food handling legislation which details how different types of food
should be stored, heated, refrigerated, or food labelling regulations
regarding what details such as ingredients need to be specified. Similarly,
for restricted products such as cigarettes or alcohol, researchers need to
be aware of legal requirements such as age restrictions, licensing laws
(for distributing alcohol) etc.

Is legislation good for research?

Why should we be worrying about data protection, or indeed any
legislation, in relation to the topic of research and quality? The majority
of world citizens could live full and happy lives without ever having
participated in research; it is not an activity which everyone feels a
burning desire or need to take part in. However, for research to continue
and to thrive, people must be prepared to participate in it, and this is
where legislation is becoming increasingly important. We all like to be
asked what we think about things, to believe that our opinion counts
and research has prospered on this basic human compulsion. But in
today’s society less and less is given for free. Most people are prepared to
freely give a few minutes of their time to answer some questions, very
few would participate in an hour and a half focus group without some
incentive to participate (and the amount needed to encourage
participation is growing steadily). On top of the increasing material
incentives needed to gather people’s views, is the broader concern about
what might happen if they participate. People would be very much more
reluctant to part with their opinions if they thought that in some way
that data might be inappropriately shared, seen, read or put in an
inappropriate context which may result in them being harmed or
adversely affected.
Data protection is vitally important in stemming these concerns. The

more protected a respondent feels, the more likely they are to agree to
share their data. In theory this should also reduce the likelihood of false
opinion being given (although experience has shown that all the data
protection in the world does not stop respondents being less than
truthful on certain topics due to wider concerns such as status anxiety).
Nonetheless legislation is not without its difficulties. In addition to the
practical limitations such as who can be contacted, and how, etc., there is
the problem of inconsistent, and in some cases downright contradictory
legislation, on topics such as data protection. In Europe for example
there is the one Directive from which all the domestic European
legislation derives, but each country has interpreted and implemented
the Directive in different ways. For global, cross-border research studies
this can make life very difficult, with different approaches having to be
adopted in different countries in order to ensure adherence to the
legislation. The European data protection framework is currently being
reviewed, with calls for greater harmonization to resolve some of these
problems, but this is not necessarily the best answer for researchers,
particularly if the laws of the countries with the most stringent regimes
are applied as opposed to those with the most liberal.
Irrespective of these difficulties, legislation does have one main benefit: it

provides a robust framework to support the ethical codes that have
served the research sector so well for over 50 years. We discussed earlier
in Chapter 4 the limitations of self-regulation, due to restrictions in

Is legislation good for research?
coverage (usually limited to membership of a professional body or trade

association) and resources. Legislation, albeit focused on wider issues such
as data collection and protection, considerably strengthens the research
code principles which should also strengthen the quality of research
produced. Whereas clients might have pushed when researchers said no
to a proposed approach due to the code principles and rules, very few
are going to push to conduct research illegally. However, for the
researchers and research sector to continue to benefit from the relatively
manageable legislative environment, it needs to continue to have strong,
ethical code principles, including informed consent, respect for
respondents and avoiding harm – in order to ensure that people continue
to be willing to take part in research.

7 Quality management and

BS EN ISO 9001
Quality assurance or management and the corresponding international

standard BS EN ISO 9001 was introduced in Chapter 5. BS EN ISO 9001 has
been widely implemented by market research companies and was a
major input in the development of standards specifically for market
research. In this chapter we examine the scope and coverage of
BS EN ISO 9001, its value to a research provider and discuss how each of
the main headings of its requirements can be applied to market research
processes. These include documented quality management systems,
management responsibility and commitment, making available the
resources needed, design, planning and control processes for research,
subcontracting of services and monitoring and measuring to achieve
quality objectives. How companies can demonstrate compliance to
BS EN ISO 9001 through assessment and certification is also covered.
Before national or international standards for the whole market research

process (rather than just data collection) were developed, some research
companies, including large ones, in the UK and elsewhere, decided to
implement and be certified to the international standard for quality
management: BS EN ISO 9001, Quality management systems –
Requirements.36 In the UK the first market research company was
certified to this standard in 1992. Since then some research companies
throughout the world remain or have become certified to
BS EN ISO 9001. However most of these companies have also
implemented either national standards for market research or,
increasingly, international standards and especially BS ISO 20252, Market,
opinion and social research – vocabulary and service requirements.
Research companies now considering being certified to standards are
more likely to choose research-specific standards such as BS ISO 20252
rather than BS EN ISO 9001. Some will choose to meet both standards
and as will be discussed in Chapters 9 and 11, adding BS EN ISO 9001 to
36
Then and now, BS EN ISO 9001 is part of the BS EN ISO 9000 series of standards. Currently
the series includes, as well as BS EN ISO 9001, BS EN ISO 9000:2005 which provides
definitions, introduction to the concepts behind the standards and some guidance on
application of the standards. There is also BS EN ISO 9004 which provides guidance on a
more TQM approach (mentioned in Chapter 5). Withdrawn standards in the series include
ISO 9002 and 9003 which were similar to BS EN ISO 9001 but with more restricted
application.

7 Quality management and BS EN ISO 9001
BS ISO 20252 requires little extra implementation but not vice versa;
quite a lot extra may be required to extend BS EN ISO 9001 certification
to also cover BS ISO 20252.
BS EN ISO 9001 was a major input into the development of national and
later international standards for research. First of all, it brought to the
attention of the research industry the concept of national and
international standards and standard bodies. In turn this stimulated
debates about quality in market research; what it means and how it is
achieved. One view was that the quality management approach behind
BS EN ISO 9001 could very usefully be applied to all the key market
research processes (although many may not have been aware of it, the
quality management or assurance approach was already used in
fieldwork standards, such as IQCS (see Chapter 8) in the UK). At the other
end of the spectrum, some did not like what was perceived as a ‘box
ticking’ culture displacing real quality (i.e. quality as excellence).
Another view, and in the end the dominant one, was that an
BS EN ISO 9001 approach was basically sound but a research-specific
variant or new standard was needed as well. As will be discussed later in
this chapter, BS EN ISO 9001 underpins quality assurance but does not
itself require any specific quality level; a research company could ensure
quality consistency but work well below recognized good research
practice, e.g. interviewer validation levels, although this would obviously
jeopardise any business in the longer term. The upshot was a consensus
to develop new quality standards for the whole market research process
incorporating BS EN ISO 9001 concepts and coupled with developing a
similar style of assessment and certification regime. Because of both its
historical and ongoing relevance to the industry, a book on quality in
market research, therefore, needs a chapter on BS EN ISO 9001; an
understanding of what the standard is (and is not) and how it can be
implemented by a market research company.
What is BS EN ISO 9001?

In Chapter 5 we introduced quality assurance and quality management.
BS EN ISO 9001 is a widely recognized international standard for quality
management and a chapter on it can also double as a more detailed
consideration of how quality assurance and quality management can be
applied in market research. However, to avoid misunderstanding, we
should make it clear that effective quality assurance does not have to
equate to BS EN ISO 9001. There can be, and are, other effective models,
and even if a quality assurance model matching this international
standard is adopted, this does not imply formal assessment and
certification.

What is BS EN ISO 9001?
As we have just stated, BS EN ISO 9001 is a standard or model for quality

assurance and, therefore, for managing quality; for ensuring that what is
produced is to specification and to requirements. The approach of the
model is that of a formal and documented quality management system
which, if effectively implemented, will (it is asserted) ensure quality
assurance and in some sense quality, especially quality as meeting client
and other requirements. BS EN ISO 9001, however, is not a standard for
any particular quality level of product or service. The desired quality level
has to be defined alongside the quality assurance system; this is not to be
found within BS EN ISO 9001 itself. Instead the specification of service or
product quality level has to be sought in appropriate external standards
(in market research these may be as per BS ISO 20252) or defined by the
organization implementing BS EN ISO 9001 to meet its own and its
customers’ or clients’ requirements. Once these are defined,
BS EN ISO 9001 should help ensure that they are met consistently and,
therefore, it is also referred to as a standard of capability; a capability to
meet the required levels of product or service standards. The level of
defined quality (in the sense of quality as excellence) does not have to be
high; if the aim is to produce cheaper products for those who cannot
afford anything else, so be it. But, with an effective BS EN ISO 9001
system, these products will not drop below some defined level of quality
(or go beyond it). The failure to grasp the idea of capability was in many
cases the reason for some hostility to BS EN ISO 9001 and this mistaken
perception lingers on to some extent in attitudes to the industry
standards such as BS ISO 20252.
BS EN ISO 9001 is a general model and standard for quality assurance in

the sense that it can be applied to any sort of activity and any sort of
business or for that matter any sort of organization including
non-commercial ones (e.g. some local authorities, police and health
bodies have taken it up). Because the standard is so wide in scope, it has
to be adapted and ‘translated’ to the needs of particular industries and,
to assist implementation, many industries have produced industry-specific
guidance documents translating what the requirements of the standard
mean in a particular context. The further the particular industry is from
this type of business, the more ‘translation’ may be needed to apply it
effectively. This point certainly applies to market research.
Not only must BS EN ISO 9001 be translated to specific industries but it

must be tailored to meet the unique requirements of particular
organizations. Because no two organizations are the same, the
requirements of BS EN ISO 9001 have to be met through a quality
management system developed specifically for the particular company
and to fit that organization’s culture and processes. The BS EN ISO 9001
standard, therefore, is not a system which can be simply taken off the
shelf and followed. A company ‘doing’ BS EN ISO 9001 must instead put
significant effort into developing its own quality management system to
meet its unique needs as well as the requirements of the standard. If

desired, the system can also incorporate requirements not covered by

BS EN ISO 9001 (e.g. in financial processes). Once the standard has been
implemented, the organization can be independently assessed and
certified as meeting BS EN ISO 9001 and this process will be described
later in this and subsequent chapters. However, the benefits of
BS EN ISO 9001 should be far greater than any kudos from the
BS EN ISO 9001 certificate. An organization may well choose to meet the
standard but not to go to the expense of third party assessment.
What BS EN ISO 9001 requires

To learn in detail the requirements of BS EN ISO 9001 you must buy and
read a copy of the standard. In this section we confine ourselves to a
discussion in more general terms of the underlying concepts and main
headings of the requirements and more particularly how these can be
applied by a market research company. The standard has eight major
Clauses and the first three of these are formal introductory sections
including ‘Scope’. This Clause states the standard can be used by
organizations wishing to demonstrate an ability to provide products (the
term is defined to also cover services) that consistently meet customer
requirements as well as relevant regulatory compliance (i.e. can show
effective quality assurance is in place) and so enhance customer
satisfaction. Building in processes of quality improvement is also specified
so the quality management system needs to be dynamic to this extent.
The scope also states that the standard and its requirements can be
implemented by all organizations regardless of type, size and the product
or service produced. All requirements specified in the standard have to
be met except where they cannot be applied because of the nature of
the processes carried out – ‘exclusions’ (examples are discussed later); in
other words you cannot pick and choose which bits to follow. The
requirements that need to be met are set out in five Clauses and we will
now discuss each of these in turn.
A documented quality management system

BS EN ISO 9001 requires an organization to have a documented quality
management system (Clause 4). As the introduction to the standard
states, a ‘process approach’ is recommended in developing the system;
the activities of an organization are seen as a series of discrete process
steps, each of which has inputs, something done to the input to produce
an output. Whether or not it is usual to think in this way, a market
research project has such process steps; taking a client brief, preparing a
proposal, drafting questionnaires, etc. A documented quality
management system needs to define these steps and what needs to be
done to ensure that each fully meets requirements. The main point of a
documented quality system is to ensure that ‘what needs to be done’ is

done consistently and in every applicable case (or in market research,

every project). As such, the procedures of a documented quality
management system are, therefore, more than good practice; internally
they are mandatory. Conversely it is recommended when drafting a
system to include few optional good practices; this can confuse users of
the system. The system is required to cover not only the ‘productive’
processes of an organization but also the processes created by having the
system itself, including controlling documentation (e.g. how it is
authorized and revised). The requirement for a documented quality
system is the same regardless of the type of organization or its business
and there are no special issues here arising from the nature of market
research processes. Arguably, the skills needed to analyse processes and
prepare documents are more likely to be found in a market research
company than many other businesses.
The standard specifies some of the documentation required, specifically

statements of quality policy and quality objectives (discussed shortly), a
quality manual, procedures and other documents and records
‘determined by the organization to be necessary to ensure the effective
planning, operation and control of its processes’ (our italics). The quality
manual needs to describe the principal processes of an organization and
how the requirements of the standard have been met including by
reference to procedures and similar documents. Any ‘exclusions’ –
requirements that cannot be met because of the nature of processes –
also need to be defined. The quality manual can be short formal
document, which can be used for quality purposes but can also provide
reassurance to clients during procurements, etc.
In most quality management systems, procedures and a procedure

manual are the core type of documentation. Typically they describe
process steps including what is done consistently to ensure quality
requirements are met. A procedure for preparing proposals, for example,
may cover how responsibility for drafting is allocated, a standard
template of proposal coverage, who within the organization needs to
provide inputs such as fieldwork costing and who is to review and
approve the document prior to delivery to the client. In Chapter 11 we
provide examples of written procedures37 and this should provide a
better understanding of them than trying to set out abstract drafting
principles. Procedures are normally prepared to cover processes relevant
to all or many projects but there is also often a need for ‘work
instructions’ to cover specific project requirements (e.g. a specification to
the field department or an outside subcontractor) and the use of these
can also be regarded as part of the quality management system;
procedures may specify when and where such written instructions are
required. Also, quality plans (referred to in the standard) may be part of
proposals or project instructions.
37
The specimen procedures are to meet BS ISO 20252:2012 but the format is equally relevant
to BS EN ISO 9001.

The standard also refers to the need for records. Critics of BS EN ISO 9001
have claimed that the consequences of implementation is a bureaucratic
explosion with a proliferation of records; a form for everything and a file
for every form! There is some truth in this but only where quality
management systems have been badly planned. There is no requirement
for records just to meet the needs of the system itself38 or to keep
assessors happy. Depending on the processes there will be some need to
record what has been done as part of the process but this need should
exist regardless of implementing the standard. If, for example,
interviewers’ work is validated in systematic ways (e.g. by re-contacting a
proportion of respondents) some record is needed to show the agreed
number of interviews are validated, which interviewers’ work was
covered, what was found out and what was done about any problems
identified. BS EN ISO 9001 requires quality improvement to be an aim,
including in systems, processes and outputs, and records of where things
have gone wrong, are an essential starting point for making positive
changes. In market research work, documents of one form or another is
the physical output of the activity and often these themselves provide
the record of the process; a form is not needed to show a questionnaire
has been drafted, the questionnaire is its own record and its conformity
to such as a standard format is there to be seen.
Finally, in the section on requirements for documented quality

management systems, the need to control documents and records is
specified. This is not difficult to understand or see the point of. One
requirement concerns version control; if, for example, procedures are
revised staff need to be able to identify the up-to-date version and work
to these. This also applies to documents produced in the market research
process and good version control of documents, such as questionnaires
which usually go through several drafts before finalization, can avoid real
and expensive problems such as the version before last-minute
client-requested changes were implemented being used in the
interviewing programme. It is explicit that the documented quality
system can all be in the form of digital files; hard copies of procedures,
records etc. may not be thought necessary at all. Computer systems can
also simplify areas such as version control and filing records so that they
are easily retrievable (a requirement of the standard).
Management responsibility
Companies implementing BS EN ISO 9001 or similar standards generally
have a quality manager responsible for the day-to-day administration of
the system. The standard uses the term ‘management representative’.
Few companies would have this as a job title but it implies an important
38
Perhaps this is not entirely true. The standard requires, for example, internal audits to be
carried out and documented. Internal audits would not necessarily have been carried out if
the system had not been implemented so there would be no records of them.

point. Implementation of standards such as BS EN ISO 9001 is not

something just involving a few specialists; it requires commitment
throughout the organization and specially commitment of senior
management – ‘top management shall provide evidence of its
commitment to the development and implementation of the quality
management system and continually improving its effectiveness …’
(Clause 5). This commitment specifically requires that at regular intervals
senior management review the working of the quality management
system using various inputs and leading to any required changes and this
is one means of affecting improvement. Again, it is implied that the
system cannot just be left to the quality manager or equivalent.
One task for senior management, when the system is being developed, is
to agree a quality policy and set quality objectives. The quality policy is a
matter for the organization and should define what it hopes to achieve.
A possible policy statement for a market research company (many others
would also be valid) might be:
The quality policy of this company is to ensure that client

requirements are fully and efficiently met and that opportunities for
improving the quality of our research services are identified and
implemented. As one means of achieving this policy, the company
has implemented a quality management system conforming to the
requirements of BS EN ISO 9001.
In this statement efficiency is mentioned; the company recognizes the

need to make a profit or at least cover costs when pleasing its clients.
The statement also recognizes that a formal quality management system
is just one way that quality is pursued; quality as excellence also needs to
be at the heart of the company. Quality policy statements can be linked
to parts of mission statements, if such are thought desirable.
Quality objectives are required to be measurable, not vague, goals. These

are probably easier to set in a manufacturing company than in businesses
such as market research. Reduction of waste material and scrappage
rates, improvement in tolerances, increases in speed of process time,
minimization of stock levels and reduction in product returns could all be
measurable quality objectives for a manufacturer. For a market research
setting comparable metrics are not so obvious. Some possible processes
where measurable objectives could be set include data collection – e.g.
reduction in detected interview errors identified through validation and
increasing response rates – or data processing – e.g. reducing coding or
data entry errors. Client satisfaction improvement could also be a
measurable objective and measuring this is in any case a requirement of
BS EN ISO 9001 (customer satisfaction improvement is likely to be an
objective of any business). The need for profitable efficiency could also
be related to a measurable objective such as reducing project costing
over-spends.

Resource management
Another important task for management is to ensure adequate resources
are made available to implement the quality management system and
more generally ‘enhance customer satisfaction by meeting customer
requirements’ (Clause 6). Resource classes specified are human,
infrastructure and the work environment.
All market research companies will be aware of the importance of people

and human resources, which are discussed in Chapters 4 and 10. The
requirements of BS EN ISO 9001 for human resources concern the need
for job competence and for identifying and delivering education and job
training. Some level of formality is needed to show, for example, that
training needs are regularly evaluated and delivered. Few research
companies will have difficulty in meeting this requirement.
Infrastructure includes workspaces, services such as communication and

transport to deliver products and the equipment used in processes.
Normal office space and associated services used by research companies
are normally adequate and probably all that is needed is for the quality
manual to simply confirm that this is the case. Process equipment for
market research is predominantly IT; the computer network and its
external links. Recording equipment may be important for qualitative
research companies and there may be specialist equipment needs for
some areas of research. The need is obviously to have the kit available in
adequate quantities and have arrangements in place for maintaining it
and having back-ups (including data) if things go wrong. Again none of
this would be considered surprising for a research company. There is also
the question of the availability, effectiveness and reliability of software.
Effectiveness is covered in a later section (Subclause 7.5, production and
service provision) of the standard. Availability needs to take account of
software licensing.
Product realization – carrying out research

By far the longest section of BS EN ISO 9001 requirements is Clause 7 –
product realization and for a market research company this means all the
research processes; doing the projects. It is particularly this section where
some translation is needed to relate the requirements to how market
research is actually carried out and nowhere more so than in Subclause
7.3 – design and development. An important general point to make,
however, is that a reasonably well-organized research company will not
need to make major changes to the way research work is carried out just
to meet BS EN ISO 9001. The processes may need formalizing as
procedures but can be carried out as before, perhaps in a more consistent
way; good practice becomes everyday practice. Also some records of
processes may need to be kept where they were not kept before or

perhaps previously poorly kept. In this respect BS EN ISO 9001 is different

to standards such as BS ISO 20252, which do prescribe how some
processes need to be carried out.
The design requirements of BS EN ISO 9001 (Subclause 7.3) can seem a

puzzle to a market research company. As set out in the standard, the
requirements for design appear to fit a manufacturer making a physical
product and particularly to meet the requirements of an individual
customer e.g. a bespoke manufacturer of timber framed buildings.
However, much industrial product design work is ‘proprietary’; the
company owns the design which is developed to meet the needs of a
market segment or anonymous group of potential customers, and at least
most of the requirements of this part of BS EN ISO 9001 would also apply
and be understandable in this situation. Of course, many businesses do
not carry out any design work themselves. Instead they make a product
or supply a service to match a detailed specification from the customer
and this is the case in market research; for example, ‘fieldwork only’ or
‘field and tab’ act as subcontractors carrying out data collection or data
processing to the detailed instruction of other research companies (the
sampling method, mode of interview and the questionnaire are all
supplied or specified). In such cases the company can treat design as an
‘exclusion’ and state in the quality manual that these requirements are
not met because no design processes are carried out.
It is assumed in the standard that design work is carried out in discrete

and successive steps after a contract for the work is agreed with the
customer; the customer’s precise requirements are defined and agreed,
the design work is planned and the required inputs identified, outputs of
the design process may be in the form of manufacturing specifications,
models or prototypes with testing (verification and validation) before
use. This model no doubt matches the common practice in many
businesses but is not very user-friendly to market research. Concentrating
just on ad hoc research, the first point to make is that the major design
work is carried out before there is any contract with the client; how the
research is to be carried out – the basic research design – is developed as
part of preparing the proposal, i.e. the quotation to the client, and no
charge is normally made. Few other sectors do major design work free of
charge in this way.
Some further design work may come after commission of the project,
such as questionnaire development, details of sampling and analysis
specification but these are subsidiary and follow the main design work
set out in the proposal. Separate steps in developing proposals may be
analytically distinct but they are often carried out more or less together.
These steps include:
• identifying client requirements (the client may or may not provide a

written brief),
• problem definition, and translating these into research objectives,

• identifying the information to be collected and designing research

methods to achieve this,
• taking account of good theoretical research practice (‘textbook’),
• professional judgement of internal or external experts,
• prior experience in the subject area
• the client’s stated or anticipated budget.
The proposal is also meant to persuade clients to go ahead with the

project and use that particular research company, which may be
presented in a section on previous experience and the expertise and
experience of company staff. This section may also cover timings, charges
and commercial terms, although these are not design issues as such. It
also ought to be mentioned that proposals for ad hoc research are very
often prepared to a tight timetable; one of the client’s needs may be to
have a full proposal response within a day or two of the initial briefing.
Proposals do not prepare themselves, and staff must be selected for the
particular job and between them agree the research design, draft the
document and probably review and check it before delivery. Typically in a
smaller company, for an average type of project, the proposal and all its
design steps is the responsibility of one member of staff alone; generally
a senior researcher with the required technical skills. Often there is no
second person review (no one else may be qualified to review the
proposal). In larger companies and for major projects a number of people
may be involved, including a second review of the proposal document,
but the process is not different in essence. In both small and larger
companies, staff outside the ‘research executive’ team may have inputs
into preparing proposals and especially the data collection department,
but more often to do with timing and costing than research design. For
‘post contract’ design work – e.g. questionnaire drafting – the same
members of staff that prepared the proposal (or direct colleagues) are
often responsible, possibly with second person review (having the field
and possibly data processing managers look at a questionnaire before use
is good practice) but not in all cases. Questionnaires may also be formally
tested but full pilot research is usually carried out only if agreed as part
of the contract. The design process for continuous research projects and
multi-client work is often no different to that for ad hoc work. However,
there may be more time available, the design stages may be more
distinct and formal testing and piloting more common. The client may
even pay for the design stage of the project.
So how do market research companies implementing BS EN ISO 9001

meet the design requirements of the standard? Generally they stick to
the established processes as outlined above; they do not change the way
they work to satisfy the standard, and providing the appropriateness of
this is covered in the quality manual and related to the stated
requirements (i.e. Subclauses of 7.3), assessors will accept that in this
respect the quality management system is compliant with BS EN ISO 9001

requirements. In drafting the documented quality management system,

however, the design steps together with responsibilities for drafting and
reviewing (e.g. level of staff competence) should be described in
procedures. Proposal coverage checklists or questionnaire layout
templates (which must be included as standard) may also be included.
Clause 7 is not only about design; 7.1 defines requirements for ‘Planning
of product realization’ but these will normally be met by general
procedures for the research process, the research proposal or quotation,
such as data collection and data processing ‘work’ instructions.
Determination of requirements (Subclause 7.2) and especially the client’s
requirements are also normally covered by the proposal document or the
specification provided by the client.
A research company may carry out all processes ‘in-house’ or some may
be subcontracted, such as to fieldwork specialists. In that case,
BS EN ISO 9001 requirements are specified as Subclause 7.4, ‘Purchasing’.
Although this Clause applies to all types of purchasing, it is implicit that
bought-in inputs with a direct impact on meeting client requirements are
the main issue here. Suppliers are required to be controlled through, for
example, approved supplier lists and their performance is to be
evaluated. The product or service to be bought needs to be specified as
well as the quality assurance processes of the supplier and this may
favour suppliers which themselves meet BS EN ISO 9001 (this is implicit
rather than explicit in the standard). There is also a requirement to
ensure the product or service supplied by a subcontractor meets
requirements. In the case of physical products this may be by inspection
of the delivered product but in a service such as market research data
collection physical inspection is less feasible or effective. Areas such as the
number of completed interviews and whether the quota plan has been
set can be checked, whether interviewers carried out the interview as
required and appropriately will be less evident and assurance by
interview validation may be required; these could be specified in the
contract with the subcontractor.
The requirements under Subclause 7.5, ‘Production and service provision’,

include a number of separate areas in market research processes. Control
of production (Subclause 7.5.1) is like production planning and is
normally met by general process procedures and work instructions.
Generally the established process steps (as per Subclause 7.5.2) are also
described by procedures and their effectiveness is not in doubt. Apart
from the work of people, processes are carried out by computer software
and whether these need to be validated should be considered; for
example, does the data processing software produce the expected
output? For proprietary software, validation can be expected to be the
responsibility of the supplier but where it is tailored for a specific
application (e.g. a spread sheet model) testing may be required.

Using job or project numbers is well established in market research and

this meets the requirement for identification and traceability (Subclause
7.5.3). Job numbers do need, however, to be used for all documents
relating to a particular project. Issues of customer property (Subclause
7.5.4) in market research generally relate to ensuring sensitive data
supplied by a client are kept secure although a product testing project
may require special steps to ensure client supplied research products (or
those otherwise sourced) are handled and stored appropriately. These
issues need to be covered in procedures or written instructions.
The final Subclause of Clause 7 is ‘Control of monitoring and measuring

equipment’. This refers specifically to ‘monitoring and measuring
equipment needed to provide evidence of conformity of product to
determined requirements’. If, for example, a component is required to be
of a specified dimension, any instrument used to check this (e.g. a
micrometer) should be accurate enough to detect whether or not the
component is within permitted tolerances. In most market research
processes such equipment is not used. Data processing software may
provide metrics to check such as the number of interviews within a quota
but this is a feature of processing rather than separate monitoring and
measuring. Apart from some very special forms of research, this
requirement (Subclause 7.6) will normally be an ‘exclusion’ and this can
be justified in the quality manual as not applicable to the processes
undertaken.
Monitoring and measurement

The final Clause of BS EN ISO 9001 concerns monitoring and measuring
the quality management system and the processes it covers. Coupled with
management review (see above) it is intended to be a means of quality
improvement. Some of the requirements will already be met to some
extent by organizations implementing the standard but others will be
less familiar.
The requirement to measure customer satisfaction (Subclause 8.2.1)

should not present difficulties to a market research company although
some, for various reasons, are shy of doing so. It is a matter of discretion
for the company to decide how satisfaction is measured; a sample
approach can be used rather than attempting to have feedback from
every client and every project. If a company has few clients a more
qualitative method can be used. Ideally, research companies should be
continually assessing their performance, allowing problems to be
addressed as soon as they occur rather than being identified when a
project has been completed when it may be too late. In dealing with
negative comments, any complaints should be logged, together with the
immediate action taken to address the problem. A quality assurance

approach should include the process of learning from mistakes, including

where these lead to client dissatisfaction.
A key element in managing the quality management system is internal

auditing (Subclause 8.2.2). This is carried out to determine whether the
system ‘conforms to…the requirements of this International
Standard…and is effectively implemented and maintained’. Once the
system is established and checked against requirements the main effort
of internal auditing concerns whether the system is working
(implemented and maintained) since unless either the standard is revised
(which it is from time to time) or the organization’s business and
processes radically change, the documented quality system, once in place,
should continue to cover BS EN ISO 9001 requirements.39 The process of
internal auditing is the same for BS ISO 20252 as for BS EN ISO 900140
and as this is discussed in Chapters 9 and 11 we will not describe it here.
However, we emphasize the importance of internal auditing. Even if it
was not a requirement of the standard, the process of checking whether
the quality management system was working or not would be necessary.
Without this, management would simply not know whether or not the
system was more than a piece of literature; the probability is that it
would be just a plan and not something that actually happens.
The processes of the organization and the output (products) are required
to be monitored and measured (Subclause 8.2.3/4). Monitoring of market
research processes, in practice, will be limited. Internal auditing will
establish that procedures for the research processes are followed and
should identify whether these fail to achieve the intended results.
Examples could include bottlenecks, which in the context of market
research could be getting completed questionnaires back from
interviewers, manual data entry, etc., that tend to affect the agreed
timetable of projects or lead to a company failing to be able to offer the
turnarounds clients require. Inspection of the finished outputs of
research, for example, reports, tables, etc., can check conformity to
standard layouts, table presentation, grammar and spelling, but not
whether the research findings are basically ‘true’ (although professional
judgement may do so to some extent). These types of checks are likely to
be covered in quality management system procedures, as are checks
earlier in the research process, from proposal drafting onwards.
More formal verification steps in data collection and data processing,

such as interviewer monitoring and back checking, are also clearly forms
of product monitoring and measurement. The findings of both process
and product monitoring need to be documented and recorded in a
39
An initial external assessment will identify if the system has failed to cover all requirements
of the standard.
40
The requirements for internal auditing are more detailed and explicit in BS EN ISO 9001
and possibly more formal documented procedures for this process are needed compared to
BS ISO 20252:2012. There is a BS standard for internal auditing (BS EN ISO 19011:2011
Guidelines for auditing management systems) which provides guidance on the process.

suitable form, otherwise checking, validation and verification will at best

only correct particular identified errors and problems and will not be
inputs to longer term improvement including reducing the recurrence of
similar problems.
Subclause 8.3 of BS EN ISO 9001 – Control of nonconforming product –

covers ensuring a product identified as faulty in some way is not
delivered to a customer until it is rectified or, in the case of large volume
production, faulty units are scrapped. In a different form, this
requirement can be applied in market research. Questionnaires with
identified errors (or the interviews carried out inappropriately) may be
corrected by follow-up interviews, the interview repeated or the
questionnaire excluded from the achieved sample. Errors found in reports
during a checking process will obviously be corrected prior to delivery to
the client and other identified errors will be corrected to optimize the
final ‘product’. Such steps should be covered in procedures.
A danger of internal auditing and monitoring of processes and output is

that it all ends up as busy work, carried out by such as the quality
manager with results nicely filed but with no real effect on the business.
Subclause 8.4 – 8.5 requires the data to be analysed and then used to
make real improvements and ensure this is a continuous process.
Improvements can include both ‘corrective action’ and ‘preventive action’.
Corrective action addresses identified ‘nonconformities’ – e.g.
questionnaires incorrectly completed – but as well as dealing with the
particular problem, seeks to analyse why it has occurred – ‘root causes’ –
and what can be done to reduce recurrence of the same type of problem.
Preventive action is more a matter of identifying current processes that
are likely to produce problems and seek better ways of carrying out the
process step. An example might be the timing of interviewer
back-checking; if the delay between the interview and the re-contact is
too long the respondent may have forgotten being contacted by the
interviewer or if problems are found it may be too late to take effective
remedial action. Management review, discussed above, is also part of the
improvement process. Senior management need to at least see that the
improvement process happens, allocate responsibility for any
investigation and should be actively involved in at least some initiatives.
Assessment and certification

BS EN ISO 9001 can be effectively implemented without outside
assessment and certification; effective quality management should lead to
internal efficiency improvements and competitive edge due to a better
meeting of customer requirements. In other words a company can be
self-certified to BS EN ISO 9001 and gain the benefits of effective quality
assurance. However, having gone to all the effort of implementing a
formal quality management system, most will wish to convincingly

demonstrate to customers and others that they meet BS EN ISO 9001

requirements. In other words the kudos of certification is a marketing
asset. Also, some customers may require suppliers to be certified to a
recognized standard and particularly if they themselves are
independently assessed. Planning for independent assessment and
certification41 is, therefore, very often implicit in ‘doing’ BS EN ISO 9001.
Apart from any benefits of having a certificate, external assessment is an
additional incentive to follow the quality management system; an
imminent visit by assessors – as was said of the prospect of hanging –
concentrates the mind.
In most countries and certainly in the UK, a company seeking assessment

and certification will find no shortage of organizations offering an
appropriate service – for a fee of course. The uptake of BS EN ISO 9001
has created or at least much expanded suppliers of this sort to a mini
industry. The question arises, however, of whether a certificate issued by
an assessment body is recognized as ‘good currency’; does the assessment
process provide reliable and trusted evidence of conformity to
BS EN ISO 9001? Unfortunately, the growth of organizations seeking
BS EN ISO 9001 (in some cases reluctantly to meet customer demand)
brought in some assessors offering quick and cheap certification based on
only limited effort to establish whether or not the organization has
effectively implemented the standard.
To ensure recognized and credible certification, accredited certification

bodies should be used where possible.42 These have been assessed by an
independent accreditation service; in the UK this is provided by the
United Kingdom Accreditation Service (UKAS) and most other countries
have a similar national equivalent. The accreditation body assesses the
certification body for specific standards (not necessarily just
BS EN ISO 9001) and for assessment of specific types of business (e.g. in
the UK some bodies are accredited for certification of market research
companies to BS EN ISO 9001 and BS ISO 20252). The accreditation
process is also carried out to its own international standards; in the case
of BS EN ISO 9001 certification, the current accreditation standard is
BS ISO/IEC 17021:2011, Conformity assessment – requirements for bodies
providing audit and certification of management systems.
The process of independent assessment and certification for

BS EN ISO 9001 and BS ISO 20252 are virtually identical and since the
latter is discussed in Chapters 9 and 11 we will not go into details here. It
is also worth pointing out that assessment is a continuous process and
41
This is also referred to as ’third party assessment’ with self-certification being ’first party
assessment’. ’Second party assessment’ is by the customer and may be used to ensure an
uncertified supplier follows at least the key relevant requirements of a standard – e.g.
checking a fieldwork subcontractor carries out validation to the required level. The term
’registration’ is also used more or less interchangeably with ’certification’.
42
A certification body that is not accredited may well carry-out assessment to a high
standard but how can we be confident of this?

not just a one-off test; at least annually, the external assessors require
evidence of continuing conformity and to identify areas for
improvement.
BS EN ISO 9001: In summary

Developing, implementing and being assessed to BS EN ISO 9001 takes
effort, time and money but also offers substantial benefits. Chapter 5
covered the argument for effective quality assurance and quality
management and these benefits are often sufficient to justify the process
of meeting a relevant standard. As mentioned above, formal certification
to a standard also gives assurance to clients and they may require it as a
condition for even tendering for the business. Up to the early 2000s,
BS EN ISO 9001 was the only available and recognized international
standard that a market research company could consider. Now, however,
more specific standards – BS ISO 20252 and BS ISO 26362 or the national
equivalents (discussed in Chapter 9) – are available and the recent trend
is for market research companies to consider assessment to these rather
than BS EN ISO 9001. It is, though, perfectly practical, without much
additional implementation effort and at little extra cost, to be assessed
and certified to BS EN ISO 9001 at the same time as to the more
industry-specific standards (also covered in Chapter 9)? One reason to do
this is that BS EN ISO 9001 has far wider recognition amongst industry in
general than other standards such as BS ISO 20252, which is likely to be
better known to specialists within client companies than to buying
departments or wider decision makers.

8 Process standards for interviewers
Standards for data collection by interview have been in place in many

countries for some years and pre-date the impact of more general quality
standards, such as BS EN ISO 9001 or those specifically covering all market
research processes. We discuss how the need for data collection standards
reflects the key importance of primary respondent data for the quality of
final output. How interviewing is carried out and how it is organized
raises major quality issues, especially those relating to variability, and
these too are considered. Bodies such as the UK Interviewer Quality
Control Scheme address these issues through standards and assessment of
participating member companies. The standards require minimum,
specified levels of interviewer training, appraisal and validation of work
and these are described. The need to define minimum levels in these
areas has been incorporated in wider national market research standards.
This and the next chapter are about formal quality standards for market
research and the assessment or inspections that enable companies to
demonstrate compliance to the standards. The next chapter deals with
the two international standards that are relevant for market, social and
opinion research: BS ISO 20252:2012, Market, opinion, social research –
vocabulary and service requirements (the main research standard) and
BS ISO 26362:2009, Access panels in market, opinion and social research –
vocabulary and service requirements, which is specifically for access
panels are a tool used for recruiting respondents and collecting research
data. Whereas these ISOs for research are relatively recent developments,
the research sector has a long history of tackling quality, and this chapter
is concerned with the much longer established process standards and
associations which were the precursors to the international standard for
market research – the Interviewer Quality Control Scheme (IQCS) and
other national standards. UK initiatives, particularly IQCS, acted as
catalysts for international change and so we will use the history of the
development of IQCS and national standards to set the context for the
more recent international developments that are discussed in the next
chapter.
Interviewing problems
The majority of market research is based on data obtained by some form
of primary data collection and whether or not this work is done well or
badly has a very direct bearing on quality, on the ‘rightness’ of the

results. No matter how sophisticated the research design, how technical

the analysis and how brilliant the reporting, it will all be worthless if
data collection is of an unacceptable standard: garbage in, garbage out.
With the technological developments in research there is an increasing
amount of data collection that is undertaken directly with the
respondent e.g. online survey research where the respondent completes
questionnaires online, self-completion via laptops, and passive data
collection where respondents’ views are collected from existing sources.
However, a sizeable chunk of research data collection still requires some
form of interviewer interface with respondents.
Profiling the interviewer

In theory, the interviewing task is one of the easier parts of the research
process: select respondents as per instructions, persuade them to
cooperate, and then administer the questionnaire as directed. The formal
skills to do this are generally considered to be limited, although certain
flair, it is acknowledged, is needed to gain cooperation (this tends to be
undervalued by research executives because they may not have done any
interviewing themselves). Yet, data collection and interviewing are
problematic and to a large extent this is due to structural reasons – how
interviewers work and how they are employed.
In field interviewing – where respondents are interviewed face to face on

the streets, in the home or at work – there is an inevitable limit to the
control that the project manager or research designer can exercise over
how an interviewer does her or his work. Interviewers work remotely
from research companies’ head offices, largely alone (direct supervision is
only realistic for a small number of interviews in any programme), and
their involvement in a project is confined to data collection. They do not
generally know much of the overall objectives of the research or the
wider methodology. Often, for good reason, they do not even know for
whom the research is being carried out. Because of this restricted
involvement, the skills of interviewers need be no more than is necessary
to do interviews – e.g. good communication skills – and they often do
not need to have fuller professional skills or for that matter professional
status and commitment. To use full professionals for data collection is
also almost always too expensive within a commercial-research context,
though there are exceptions, for example qualitative depth interviewing,
business-to-business research with senior leaders, and piloting.
Interview organization structure

The above aspects of field interviewing potentially create some problems,
but these are made more acute by the commercial structure in which
interviewers are employed and work. If we look back, field interviewing

was largely structured as loose networks. Interviewers were

‘self-employed’ – paying their own taxes, working for local supervisors
and having little direct contact with research company head office staff
unless a project involved interviewer briefing in person. The supervisors
were also largely self-employed and home-based and it was the
supervisor’s job to put together a team to meet the fieldwork required in
their own patch. Supervisors, like their interviewers, worked for a
number of research companies, sometimes for many companies, although
as in all businesses, some ‘customers’ would be bigger than others.
Supervisors also tended to know counterparts in other areas and a
start-up research company could quite quickly, and at virtually no cost, be
able to claim national field coverage; an interviewer even in the remotest
regions could always be located through a few network phone calls.
Unless personal briefing was required, the small agency need never meet
its interviewers face to face.
Obviously this structure was not conducive to the best quality fieldwork
since research companies had almost no direct control over interviewers
and could only hope that the work was done to an acceptable standard,
whatever that might be. Interviewers also had little reason to have any
feeling of loyalty to research companies since their ‘employer’ was a local
supervisor who tended to have more regard for their own team than the
‘clients’, the research company. In any dispute about quality of the
output the supervisor, as likely as not, was defensive about ‘their team’.
The tangible rewards offered to interviewers were low – unskilled or
semi-skilled rates just above minimum wage in some cases – but
supervisors (who tended to be drawn from the pool of experienced
interviewers) did a bit better. Interviewing was a part-time and spasmodic
job and was generally regarded as offering ‘pin money’ rather than a
living and, it was assumed, well suited returners-to-work (generally
women with families) and pensioners with a bit of time on their hands.
This ‘cottage industry’ structure existed for economic reasons. The

research sector was and still is relatively small (e.g. in the UK the
advertising sector by comparison is more than seven times larger than the
research sector43). Research companies themselves were almost entirely
small businesses, but needed to be able to offer a capability of carrying
out nationwide fieldwork. However, because job throughput in the
typical research company was low, it was impossible to usefully employ
local interviewing teams on any sort of continuous basis. Also, regardless
of where fieldwork was needed, the total field requirement fluctuated
with client commissions; some months there would be very little need for
interviewers and some months too much. From the interviewers’ point of
view, continuity of work necessitated working for several research
companies, since none could offer reasonable workloads over the year.
43
According to the AA/Warc online Expenditure Report, total UK advertising revenue was
£15.5bn in 2010 compared to the £2.06bn total research revenue reported in the 2010
annual survey produced by The Market Research Society.

Also, due to their relatively small size, many research companies lacked
much in the way of in-house administration and were not, therefore,
able to manage interviewers directly and were more than happy to turn
over the responsibility to a local supervisor. The low rates offered to
interviewers reflected supply and demand; with little or no training
required to become an interviewer, supervisors could always find
someone to take on work and the pay levels fed into the prices charged
by research companies to clients; research companies could not afford to
pay more than the going rate. It should also be remembered that
historically research did not have much competition from other sectors
offering more favourable work terms. Before the era of working time
regulations, and flexible and part-time working, research interviewing
was one of the few jobs where you could work your own hours, could
accept or reject an offer of work and where you needed little or no
qualifications to get into it.
The changing relationship between interviewers and

research companies
Is all this an exaggeration? Not much. Some market research is still
organized in ways that are not drastically different to this picture. Market
research still has a cottage industry aspect to it. However, there were
always larger research companies with full-time supervisors that could
provide much of the interviewing team ‘on the books’ with reasonable
continuity of work (particularly those companies which had longitudinal,
tracking studies which required constant data collection week in and
week out). Such companies ran more formalized interviewer training
schemes and had some form of quality control in place to monitor
interviewer performance. In this respect, smaller companies tended to live
off the efforts of the larger ones which had training programmes. If
interviewers had any training it was from larger companies. Over the last
few years, larger companies have tended to do a greater proportion of
all fieldwork and, therefore, more of it is done to the standards set by
such companies.
There have also been other developments which have favoured closer
contact and a degree of control between the research company and the
interviewer. One factor is that most interviewers are no longer generally
classed as self-employed. European legislation such as the Working Time
Directive has resulted in a reclassification of most interviewers. In the UK,
for example, most interviewers are classed as ‘workers’ with the
exception of qualitative recruiters (specialist interviewers) who still tend
to be self-employed. This is in recognition that employers have a
relationship with the interviewers, some continuity of contact and
therefore some obligations to them. Although most interviewers are not
legally classified as ‘employees’, except in cases where research companies
have actively decided to employ them (this is mainly found in telephone

centres and interviewers engaged in tracking studies) they are classified

as ‘workers’ and as such they retain many of the rights that employees
have (the most significant being the right to holiday pay). Research
companies are required to account for tax and National Insurance and
operate a proper payroll for their interviewers. With typically hundreds
of interviewers on the books, this has created an additional
administrative burden and some incentive to prune the field force so that
fewer interviewers are used, but each gets more work. Another factor is
technology. Since the 1990s larger research companies have been moving
from paper questionnaires to CAPI (computer assisted personal
interviewing), and now more are moving away from interviewer CAPI to
self-administered CAPI, and away from face-to-face interviewing
altogether towards online. As previously mentioned, this offers efficiency
and some quality benefits (although online has its drawbacks) but it also
favours the use of smaller and more intensively worked field forces; if
you have to pay and treat interviewers as workers then there is a greater
desire to retain them and maximize the investment in them.
There are also changes in the socio-demographic make-up of

interviewers. Instead of middle-aged, middle class women working for
interest after raising a family, field forces are now much more diverse:
women for whom it is an important source of income, more over 60s,
more men, and more people from ethnic backgrounds (which represents
not only the changing demographics of the workforce but also the
demand within research for interviewers with language skills and diverse
cultural backgrounds to undertake research within their communities).
Such a trend might be expected to encourage interviewers to seek a
more professional status and an increase in pay. However, it may also
lead to new quality problems; as interviewers seek larger earnings,
shortcuts in work may be tempting.
Controlling fieldwork quality

Given that the research sector has historically had a rather chaotic
fieldwork structure, what are the implications for the quality of output?
There is clearly the potential for problems and of two main sorts:
interviewer variability and interviewer cheating.
Structured fieldwork assumes zero interviewer variability; the responses

should be determined by the respondent and it should not matter who
administers the interview. In practice, some level of interviewer variability
is inevitable; minor variations in how respondents are selected, how they
react to the interviewer, how the interviewer articulates questions, word
stress and many other things (without the interviewer doing anything
which can be classed as bad practice) will lead to some level of variability
even though it may be possible to ignore the effect in practice. This
inevitable level of variability will be aggravated by any poor practice such

as not following respondent recruitment to the letter, quota squeezing

(moving a borderline respondent to one classification or another as the
need arises), prompting and leading respondents, not following
questionnaire wording and inferring responses. There can be no doubt
that such malpractice occurs; often through poor training, a desire to
complete the work quickly, or meet ‘impossible’ work demands. Where
the problems are rooted in a lack of technical training a remedy is
obvious. Training or other forms of contact between the agency and
interviewer may also improve motivation and so reduce conscious
malpractice. Various forms of supervision and validation may also
improve things.
The problems just discussed can be put down to incompetence rather

than wickedness. The borderline between bad practice and cheating is,
however, hazy. Fudging on quotas, for example, is certainly a form of
cheating. Real cheating is perhaps distinguished by some ambition; the
responses to whole parts of the questionnaire are skipped and invented
and later perhaps whole interviews are concocted or respondent
demographics grossly changed to match quota requirements. These
practices can be found in all types of field interviewing and there are
some specialities in qualitative recruitment and especially for group
discussions (e.g. fraudulent recruitment where respondents are not what
they claim to be).
At the extreme end of qualitative group recruitment malpractice,

interviewers and respondents collude, so that the latter misrepresent
themselves in terms of recruitment criteria; not so much crude
demographics (sex and broad age bands at least, are hard to contrive)
but in product usage criteria required for quota purposes to qualify as
group members. The incentive for the respondents to do this will be the
reward given for attendance (£30 tends to be the minimum) at
something which is hardly difficult work, requires about an hour and a
half of someone’s time and can often be a pleasant evening out with
plentiful refreshments supplied. It is not that hard for a primed
respondent to claim to be a user of a particular brand of foodstuff
without the group moderator initially realizing it (although a good
moderator can usually spot a cheat with good group interviewing
technique; it is actually fairly difficult to sustain a lie for an hour and a
half without making at least one mistake).
There is also the problem of ‘professional’ group goers. Regularly taking

part in such research makes the respondent atypical and it is usual
practice for the researcher to insist that respondents have not been to
another group in about six months. Most experienced qualitative
researchers will have eavesdropped on comments such as ‘what are we
talking about this week’ while waiting for respondents to be brought in.
The recruiter’s job, however, is made easier if they can draw on
respondent ‘panels’ even though respondents on these, by their nature,

are no longer typical of the population due to being more familiar with
research (we’ll discuss more about panels when we get to
BS ISO 26362:2009 in the next chapter).
The problem with cheating of any sort is that it is corrupting. Once done
and if not detected, it will be seen as easy and will almost certainly be
repeated. What started off as a one-off to solve a pressing problem
becomes standard practice. Research companies’ remedies against
cheating include dissuasion through morale building and training, close
contact with interviewers, supervision where possible and validation
methods to detect it.
The above puts all the blame on interviewers for both bad practice and
cheating. But this is hardly fair. Research companies can do much to help
their field-workers through better training and administration.
Furthermore, researchers must shoulder some of the blame. Poor
instructions, badly drafted questionnaires, quotas that are virtually
impossible to meet, and other design defects make the interviewer’s task
that much harder and are no encouragement to good work. Part of the
problem is that research executives often have little or no experience of
the fieldwork they are ordering. In an ideal world, all trainee research
executives should serve at least six months as interviewers.
Quality and telephone interviews

So far we have considered only fieldwork rather than telephone data
collection. Telephone research is almost wholly carried out at central
locations – in call centres, some with hundreds of work stations and
equipped with a networked phone system and nearly all now with CATI
(computer assisted telephone interviewing) terminals. The fact that
interviewers are together in one place means that in various ways it is
easier to control both variability and cheating. Gross cheating is actually
much less likely even without very active steps to prevent it. Assuming
some sort of supervision, the interviewer has to at least appear to be
interviewing and it is usually as easy to do the work as make it up.
Telephone systems also usually allow ‘real time’ monitoring – a supervisor
can listen in silently (unknown to the interviewer44) to the interview and
check how the questionnaire is administered, respondent qualification
etc. Monitoring can be closer still with CATI; the responses entered can be
seen (on the supervisor’s terminal) while the interview is overheard (and
with the increasing amount of fully recorded telephone interviews,
monitoring evidence can be supported with actual recordings).
44
In most countries legal requirements means that monitoring can only carried out if the
interviewer is aware this may occur. They do not need to know when exactly it would
occur (this would defeat the purpose) but that it may. Most research companies cover this
off legally by stating in their worker contracts, staff handbooks, etc. that monitoring
and/or recording of interviewers may take place.

Training can also be carried out more frequently and at lower cost than
for field interviewers and every project can be briefed adequately, with
problems solved as they arise. Morale is also often stronger because
interviewers work together and at the research company. Their pay and
conditions may be no better than interviewers out in the field (many
interviewers are still paid by the hour) but with only a modest
management effort they can develop good team loyalty and commitment
to the company. Unlike field interviewers, they often work for one
research company at a time and perhaps for long periods (although every
call centre will have a percentage of the staff that is temporary). Due to
its nature, telephone interviewer quality is, therefore, much easier to
control and effective measures can be taken by research companies at
modest cost (which is less true in the case of field interviewers). However,
complacency should be guarded against and the effects of telephone
interviewer variability cannot be assumed to be minor.
As well as normal field and telephone interviewing, there are some more
unusual forms of data collection which have specialized quality concerns.
These include the various sorts of observation techniques associated with
retail auditing and the increasingly used ‘mystery shopping’ approach.
With the latter, one school of thought believes that any more than
limited training actually reduces quality. It is the reactions of ‘naïve’
shoppers which are sought.
How the world sees interviewers

A final issue is the effect on quality of the public perception of
interviewers. Overall, their objective status has not improved and their
publicly perceived standing has almost certainly gone down. More and
more interviewing is done and this itself makes respondents
interviewer-wary. How many of us would cross the road to avoid
interviewers? Also, there is the problem of people who are not carrying
out real research posing as interviewers – timeshares, double glazing,
fundraisers and many other salespeople have been known to use a
research guise as an ‘opener’, for example: ‘May we ask you a few survey
questions? Do you own a property? How many windows does your
property have? Do you want some new windows?’
Legally, research is in a much stronger position now in that many

countries have introduced some form of legislation to avoid the use of
mis-selling tactics. Similarly data protection legislation requires data
collectors to be much more transparent about the purposes of their data
collection. However, the problem has certainly not been eradicated and
no legislation in the world will prevent a ‘cowboy’ trader from doing
what they want (the very definition of a cowboy!). More worrying is that
market research is sometimes claimed to be used as a cover for crime,
and some trading standards and police force leaflets have warned against

The role of IQCS
the dangers of fake market researchers along with door-to-door beggars,

gold buyers and dodgy tradesmen. This has resulted in some areas
establishing ‘No Cold Calling Zones’ to prevent cold callers in residential
areas (although the legal scope of such zones is to prevent door-step
selling, often the way they are policed means that interviewers can be
prevented from entering the zones). Certainly, response rates are falling
and this is very much a quality problem because of the inevitable increase
in non-response bias.
The role of IQCS

The IQCS was a UK initiative created to address most of the telephone
and field data collection problems discussed in the previous pages of this
chapter. IQCS was formed in 1986, but its origins can be traced back to a
previous organization; the Interviewer Card Scheme (ICS), set up in 1978.
This scheme was intended to deal with quality in general but its primary
concern was the standing of interviewers so that the public could
differentiate the genuine interviewer from the ‘suggers’ (those selling
under the guise of market research). The tangible form of this was an
industry-standard interviewer identity card which has since become well
established and virtually essential to an interviewer. The ICS, however, did
not progress in establishing agreed and validated standards for
interviewing and interviewers and by 1986 these had come to be seen as
a necessity for the sector. In that year, therefore, ICS was replaced by
IQCS, which had the primary aim of developing minimum standards and
setting up a system for establishing companies’ compliance with these
standards. Questions and problems of interviewer identity were left with
MRS as the professional body for research, and MRS continues to oversee
the interviewer identity card scheme and MRS Freephone service (which
enables respondents to confirm the bona fides of organizations
conducting research).
The IQCS is a corporate member organization and to join a research

company or user of bought-in fieldwork has to both agree to work to
the published minimum standards and be assessed or inspected against
these. Only companies which pass this process can become or stay as IQCS
members. Today there are around 40 companies that are members of the
IQCS. This is less than in the past primarily due to BS ISO 20252 taking
precedence. There are two types of members: Full Members, who enjoy
all the benefits of membership and have their fieldwork annually
assessed and accredited by IQCS auditors; and Member Only companies,
who are not assessed by the IQCS (as they are assessed to BS ISO 20252
but have elected to have a separate IQCS recognition). Most IQCS
assessed companies are either full service research companies or data
collection-only companies. Client buyers have also been encouraged to
lend support to the idea of minimum standards, not least by recognizing

that quality fieldwork is likely to cost more than inadequate

interviewing, but will be of much greater value.
The IQCS has two main functions: defining and revising minimum
standards (in the light of feedback from members), and administering an
assessment or inspection scheme to establish members’ and candidate
members’ compliance with the standards.
IQCS minimum standards

IQCS minimum standards are a form of service standard; they define how
certain sorts of activities should be carried out and, therefore, have an
analogy with product standards such as British Standards. They are
narrow and focused in scope in that they only cover one part (if a vital
part) of the research process: data collection. In practice the scope is even
narrower since, despite the full title, the standards are entirely concerned
with how interviewers will be managed rather than other aspects of data
collection. There are three key parts to the standard – training, appraisal
and validation – and these are linked to a requirement for record
keeping and documentation so that compliance with the standard can be
established. The essentials covered by the key parts of the standard will
be discussed, although the standard itself should be read for full details.
A point we would stress is that whatever criticisms may be made of the
standard or of IQCS as a body (since its inception there has been some
opposition to it within the sector), it is hard to argue that interviewers
should not be adequately trained, their performance appraised and their
work validated in some way or other. Given the structural problems
discussed, a denial of this is a recipe for poor quality.
The training requirements are relevant to all interviewers (field and

telephone) and no interviewer should do any client work without some
training organized by the particular research company. The level of
training depends on the work to be done by the interviewer and
whether or not they have any previous experience of market research
interviewing with another IQCS or BS ISO 20252 company (experience
with non-IQCS/ISO companies is ignored). Even where an interviewer has
long and relevant experience with other companies, a minimum level of
training must be provided; at least half a day (three hours) covering the
procedures of the research company and to confirm that the individual
does have the knowledge claimed. Where shorter training is given to
experienced interviewers, the research company must confirm the past
record of the interviewer by contacting the other research companies for
which they have worked. The broad content of the training is specified,
as is the method of delivery, for recruiters, telephone and face-to-face
interviewers at least half of the training should be conducted face to face
although it can take place anywhere and be given by a local supervisor as
well as head office staff. On completion of training, the new interviewer,

whether with prior experience or not, must be accompanied (or

monitored in the case of telephone interviewers) by a supervisor when
the first assignment is carried out and no work should be used until this
is done.
The general principles of the training programme apply to both

telephone and field interviewers. However, training telephone
interviewers working in one location is much easier to organize and far
cheaper than for field interviewers. Telephone interviewers have to come
into the offices to do their work and the trainer is likely to be a full-time
member of the company’s staff. Monitoring during the first assignment is
also easily carried out. The training of field interviewers is considerably
more expensive; interviewers are generally paid during their training
time and the supervisor giving the training certainly expects to be paid.
Even if a number of recruits can be trained together (and, therefore,
spread the supervisor’s cost) initial accompaniment must be one to one.
An effect of these requirements is to favour interviewer teams which are
used regularly; the cost of training new interviewers for one-off jobs, or
who may seldom be needed again, becomes relatively very high, but the
alternative of not training at all is hardly a basis for quality. IQCS field
training, therefore, has some in-built economies of scale and relatively
favours research companies with a larger work throughput but that keep
their field force as small as possible while coping with the workloads (not
an easy balance). There are also economies in head office administration
but these are not as great; such costs tend to go up in steps. The small
research company wishing to offer nationwide coverage is at some cost
disadvantage, although IQCS members do include smaller as well as
larger companies.
Expensive or not, the IQCS-prescribed training levels are hardly excessive.

A complete beginner is scarcely adequately prepared for a career as an
interviewer on the basis of two days training plus accompaniment – the
minimum specified for those without proven prior experience. The half
day for experienced interviewers may be more debatable (company
procedures can be communicated in writing) although the element of
testing skills needs to be covered in some way.
Interviewer appraisal
The second part of IQCS standards – appraisal – can be regarded as a
continuation of initial training. The latter may be deemed adequate for
the interviewer to start working but what is to ensure that he or she
does not forget what was taught or slip bad habits? Field work, at least,
is carried out without any direct and on-going supervision. Frequency and
methods of appraisal are specified in IQCS standards. As a minimum,
every field interviewer used on five or more projects/assignments a year
must be appraised annually and this must include an element of

accompaniment during interviews. Field interviewers used on ten projects

or more in a calendar year must be appraised at least twice a year
although one of these appraisals need not include accompaniment
during actual work or any face-to-face contact. Like training, appraisal
has economies of scale with the appraisal costs per working day or
interview declining with the amount of work carried out by the field
interviewer. Appraisal is also required for telephone unit interviewers,
although in practice this presents fewer problems or additional costs.
As with training, it is hard to argue that the appraisal requirements are

any more than necessary to give some confidence that the interviewer is
up to the job and that they help to reduce interviewer variability. It is
not, however, any safeguard against outright cheating. Interviewers will
hardly misbehave when a supervisor is with them. Safeguards against
malpractice, therefore, largely depend on the third part of IQCS
standards: validation.
Interviewer validation
Validation is principally about checking interviewers’ work directly with
the respondents concerned. In telephone units this can be done by
monitoring – listening-in without either interviewer or respondent
knowing that this has been done. Such validation would certainly catch
cheating but, as argued, this is unlikely to be a major problem in a
telephone room. Telephone monitoring is, however, an effective method
of establishing levels of interviewer variability and reducing it through
follow-on training and by feedback to the interviewers.
Telephone interviewing validation is, therefore, done in real time. By

contrast field validation is nearly always historical (except pre-group and
on-site group validation). Sometime after the questionnaires or other
data are returned, the respondent is re-contacted (by post, telephone,
electronic media or face to face) and a ‘back-checking’ interview carried
out to establish recall of the initial interview and to check some of the
responses (e.g. key demographics or product usage which will be relevant
to quota selection or respondent qualification). Of course, re-contacted
respondents may have forgotten the interview and, therefore, give a
seemingly negative result, but a consistent pattern of such results will
certainly indicate problems where further action is required. The standard
specifies the proportion of interviews completed in a project which must
be validated (ten per cent generally for field-based interviewing
compared with five per cent for telephone interviews), the methods to
be used, as well as content coverage, and also requires that each
interviewer is systematically selected for validation of their work.
The prescribed validation of field interviewing provides some safeguards

against cheating; the interviewer who consistently makes up
questionnaires is likely to be caught out when re-contact is attempted

with imaginary respondents. Similarly, back-checking may identify some

less extreme malpractice such as gross quota fiddling. It is hardly,
however, a method of establishing or minimizing interviewer variability;
post-interview re-contact is just not sensitive enough to achieve this.
However, the prescribed levels and selection of interviews for
back-checking appear implicitly to assume that validation is a form of
process control rather than the detection of fraud.
Another approach to validation is the checking of completed

questionnaires. This can identify problems which need correcting through
retraining (e.g. adequacy of recording open-ended responses,
completeness of questionnaires, skip patterns, etc.) and may even give an
indication of fraudulent interviews (inconsistency in the made-up
responses). Such ‘editing’ can involve inspection of completed
questionnaires (all or a sample) or by running various validation tests of
data. CAPI or CATI interviews arguably need less validation of this sort
since with automatic routeing, etc., there is less chance of making errors
(and automatic recording of times of interview makes cheating harder or
less attractive). However, in this case as well, some post data entry
validation can be usefully carried out. IQCS standards, however, only
briefly cover the editing of paper questionnaires although they have
changed this requirement from merely good practice to mandatory.
Validation of fieldwork through feedback from data analysis is not
covered at all; this is outside the ‘field’ domain.
The above summarizes the requirements of the three key parts of the
standard, all of which are also requirements of the data collection
element of BS ISO 20252. In addition there are important requirements
for adequate record keeping and documentation. These enable the
research company to control fieldwork quality management and in
addition are the means by which IQCS, through its team of inspectors,
establishes member or candidate member compliance with the standards.
Which records must be kept are specified but the general principles are
that project records have to show which interviewers worked on the job,
what they did and what validation was carried out. In addition
interviewer files must be maintained to show the training and appraisal
records of interviewers, the work they have been assigned, the outcome
of validation of their work and various administrative details. The
standards also specify documentation that must be given to interviewers
including training manuals and an annually issued MRS identity card.
What is not specified in IQCS is any need for formal quality assurance;
systems to assist research companies in ensuring consistent compliance
with the standard.
IQCS standards also cover other areas which we have chosen not to
discuss at all. The most important of these are the requirements
concerning field based supervisors. It is they who largely recruit, train and
appraise interviewers.

Assessment and inspection

From inception, the IQCS has required its members to be inspected to
establish that they do comply with the standards. Membership of the
IQCS is, therefore, more than mere good intentions. It is a commitment
to what is, in effect, third-party assessment.
Inspections are carried out by a small team employed, part time, by the
IQCS. The individuals concerned all have practical and long-term
experience of market research fieldwork. Inspections are carried out
when a company applies to join the scheme and annually thereafter. The
inspection work takes place at the research companies’ office with the
time spent linked to the size of the company (effectively a minimum of
one day) and very largely consists of checking records with cross
referencing of project and interviewer files. This establishes whether
training, appraisals and validation have been carried out to the
standards, although, strictly speaking, the inspection only proves that
records exist indicating that the requirements have been met. There is,
therefore, some potential for the companies to cheat through false
recording but this probably happens to only a minor extent. It is hard to
imagine that a company would go to all the trouble involved without
having a basic willingness to comply. The IQCS has been criticized because
its inspection is largely concerned with historical records rather than
directly establishing fieldwork competence. Realistically, however, it is
hard to envisage how any inspection or assessment, at a realistic cost, can
be anything other than largely record based. It should also be said that
inspections are no walk-over. Every year, some candidate companies fail,
as do, occasionally, existing members.
Inspection of course has a cost to the companies. The inspection fees,

however, are the minor part of the total costs of complying with the
standard. Training, appraisal, validation and the maintenance of records
all have internal costs (which for smaller companies can be comparatively
significant) and, as indicated, the basis of these favour larger rather than
smaller companies. Arguably, these should be seen as the price that must
be paid for maintaining levels of data collection quality consistent with
meeting clients’ needs rather than as just IQCS costs.
Is it all enough?
One point to make immediately in considering whether IQCS standards
are adequate is that they must be acceptable, taking into account the
costs entailed, to the companies required to implement them. It is easy to
think up more and more quality hurdles, but unless members are willing
to follow them they will vote with their feet, and leave the scheme.
Similarly there is no point in being critical because IQCS standards do not
address the full scope of data collection – there are other standards for

Standards for the whole research process
this (discussed in the next chapter). This is not the intention; only the
work of interviewers is addressed and not the important contribution of
design work (a bad questionnaire will fail to produce valid data
regardless of the interviewers) or the project management of fieldwork.
In relation to ensuring that interviewers are trained and for on-going

appraisal, the requirements of the IQCS, are, in our opinion, the
minimum that can be regarded as acceptable. In that the IQCS also
mirrors the data collection requirements of BS ISO 20252 means that the
standard works with the broader quality process standards framework. As
with all standards they set an acceptable minimum standard, those
companies wanting to achieve something more – real quality – should
aim higher. In other businesses, if the minimum quality level cannot be
met or it is not offered at all, the operators fade out due to discernment
in procurement. The principle seems fair enough in market research as
well, although the reality is that not enough clients demand standards as
a necessary minimum (whether these be the IQCS, ISO, professional
memberships and accreditations or code adherence).
Standards for the whole research process

Following the development of IQCS, similar interviewing standards were
developed by other countries (e.g. Australia). Inevitably this led to
demands for standards for the whole research process, not just the data
collection element (although this element remains the area with the
highest risks). Corresponding standards were developed; Australia,
France, Germany, the Netherlands, Spain, Italy and the UK all developed
their own national standards for research, sometimes with separate
associations forming initially to complete the drafting of the standard
and thereafter to oversee its assessments (associations such as the UK’s
Market Research Quality Standards Association (MRQSA) and Market
Research Quality Assurance (MRQA) in Australia) or under the auspices of
the national standards bodies.
After a rather bumpy start – with some large research companies publicly
castigating the standards as having no value – the national standards
gained traction in the marketplace and following their launch became an
accepted feature within the research sector in the same way that the
IQCS had. Following the success of the national standards for the whole
research process, and the increasing globalization of the research sector,
there was some demand to develop one international standard for
research. The national standards – particularly the UK’s BS 7911:2003,
Specification for organizations conducting market research, which had
influenced other national standards such as Spain’s and Australia’s –
became the foundation for the first international standard for market
research – BS ISO 20252 – originally issued in 2006 and subsequently
updated in 2012. This in turn encouraged the development of a further

international standard, BS ISO 26362:2009, Access panels in market,

opinion and social research – vocabulary and service requirements, which
was developed specifically for one of the growth areas in market
research – access panels. Both of these standards will be explored in
greater depth in the next chapter.

9 International quality standards

for research
After the development of national market research standards, the

increasing international scope of market research led to the drafting of
international standards through the International Organization for
Standardization (ISO). Two standards have been published with one –
BS ISO 20252 – now in its second, updated version. Both are process and
service standards covering: meeting requirements (especially the client’s),
planning and controlling the execution of a research design and the
service delivered to the client. The main headings of the standards are
discussed and summarized. Most organizations implementing the
standards seek to demonstrate compliance through assessment and
certification and this process is described. The practical implementation of
the standards prior to assessment is also outlined and expanded in
Chapter 11.
Following the success of the interviewer quality standards and the

creation of national research standards for the whole research process
(discussed in Chapter 8) the next step for the research sector was the
creation of an international research standard. This was an obvious step,
not only in terms of standards development but also due to the changing
nature of research. The primary objective of international standardization
is ‘to facilitate business development and growth particularly between
different national and regional markets’.45 Global consolidation and
acquisition within the research industry, increasing demand for
multi-country international research projects and the need to ensure
consistency in standards across borders meant that there was a clear
business case for developing an international research standard. The
sector, never one to keep things simple, has in fact developed two
international standards, and in this chapter these standards will be
discussed in turn.
45
From the introduction to BS ISO 20252:2012, Market, opinion and social research –
vocabulary and service requirements.

9 International quality standards for research
The BS ISO 20252 standard

The first standard to be developed was BS ISO 20252:2006, Market,
opinion and social research – vocabulary and service requirements, which
was subsequently updated in 2012 with the issuing of the revised
standard BS ISO 20252:2012.
As with the national standards which preceded it, BS ISO 20252 was
developed to cover the whole of the research process and draws heavily
from various national standards that had already been developed,
including the UK standard BS 7911:2003, Specification for organizations
conducting market research. Following the creation of the international
standard, most countries withdrew their national equivalent in
preference for the ISO. However, there are still many national standards
bodies which have yet to implement BS ISO 20252 and the reasons for
this and the impact this has will be discussed later in the chapter.
What does BS ISO 20252 not cover?

Before describing what BS ISO 20252 is, it is useful to clarify what it is
not. There is often a misunderstanding of the concept of quality
standards that results in undeserved criticism of the effectiveness of ISOs,
and this has certainly been the case in some quarters of the research
sector.
By and large what is not covered is any attempt to define what is right
and/or valid research design. When and where qualitative rather than
quantitative research should be used, sampling and questionnaire design
principles, on what basis and how data can be generalized and how
inferences are drawn, are all examples of what is not covered in this
standard. These all touch on issues of quality as excellence rather than
relative quality. It is not that these areas are considered unimportant to
quality in research. On the contrary, these are fundamental matters which
would be inappropriate to be dealt with within ISO type standards and
assessment frameworks. Instead issues of what constitutes ‘right’ research
are professional issues to be defined by an informal consensus, as with all
matters relating to a profession. At the leading edge of research practice
there may be debate and dissension on what is ‘right’ but nevertheless,
there is a large body of research practice which is reasonably certain,
accepted and understood by all practitioners of any experience.
The reason that ISO could not sensibly define right methods is because of
the difficulty in reaching a consensus on what is right. This is primarily
because of the inherent diversity and complexity of the business
problems that research seeks to address. Every business problem requires
a bespoke research solution, some of which may be relatively
straightforward but others much more complex. It is these ‘softer
elements’ of quality research design and problem definition (Smith, D

and Dexter, A 1994) that means that research ISOs and other research
quality standards such as IQCS will not be suitable to resolve these types
of research issues, whilst still being fundamental to ensuring robust
research processes. This in turn reflects the shifting intellectual ground on
which market research is built and that market research is still a very
open and evolving profession. Any attempt to define ‘right’ research, in
simple statements, would inevitably have been criticized by people who
could reasonably claim authority within the sector. Whatever else, ISO
research standards development requires wide consensus which would
have been impossible to achieve for professional service standards of this
nature.
There are, therefore, good practical reasons for the course taken but it
should be noted that in other knowledge businesses, ‘right’ methods are
defined and set as standards. Accountancy is a good example. There can
be legitimate debate on how goodwill should be treated in accounting
statements, but this has not prevented standards from being set.
However, accountancy is not market research. Probably researchers would
contend that although some aspects of accountancy might be
sophisticated and require extensive knowledge, because it deals with
normative knowledge, it is all more certain than what is appropriate in
researching markets and consumers. No doubt accountants would not
wholly concur with this view. However, the fact remains that
standardizing ‘right practice’ in market research has not proved to be
practical as yet and is unlikely to be so in the foreseeable future.
Ultimately excellence in research comes down to a combination of
suitable quality standards (such as ISOs), appropriate professional ethical
codes and individual knowledge and competence. Only when all these
aspects are applied properly can claims of research excellence be made.
To blame ISOs for poor research is a bit like criticizing a footballer for
being bad at cricket; it just misses the point and purpose of ISOs.
BS ISO 20252: an overview

What BS ISO 20252 does seek to encompass can be listed under three
broad headings:
• Meeting requirements,
• The execution of a research design,
• The services supplied to a client over and above ‘right’ research
practice.
BS ISO 20252 is concerned with establishing the client’s requirements

from research and practical steps to ensure these are met in practice.
While the standard does not seek to address what is ‘right’ research
design, it does cover how research design is put into practice, with

various controls and checks specified to increase the likelihood that what
is intended in the design happens in practice (and at the first attempt).
Client services cover things such as keeping clients informed of progress

and of any problems arising, timeliness, involving them where
appropriate in decisions (e.g. questionnaire approval), how the research
should be reported – and the back-up that clients should expect, e.g. in
terms of record retention. Arguably these, and for that matter the
execution of the research design, can all be considered aspects of
meeting requirements.
This, then, is the general nature of the BS ISO 20252 standard. We shall
now consider the contents of the specific sections of the published
standard in the order in which they are set out in the document.
Scope, terms and definitions

BS ISO 20252 establishes the terms and definitions in addition to the
service requirements for conducting market, opinion and social research.
These definitions are vital as they set the parameters of what is covered
by the research ISO. Other front-end parts of the standard are formal
statements of perhaps limited interest to those actually using it.
Research process management system requirements
In the 2006 version of the standard this Clause was titled ‘quality
assurance’ and the change is of wording only and has been made solely
to comply with ISO drafting rules;46 the section is still concerned with
quality assurance or management and quality systems. Since the original
quality research standards such as BS 7911:2003 and IQCS were born out
of the perceived limitations for research of the early versions of the
broader quality assurance standards, such as the BS 5750 and
BS EN ISO 9000 series, it may seem odd that the first substantial section
of the ISO is in effect devoted to quality assurance. However, the need
for companies to ensure that they had effective quality assurance had
been accepted from the start. In this respect it was intended that
BS ISO 20252 should overcome a weakness of standards such as IQCS;
which do not contain requirements for internal quality assurance
methods.
The ISO quality assurance/process management requirements are similar

to those of BS EN ISO 9001, covering the main areas of quality
management system requirements, management responsibilities,
46
ISO policy is to limit standards for quality management to broad, all-business standards
such as BS EN ISO 9001 and to not allow industry-specific quality assurance standards, as in
the original BS ISO 20252 produced in 2006. As a compromise with the drafting committee
it was agreed that the section could remain if ’quality system’ was renamed as ’research
process management system’ – old wine in a new bottle.

documentation requirements, etc., but with rather less specified,

mandatory detail, allowing for more flexibility which is more suitable for
many of the smaller research companies insufficiently resourced to
necessarily be able to undertake a series of different standards. However,
despite differences in wording, when it comes to implementation,
BS ISO 20252 has the majority of what is needed to meet BS EN ISO 9001.
Once a company has developed a system to meet BS ISO 20252, the extra
step up to BS EN ISO 9001 is not so very great and more concerns the
documentation of the quality system than any changes which impact on
day to day operations, processes and the work of staff.
BS ISO 20252 requires a documented quality policy (Subclause 3.1.1) and

this is also a requirement of BS EN ISO 9001 but in addition the latter
requires a ‘quality manual’ (not the same as procedures but may cross
reference them) and ‘measurable quality objectives’. These two
documentation requirements were discussed in Chapter 7. In addition,
BS EN ISO 9001 is rather more explicit on requirements for ‘management
review’ and ‘internal audits’ and procedures meeting BS ISO 20252 may
need expanding a little in these two areas. BS EN ISO 9001 also has a
requirement for ‘preventive action’ (again see Chapter 7) – anticipating
when processes may lead to problems – and this should be explicitly
covered in procedures meeting BS EN ISO 9001. The BS EN ISO 9001
requirement for ‘monitoring processes’ is in effect covered in various
ways in BS ISO 20252 (not just Clause 3) and nothing extra is likely to be
needed. The longest section of BS EN ISO 9001 requirements, ‘product
realization’, is also met through meeting BS ISO 20252 requirements as
set out in all sections including those relating to the design process.
In practical application, the quality assurance requirements of

BS ISO 20252, or for that matter BS EN ISO 9001, concern having effective
procedures in place which define what needs to be done as a minimum
to offer clients a quality service. The requirements set out in the other
parts of BS ISO 20252 also need to be brought into the quality system.
What of those companies which already have BS EN ISO 9001 or wish to

have it alongside the research ISO standard; perhaps because they believe
certification to an established, general and international standard is well
worth having? While some of the requirements of BS EN ISO 9001 are
not covered by BS ISO 20252, it is a different matter the other way
round; nearly everything in the BS ISO 20252 quality assurance model –
i.e. Clause 3 – is covered by BS EN ISO 9001 (and in most areas in greater
detail).47 It therefore follows that a company assessed to meeting the
BS EN ISO 9001 standard also meets virtually all the quality assurance
element of the BS ISO 20252 and this has been taken into account in the
47
BS ISO 20252:2012 has a clause relating to confidentiality of research (3.2). This is not an
explicit requirement of BS EN ISO 9001 though it is implicit in various ways. Also
BS ISO 20252:2012 requirements for subcontracting are more specific to research activities,
but arguably no more demanding than in BS EN ISO 9001.

assessment arrangements. A company with an BS EN ISO 9001 system

already in place is also at a practical advantage when it comes to
incorporating the other requirements of BS ISO 20252 into the
documented system; the framework to do this will already be in place
and staff are used to working to a documented quality system.
To avoid possible confusion, we stress that quality assurance models in

general concern the application of quality standards and do not define
what these are and in this respect BS ISO 20252 Clause 3 is no different
to BS EN ISO 9001. It is in the rest of the standard where the quality
standards for research are defined. It is the purpose of the methods set
out in the quality assurance part to ensure consistent and constant
adherence to these standards. To illustrate the range of the quality
assurance requirements in BS ISO 20252 we have included the Clause
subheadings:
Organization and responsibilities

• The research process management system
• Senior management responsibilities
• Appointment of a quality manager
Confidentiality of research
Documentation requirements
• General
• Control of documents (other than project-related documents)
• Control of project documents
• Control of records
Competence and training
Subcontracting/outsourcing
Reviewing the effectiveness of the research process management
system
• Management review
• Problems and complaints management
• Internal audits
We do not intend to go into great depth on each of these topics since it

is better to read the standard itself. However, before we leave the topic
of quality assurance there is merit in discussing research subcontracting
and outsourcing.
There has always been a fair amount of subcontracting within research,

particularly with operational activities such as interviewing, data
inputting, coding, transcribing, etc. Traditionally this used to be to other
specialist companies usually based in the same country as the research
supplier. However, outsourcing in research is now conducted on a much
broader, global scale. Many of the larger research companies will
frequently utilize multi-site, multi-country operations for single research
projects e.g. telephone data collected in India, online data managed in

Eastern Europe, data analysis in the UK, etc. The reasons for this are
primarily financial, in that certain countries can provide certain elements
of the research process significantly cheaper than domestic operators
(and often these services will be provided by companies that are part of
the same group as the main research service provider). However, this cost
saving approach does not come without its risks.
Within BS ISO 20252 the responsibility for subcontractors’ remains with

the research service provider, and indeed this would be the case with
BS EN ISO 9001; however, BS ISO 20252 has yet to achieve the same level
of global implementation as BS EN ISO 9001 (reasons for this will be
discussed later). The result is that there are fewer subcontractor research
suppliers with BS ISO 20252, and as such research organizations need to
be much more vigilant in ensuring that subcontractors from countries
without a national BS ISO 20252 assessment regime are meeting the
requirements of the standard. In the absence of a third party assessment
and certification to the standard, suppliers would need either written
contractual requirements, and/or other documentary evidence of
appropriate checking to the standard, and/or checking, validations and
verification by the research provider. This requirement (Subclause 3.5)
means that research service providers need to have a significant amount
of trust in their global suppliers.
Managing the executive elements of research

Standards that focused on how the executive part of research should be
carried out when initially introduced was a new departure, and had the
potential to be controversial. The fact that there has been little negative
reaction reflects that during the original development of the standard
the broadest possible consensus was sought and that what is included is,
for the most part, already well-established good practice in many
research companies – in general, the standards were meant to reflect
existing good practice.
The scope of this module of the standards covers the tasks typically
carried out by a research ‘executive’, including responding to research
requests, contents of research proposals, research design and proposal
writing, questionnaire development, briefing of interviewers and
reporting of research results, as well as overall project management and
liaison with the client. However, the ‘executive’ can go by any title and
the standards allow for more than one person to have executive
responsibility for a project. There is, therefore, sufficient flexibility to fit
any organizational structure for both ad hoc and continuous research. At
some critical stages in the process – e.g. the sampling procedure,
questionnaire or discussion guide development, data collection, analysis,
etc. – ‘monitoring’ is required; this could include checking the executive’s
work to ensure that such activities are conducted according to agreed

specifications. However, depending on the size and structure of a

research company, it may be practically difficult to have another person
review the work of the first executive – quite possibly no one else will be
sufficiently knowledgeable – and the standards leave considerable
flexibility in this respect. The requirement within the standard is that a
research provider should have defined methods for checking all necessary
resources and appropriate internal and external expertise before
responding to a research request. The client’s acceptance of a proposal,
for example, could be a form of review. While this may be a practical
solution to an organizational problem, it is arguably less of a quality
solution.
The executive module is prescriptive about what has to be done at

various points in the process. Some of the contents of quotations and
proposals are defined (proposals are assumed to have a research design
element while quotations are responses to a tight specification). Minimal
levels of client contact and liaison, including through client briefings, are
specified as is the need for ensuring the adequate briefing of
interviewers and moderators. At the reporting stage, results relevant to
the aim of the research must be available in full. As a minimum research
reports must include sample sizes, period of data collection and method
of data collection. However, research companies must document the full
methodology in sufficient detail to allow any project to be replicated at a
later date. Clause 7 of BS ISO 20252 specifies what this would include for
quantitative and qualitative research projects. These might appear quite
onerous and to go to a level of detail which many clients may not wish
to know. Indeed this information only needs to be available to clients; it
is up to clients whether they avail themselves of this information.
Arguably, professional work should be transparent in this way and in any
case the standards require no more than is specified in many of the
relevant ethical reporting rules detailed in the various research codes of
conduct. The difference is that, unlike observance of a code, adherence
to ISO standards is validated via assessment. The executive standards also
cover the storage and safeguarding of project records (again generally no
more than set out in the sector codes which themselves are increasingly
based on legal data protection requirements) and looking after materials
supplied by clients such as test products, visuals for prompts and
information of all sorts. Finally, the executive module specifies checking
at various points in the process, to ensure the accuracy of all reported
data.
The requirements of the executive standard fit into the quality assurance
requirements of Clause 3 of BS ISO 20252 and those of BS EN ISO 9001. In
terms of the latter, the standards specified relate mainly to product
realization, customer-related processes, design and development and
production and service provision.

As mentioned at the beginning of this section, there is little that is

controversial in the executive standards because for the most part no
more than the codification of common practice is attempted. Therefore,
the requirements are arguably quite low. However, they do provide a
standard against which conformance can be established and, therefore,
potentially a means of differentiating those companies which publicly
adhere to minimum acceptable standards from those which do not. Any
research company is of course free to exceed the standards and over time
they may become more extensive and tougher. The minimum can always
be raised if the consensus to do so exists.
Data collection
The data collection section is the most extensive in BS ISO 20252 but this
is understandable as arguably the research data collection stage is the
most prone to error. The already well-established UK interviewer
standards from BS 7911:2003 (which were originally based on the IQCS
standards) were the start-point for the data collection requirements
detailed in BS ISO 20252. Following the completion of the first edition of
BS ISO 20252, in 2006, the IQCS amended its standards so the two would
remain compatible. The principal data collection requirements of
BS ISO 20252 covering interviewer activities (training, conducting
fieldwork, validation methods, back-checking etc.) are already detailed in
Chapter 8. The additional elements of Clause 5 are mainly concerned
with self-completion data collection, observational data collection and
data collection from secondary sources and the revised 2012 version
includes these new additional areas compared with BS ISO 20252:2006.
Data management and processing

Data management and processing practices vary quite widely throughout
market research and it was this element that initially proved to be more
controversial than the executive elements. Until the emergence of
national research standards such as BS 7911, data processing operations
had only limited exposure to formal service standards. Data processing is
clearly a vital link in achieving quality and the work involved includes
both routine administrative operations (record editing, coding of
open-ended questions and data entry) and more skilled and professional
level work (specifications for data analysis and the associated statistical
analysis of data). At each stage there is potential for error through either
simple mistakes or for more fundamental reasons. Coding, for example,
although usually regarded as a straightforward administrative task, is
quite problematical and may introduce more errors than sampling or
even data collection. At the worst, coders misunderstand interviewers’
recording of responses and interviewers themselves may well have

mis-recorded or misunderstood what the respondent actually said (and

this may not reflect what is truly believed).
BS ISO 20252 also defines requirements for data management, including

how parts of the process should be carried out and requiring checking or
verification at various points. There are also some ‘housekeeping’
requirements concerning the back-up, retention and security of data
records and ensuring all data are stored and retained in accordance with
any code requirements (some codes specify retention requirements),
applicable law requirements (such as data protection or freedom of
information) plus any contractual agreement with clients.
Standards for carrying out some of the main steps in data processing are
prescribed and these include developing a code frame for open-ended
responses, coding training, data editing, data analysis, and the
presentation of output including composition of tables and electronic
data delivery.
Inspection, approval or verification is specified at various points, including

the editing of records, preparation of code frames, coding itself, data
entry, data file editing and the final output. Second person verification is
specified at most of these stages and, in coding and data entry, minimum
levels of verification are required. This in particular has proved to be
controversial. A not wholly resolved aspect of verification is whether the
purpose is to identify any errors which arise out of the nature of the job
(e.g. data entry mistakes because of questionnaire layout) or because of
deficiencies in people (e.g. a data entry worker is simply inaccurate).
The dominance of computer assisted interviewing (CATI, CAPI or

automatic data entry through, for example, optical readers) clearly has a
fundamental impact on data processing. Data entry as such, at the
processing stage, for example, is obviated with CATI and CAPI. However,
the use of electronic equipment in the interviewing process only transfers
data entry to an earlier point in the process, where the interviewer acts
as the data entry clerk but any problems arising from this are not
separately recognized within the ISO. Conversely the growth of
interviewer assisted self-completion (where the respondent enters their
own responses but an interviewer is present), and online self-completion
research means that increasingly respondents will be inputting their own
responses directly for analysis. However, the robustness of this
information will be dependent on the robustness of the data collection
design and whether it allows for respondents to accurately express their
views. This has always been the case but one of the benefits of having a
good interviewer is that in the past their involvement could mitigate bad
research design by accurately recording views the respondent wanted to
express.

Assessment
Assessment and certification to ISO standards may be carried out by
independent bodies (‘third party’ assessment).48 In some countries, such
as the UK, the assessment companies follow a national assessment and
certification specification and are also separately accredited (in the UK by
the United Kingdom Accreditation Service) – see also Chapter 7 for the
need for such accreditation to ensure that they are suitable qualified to
carry out assessment for the specific ISO standard.
During initial assessments all parts of a research company and all relevant
parts of the standard are covered. The work involved includes both
checking the quality system documentation to ensure the requirements
of the standard are fully met and documented and then establishing
whether the system is actually followed in practice. These are separate
steps in the process and the review of documentation may be done
‘offsite’ with any issues arising and the need for changes (e.g. some
requirements not adequately addressed) discussed with the research
service provider before starting the second stage. The length of time
involved in initial assessments depends very much on the size of the
research company concerned but is unlikely to involve less than a full
day’s work by the assessment body’s staff. A greater input will be
involved for larger companies. Where ISO standard assessment is
combined (e.g. BS ISO 20252 and BS EN ISO 9001), the actual work
undertaken by assessors is not likely to be significantly different to an
BS ISO 20252-only assessment. In either case the same quality system is
assessed.
After initial assessments there is a requirement for continuous follow-up

assessment. The normal practice in this respect is for an assessment body
to make a surveillance visit every year and on each occasion to cover part
of the research company and part of the standards but with complete
coverage achieved over a three-year cycle. After each surveillance visit,
assessment bodies complete a report with a recommendation (if justified)
that the relevant research company remain certificated. The continuous
assessment process is modelled on BS EN ISO 9001 and the two can be
combined.
Companies cannot pick and choose which parts of the standard to follow;
all requirements defined in the whole standard must be met if the
relevant process is carried out at all. All research companies are required
to meet the quality assurance requirements of the BS ISO 20252 standard
and assessment of this part is either by the assessment body against the
quality assurance model defined in Clause 3 of the BS ISO 20252 standard
48
BS ISO 20252:2012, as all similar standards, does not specify how assessment should be
carried out. There is a general principle that all such standards can be assessed by a first,
second or third party. First party is internal self-assessment, second is by the client and
third is by an independent party.

or by prior or concurrent assessment to BS EN ISO 9001. All research

companies are also required to be assessed against the executive module,
regardless of whether they are a full service agency or not. It is assumed
that even a purely field and tab operation will necessitate some
‘executive’ involvement but probably not some of the processes included,
such as questionnaire design or reporting. The requirements, however,
also must be followed when a process is sub-contracted – such as data
collection or data processing – and it would be the responsibility of the
research company to ensure procedures are in place to ensure that third
parties are meeting the requirements of the standard, which can be
difficult if the supplier is not separately assessed to the standard. The
requirements for some types of data collection covered in BS ISO 20252
and for data processing obviously cannot be met if the processes are not
carried out at all. A qualitative research specialist, for example, would
probably not carry out data processing as per that covered in the
standard and only limited elements of data collection might apply.
Implementation
Implementation of BS ISO 20252 is no different in principle (or largely in
practice) to BS EN ISO 9001. In either case research companies need to
develop a unique quality system which both matches the requirements of
the standard and the particular needs and structure of the business. Once
the system is designed, it is then a matter of getting the company and its
staff to work to the system. When the system is working, assessment can
be arranged. The UK assessment specification requires that the system
must have been operating for at least three months before assessment
(in other countries the period may be different), although in practice the
elapsed period is likely to be longer; to carry out a meaningful
assessment enough time needs to have elapsed for there to be adequate
evidence of compliance or not.
A company can choose to implement solely BS ISO 20252, including its

quality assurance model, or together with BS EN ISO 9001 (or indeed
other relevant standards such as BS EN ISO 9004, Managing for the
sustained success of an organization -- A quality management approach,
which complements BS EN ISO 9001). As suggested above, the extra
demands for the other ISOs are not that difficult to meet. Chapter 11
discusses implementation of effective research systems and preparing for
assessment in more detail.
How successful is BS ISO 20252?

Another way of asking the question is, why is a service standard for
market research needed at all? The case for a separate ISO standard

includes the effect on raising standards, providing reassurance to clients

and a flatter playing-field for suppliers.
Widely accepted service standards helps prevent grossly substandard

services by defining the minimum service levels that any research
company should provide to its clients; both buyers and suppliers should
know the minimum that ought to be expected. Assessment to the
standards also means that commitment is demonstrable and not just a
matter of good intentions. This is in contrast to standards such as ethical
codes where there is no systematic mechanism for ensuring adherence as
generally only research that generates complaints or adverse publicity are
ever likely to be investigated. A more uncertain question is, however,
whether defining minimum standards – and that is essentially all the ISO
aims to do – encourage research companies to just meet them or surpass
them. Theoretical arguments can be advanced in either case and
eventually the matter needs judging empirically, but it is still far too early
in the lifecycle of BS ISO 20252 to do this.
BS ISO 20252 provides reassurance to buyers and users of research; they

can assume that an assessed supplier works to a minimally acceptable
level and in many areas which would not normally be transparent – in
many aspects of the research process, there is no way a research buyer
can directly establish whether the project work has been done to
acceptable standards. Independent assessment provides some comfort.
There is of course no guarantee that an ISO certified research company
will do a brilliant job – particularly as the standard does not access the
rights and wrongs of research design, but the chances of it being really
bad are probably significantly less. Research buyers are increasingly
requiring BS ISO 20252 assessment as a necessary pre-condition for
inclusion on preferred supplier lists. But other considerations will of
course still apply, including details of any specific research proposal, prior
experience of research companies, its general reputation and of course
price.
Linked to this is the fact that research companies working to the same
minimum standards will be competing in a more comparable way.
Research prices can always be brought down if corners are cut, but with
research companies working to the same minimum standards, clients are
protected from suppliers tempted to win business on the basis of offering
research which seems to meet the need but is in fact deficient. Research
companies implementing BS ISO 20252 will certainly incur costs, as do
those which are members of IQCS, and they will wish buyers of research
will be able to distinguish suppliers of this standard from others which
may offer lower prices. If quality is compromised, lower prices do not
mean better value. Poor research is never a bargain.
One of the main difficulties with BS ISO 20252 is its fragmented global
implementation. Most countries which had domestic research standards
were able to establish an ISO assessment and certification scheme

relatively quickly after the first standard was introduced in 2006.

Therefore countries such as Australia, the Netherlands, Spain and the UK
were quick off the mark. However, some countries which had national
research standards still have to implement domestic assessment schemes
for BS ISO 20252. This includes some of the largest research markets:
Germany, France and Japan. The largest market, the US, set up an
assessment regime in 2010 and others are expected in the next few years.
With research increasingly multi-national the need for comparable and
consistent implementation of BS ISO 20252 is becoming pressing. In the
end it will be the large suppliers and buyers that will accelerate this
process. Large research suppliers will want all of their outlets to be
assessed to the same standard, to ensure global consistency of delivery
(indeed some are already making their own arrangements in countries
where no national schemes exist); and international research buyers will
expect all parts of the research they buy to be delivered to consistent
standards. Once there is a true global take-up of BS ISO 20252 it will be
possible to take stock and to start to assess its impact on raising research
quality.

So why when the research sector has a comprehensive standard in
BS ISO 20252, covering the entire research standard, is another ISO
standard necessary? The intention and objective of BS ISO 26362:2009,
Access panels in market, opinion and social, research – vocabulary and
service requirements is to ‘apply the quality requirements as specified in
BS ISO 20252 to market, opinion and social research by the use of access
panels and to harmonize already available national standards on this
issue.’49 So this raises the more pertinent question, which is why do access
panels as opposed to any other research approach require a separate
standard?
An access panel, as defined by BS ISO 26362, is a ‘sample database of

potential respondents who declare that they will cooperate for future
data collection if selected’. The emergence and popularity of access
panels has come about primarily due to decreasing response rates:
time-poor respondents, apathy to research, concerns about privacy,
confusion regarding the difference between research and other data
collection activities (such as direct marketing) and fear of the unknown
(in terms of quality of the experience, how data will be used) have
created a potentially significant problem for research. People –
respondents, participants, etc. – are the lifeblood of research, without
them there is no sector. But with willing respondents increasingly difficult
to find, the research industry needed to find a way to reach those willing
49
From the Introduction of BS ISO 26362:2009, Access panels in market, opinion and social,
research – vocabulary and service requirements.

to participate. It responded through the creation of ‘access panels’ of

respondents who were willing to cooperate in future research. Virtually
non-existent before the start of the 21st century, the benefits of this
approach were obvious: lower costs, less recruitment, quick turnaround,
etc. Access panels work well with self-administered on-line data collection
although more traditional forms of data collection can also be used with
access panels.
The US was one of the countries which led the way in developing online
panels, Inside Research (2009) estimated the total US spend on online
research in 2009 at about $2 billion, the vast majority of which was
supported by online panels. In ESOMAR’s Global Market Research Report
2011, online quantitative research methods accounts for 22 per cent of
research spend, a significant amount of which will be via access panels.
Increasingly there has been concern about panels, particularly in terms of
panel quality. Procter & Gamble, and a number of other big research
buyers, reported that various online research panel projects appeared to
produce inconsistent results. Questions began to be asked about whether
panel respondents were in fact professional respondents, whether
frequently used panel members became increasingly unrepresentative of
the population, whether some respondents were taking part in too many
research projects (including amending screener questions to qualify for
participation) and the rapidity with which some respondents were
completing research surveys and whether such results could be relied
upon.
To stem this flow the sector needed to take action. The sector’s response
needed to be a layered approach due to the nature of the problems,
which were a mixture of research process, design and buyer
education/awareness. Initiatives such as the American Association for
Public Opinion Research Opt-In Online Panel Taskforce was formed in
2008 to review the ‘current empirical findings related to online panels
utilized for data collection’ with an aim to ‘provide key information and
recommendations about whether and when opt-in panels might be best
utilized and how best to judge their quality’ (AAPOR, March 2010).
Similarly ESOMAR developed its 26 Questions to Help Research Buyers of
Online Samples with the aim of aiding researchers and their buyers to
decide whether an online sampling approach is fit for purpose in relation
to specific research objectives. Other countries such as Australia
developed domestic process standards for online panels (the QSOAP Gold
for online access panels). Other initiatives such as the Association
Collaborative Effort (‘ACE’) a collaboration between six research
associations (primarily in the US) was formed to address and improve
online research quality, and this has resulted in initiatives such as the
release of a set of online research definitions in 2009. New initiatives
continue to emerge, in 2011 ESOMAR and CASRO50 announced plans to
50
CASRO is the US Council of American Survey Research Organizations.

produce a standard for online sample quality. It was against this

backdrop that BS ISO 26362 was developed and it forms another part of
the quality jigsaw, sitting alongside the initiatives of AAPOR, ACE,
CASRO, ESOMAR, etc.
Scope and definitions

BS ISO 26362 is applicable to all types of access panels including those
delivered offline (e.g. via telephone, post or face to face) as well those
online. The standard ‘develops the criteria against which access panel
providers can be evaluated and against which the quality of access panels
can be assessed’.51 One of the key benefits of the standard is that it
contains a comprehensive set of definitions which sets out the
parameters of the standard. For example, in addition to defining an
‘access panel’ and ‘panel member’, the standard also defines what
constitutes an ‘active panel member’: as a ‘panel member who has
participated in at least one survey if requested, has updated his/her
profile data or has registered to join the access panel, within the last 12
months’. The development of such definitions is essential in developing
quality within access panels, and it was the need for this type of very
specific definition which justified, at the time, the separate ISO standard
being developed.
As would be expected the document is entirely consistent with the

requirements of BS ISO 20252 and contains cross references throughout
to the main research standard. Indeed, one of the reasons for the 2012
update of BS ISO 20252 was to ensure that the main research standard
was consistent with the requirements as defined in BS ISO 26362 (which it
now is). BS ISO 26362 is in essence a sub-set of BS ISO 20252, as opposed
to an isolated access panel standard. A research company that meets all
the requirements of BS ISO 20252 should be able to meet BS ISO 26362
for its panels. However, a panel company that meets BS ISO 26362 would
have more to do to meet the requirements of BS ISO 20252 (although
not necessarily a large amount depending on the range of research
services being undertaken).
Requirements
The requirements of BS ISO 26362 are based on the principle that any
research access panel will be ‘actively managed’ rather than just a
database of respondents who have at some time in the past indicated a
willingness to participate in research. The panel also requires panel
providers to have a quality management system in place which covers all
the requirements of BS ISO 26362, plus the organization and
51
From the Scope section of BS ISO 26362:2009, Access panels in market, opinion and social
research – Vocabulary and service requirements.

Are two ISO standards needed?
responsibilities, documentation requirements, competence and training,

and subcontracting/outsourcing requirements as detailed in BS ISO 20252.
The remainder of the standard’s requirements focus on the management

of panels, and ensuring management is transparent so that clients and
panel members understand the basis on which panels have been
constructed and maintained. For example the section on recruiting panel
members includes requirements for confirming panel members’ identities,
the section on panel structure and size includes characteristics for
determining an access panel size and capacity, panel management
includes requirements on panel maintenance including panel member
history, etc. All of these requirements exist to address the concerns about
who panel members are and whether, over-time, these respondents
become survey answering machines no longer giving true views and
opinions in their responses. The requirements of the standard addresses
some of these issues but broader issues such as sampling bias have to be
documented and reported as a requirement of the standard, as opposed
to requirements for specific methods; this again is a research design issue
and therefore outside the parameters of this type of standard. Other
more general research requirements within BS ISO 26362 such as
questionnaires, pre-testing, translations, validation of data and reporting
are all based on BS ISO 20252 requirements.
Are two ISO standards needed?

Even with the recent update of BS ISO 20252 to add the BS ISO 26362
requirements, there is still an argument for the continuance of
BS ISO 26362 as a separate standard. Many of the new entrants into the
research sector are suppliers that specialize in delivering online panel
research. The existence of BS ISO 26362 ensures that such companies have
a quality standard that is exactly suited to their business without
necessarily undertaking all the requirements of BS ISO 20252, many of
which may not be applicable. This is not dissimilar to field only
companies obtaining the IQCS (which duplicates the data collection
section of BS ISO 20252) instead of the BS ISO 20252 standard. For
ensuring quality in market research the existence of a separate specialist
quality standard at least means that all dimensions of the sector have the
opportunity to work to some form of process standard.
Panel owners in some countries, most notably the Netherlands and

Australia, have adopted the ISO 26362 standard, the first company in the
US to achieve the standard did so in 2011 and other countries such as the
UK have a number of companies working towards achieving the
standard. Aside from these pockets, penetration of the standard is still
low and it is still too soon to assess whether the research sector can
support both ISO standards. Nevertheless, the success of any standard will
largely be determined by the market, particularly buyers. BS ISO 26362 is

a welcome addition to addressing particularly the way in which panels

are constructed and managed. If buyers make adherence to the standard
a procurement requirement, the standard will succeed. Whether this
turns out to be the case is still too early to tell. Indeed, this is also true
for the other industry initiatives that have been developed to address the
issue of panel quality. Online research is definitely here to stay, but what
further steps will the sector need to take to address the quality concerns
of clients and to protect the long-term future of research?

10 People quality – Investors in

People
Whatever systems are in place, delivering quality requires workforce skills

and commitment and in turn this requires businesses to provide staff
training and development. To carry this out efficiently, a planned rather
than a purely reactive approach is needed. This chapter discusses, in
general terms, appropriate training planning and shows how this fits well
into the framework provided by the Investor in People (IiP) standard. The
principles of this short standard are reviewed together with the required
indicators of commitment to these principles. The Investors in People
assessment system, with its stress on commitment, is also described,
together with the role that National Occupational Standards can play in
supporting skills assessments.
Quality, of whatever sort, depends on people and no more so than in a

market research business. Systems and standards can help deliver quality,
but not without trained and willing managers and staff. A major need in
quality management is, therefore, to ensure staff members have the
necessary skills and commitment.
What is required?
What is required of staff to deliver quality? There are two major and
linked requirements: commitment and skill. Commitment is about
accepting the importance of quality and wanting to deliver it, all the
time. Various morale building approaches – management leadership
(including by example and exhortation) – all have their part to play, as
does staff involvement in quality improvement through internal audit
and quality teams, special task groups, and involvement in the
problem-solving aspects of a quality system. By these and other means,
the managers and staff, at all levels in a company, can develop a regard
for quality – the need to strive for excellence in research, meeting or
more than meeting clients’ needs, working to the specification, and
maximizing efficiency. However, for any of this to work, staff need the
appropriate knowledge and skills to perform their jobs in the best
possible ways. Regardless of the morale level, quality cannot be delivered
if the knowledge and skills are not there and in the long run frustration
will undermine morale.

10 People quality – Investors in People
To meet a short-term need, some knowledge and skills can be bought in.
Staff can be hired to enable a company to offer a new sort of expertise
and sometimes this is essential. If a quantitative agency decides to extend
its services to qualitative research, there is no alternative but to recruit at
least one or two key people if they want to do it properly. To fill a
shorter-term need, temporary or freelance staff can be brought in.
However, recruitment can never be the only or even the main means of
building up a knowledge and skills-base. Existing staff have to be trained
and their skills developed. This process has to be continuous because
knowledge moves on relentlessly; no one ever knows enough or has
enough applied skills. Even when new staff members are taken on, they
must also receive some in-house induction training to fit them into their
new team, corporate processes, etc. No company, least of all a market
research company, can ever ignore learning and staff development.52
Aspects of training and market research

All market research companies will train and develop their staff to a
greater or lesser extent, and this can be a significant investment in an
individual for smaller companies. Professional development expenditure is
inevitable but it can easily be wasted. Spending more does not always
mean better learning and development; it all depends on how it’s
managed and built upon. This can be illustrated by describing two
models – a bad one and a better one – of professional development
focusing on training in particular.
Some of the training carried out by market research companies (and

many other businesses) is unplanned and reactive. There is recognition
that new skills, knowledge, and training are good things in general and
there is some willingness to make money available and perhaps even set
a budget. However, the training selected is unrelated to any broader
planning of the company or even at the individual level. A common
reason why someone goes on a particular training course is that a course
is widely promoted, and a member of staff (or his or her manager)
‘fancies the course’. In this way various members of staff are trained but
no one considers the development priorities, whether the particular
training actually delivers the learning outcomes, or conducts any
post-training assessment to review and build upon the learning from the
course. Eventually, during the year, the money runs out and training is
put on hold. Perhaps any staff members not receiving promised training
are told that it will be their turn next year.
52
We shall not make a hard distinction between staff training and professional development.
However, training can be thought of as short-term and structured around specific learning
outcomes, while professional development is more of a personal development plan based
around specific, identified learning needs which may need to be fulfilled via a variety of
formal and informal development activities.

What is required?
In case the reader is in any doubt, the above is the bad model of
training. It may be an exaggeration. Perhaps market research companies
are not that bad! But in many cases the organization of training is
probably not much better than this. So what is a better model?
The key difference is in planning, starting with what is needed. At least

once a year or perhaps even every quarter, the managers of the company
need to consider what staff knowledge and skills will be required. This
may be over different timescales but perhaps most of the emphasis
should be on the year ahead, with some regard for the long-term
objectives for a staff member e.g. Are they a promotion prospect? A
future manager? What skills will they need to do this effectively? The
development considered will include, of course, professional market
research expertise but also the skills required by other staff as well:
interviewers, IT, administration, and finance, for example. As well as
technical and functional skills, it may be important to think of more
general ones such as business development for staff involved in
client-facing roles.
The impact of technology also affects professional development

requirements – all professional staff members require basic technology
skills and those involved in online research will require considerably
more. The same is true for interviewers who for the most part would be
expected at the very least to cope with CAPI and CATI technology. Above
all, however, skill requirements cannot be divorced from other business
planning. If the turnover objective for the coming year is a 20 per cent
increase, where will the new business come from and what resources,
including skills, will be needed to gain and deliver the extra business?
Assessing skills
Once the professional development needs are defined, the next step is a
‘learning and skills’ audit. To what extent are the required skills present
already and what gaps exist? Such gaps can be both quantitative
(numbers of staff with the appropriate skills) and qualitative (key skills
entirely absent or at too low a level). Thought can then be given to how
the gap is to be filled. The two broad solutions are buying in –
recruitment – or internal training. Qualitative gaps are more likely to
involve the former, although there are no hard and fast rules. The
professional development plan (but not the broader business plan),
however, will be less concerned with the recruitment side.
So far the plan has been developed by the company’s management

without much staff involvement. Now is the time to consider the training
needs of each individual member of staff. In a small company one
manager can take on this task, but above a certain size the work will
have to be delegated to departments. The best way is face-to-face
meetings with individual staff. These can be informal or tied into

something such as an appraisal system. When past and future

performance is being considered, training will inevitably have to be
thought about.
Individual training needs can and should be considered in relation to

three dimensions: the company’s overall skill needs, the demands of the
person’s role, and his or her own wishes and personal career
development plans. The broad knowledge and skill needs of the company
should be known by this time and it is now a matter of seeing how
individuals can be developed accordingly. The demands of the job and
the individual’s performance may also define where further professional
development is needed. Often skills are sufficient to perform the present
job adequately, but what is needed to really excel or meet new needs as
the job changes? (Tools such as National Occupational Standards can help
in the process but more about those later.) Also, a person planning a
long-term career with a company should be encouraged to think of the
skills needed to move into a new role. This then merges into personal
wishes and ambitions – staff members usually want training to improve
their own employment position and earning potential. Most company
and personal plans can be reconciled, but this is not always the case and
possibly some training or development cannot be sensibly
accommodated, or at least substantially financed by the company. It may
be very commendable that someone wishes to study Victorian art but the
practical contribution to business growth may be limited.
With needs at both company and individual level defined, an action plan
can be drawn up covering the forthcoming year and possibly longer
term. The plan will set priorities in terms of what training is to be given,
to whom, and with what amount of the budget (there may obviously
have to be compromise and trade-off at this point). The methods of
delivering learning also need to be decided. This may involve sending
staff on external training courses or developing a bespoke course to be
delivered exclusively for staff in-house. However, it is a mistake to assume
that effective training always has to involve external, paid-for courses. In
any company, there is always potential for skill and knowledge sharing;
senior staff and staff who have been on external courses can hold their
own seminars. There is also the development side: skills and knowledge
can be self-taught and, in truth, any theoretical subject can often be as
well mastered by old-fashioned reading as by attending seminars. With
the advent of online resource there is a wealth of information that is
now readily available as either free training or low cost online training.
Informal coaching, hands-on learning and buddy systems (where staff are
put with more experienced staff from whom they can watch, listen and
learn) are also effective means of knowledge and skill delivery and the
only drawback to them is that they are often not recognized for what
they are. Staff may be well trained, but might not think of informal

What is required?
methods as training; often the price tag determines the value that staff
place upon it but often the best training can be the free hands-on
learning from colleagues.
Learning and professional development in market research should be

linked into the profession, and the programmes organized by the various
professional bodies are usually the best place to start.
Evaluating skills
To complete the training cycle, evaluation of the implemented plan is
required, with any lessons learned applied to the next year’s plan. First of
all, it should be established whether the plan was substantially delivered.
Perhaps for one reason or another some of the training was just never
delivered. Why? There may have been good reasons and these should be
considered next time round. Also the plan may have had to be changed
to meet circumstances – the need for additional knowledge and skills
may have been recognized over the year. Planning is essential but it must
not be so rigid that it cannot adapt as needs change.
Evaluation should also concern whether the training delivered was

effective. This should involve the recipient’s own judgements and can be
communicated informally or via course satisfaction scores, for example.
However, for the evaluation to be complete there should also be a
judgement on whether staff were not only happy with the courses, etc.
but subsequently proved they applied new knowledge and skills in the
business. Sometimes, because of the time lag between training and
application, this sort of judgement is difficult to make. Activities such as
post-training assessment, learning logs, and professional development
plans can all help in measuring the effectiveness of training and the
impact on an individual’s progress. It is also often difficult to measure
improvements in performance resulting from training. In the case of
simpler process skills it may be possible to measure productivity gains, but
the benefits of more professional skills are much harder to quantify;
there may be clear advances, but to what extent are they a direct result
of training as opposed to experience? In most cases it will be a
combination of the two. Although we may learn by doing, we often do
not recognize what we have learnt.
However, rather than simply knowing how to do something, it is also

important for staff to show they know why they are doing it that way.
Having to take time out to think about work, and the principles
underpinning it, helps individuals to recognize how much they are
learning at work. This is where qualifications can be very helpful. By
having to articulate about market and social research issues, in an
examination for example, staff practice explaining what they know. From
an employer perspective, qualifications can also help to identify those
who really understand what they are doing and why they are doing it.

Finally, there needs to be an evaluation of whether the overall learning

and development carried out over a year has made a positive
contribution to the company’s progress. Again, much of this will have to
be judgement rather than measurement, because causes and effects are
often so difficult to separate. During the year the turnover targets may
have been met, but how much of this is down to extra skilling and how
much is the result of other factors such as better promotion, a strong
economy or just self-sustaining growth? However, difficult or not, overall
evaluation is needed in order to sensibly plan the next year’s training and
development.
This is the recommended ‘better’ model for learning and development. It

just so happens that the model put forward closely matches the IiP
standard (all right, it is not entirely fortuitous that it does so) and later in
the chapter we shall describe IiP and suggest that it can be usefully
adopted by research companies. The model (and IiP) also links in to other
standards. As discussed in Chapter 7, BS EN ISO 9001 has a requirement
for training as does BS ISO 20252, which also includes training for
interviewers discussed in Chapters 8 and 9. The recommended training
model and IiP dovetail into BS EN ISO 9001 and BS ISO 20252, but go
beyond the requirements for training in these standards. A company
carrying out training to the IiP standard will more than meet the general
training and staff development requirements of BS EN ISO 9001 and
BS ISO 20252. The standards for interviewer and supervisor training can
also be easily fitted into IiP. But before all this, we need to consider one
of the tools available for assessing people – National Occupational
Standards.
National Occupational Standards

National Occupational Standards (NOS) specify what a person must know
and what they must be able to do in order to be considered competent
in their job roles. They are standards because they define a general
accepted quality benchmark that considers all the specific requirements
for a workplace. They are occupational because they provide clear
descriptions on the roles and specific tasks required by an occupation.
They are national because they describe the skills, knowledge and
understanding needed to undertake a particular task or job to a
nationally recognized level of competence (see Carroll and Boutall 2010).
Developed by employers, NOS have gained traction in the last few years
as they are used as a benchmarking tool to assess the performance of a
broad range of workers to match the requirements of a global market
(such as the need for staff mobility across country borders).
NOS have been developed by different countries, across a wide range of

job roles such as business and administration, management and
leadership, customer service etc., and most share common characteristics.

One of the most comprehensive NOS which covers market research are
those developed in the UK and we will describe these to illustrate how
NOS can be used for developing and assessing research staff quality and
competence.
The research NOS are part of a wider set of NOS covering the whole of
marketing and marketing communications. The basis for marketing NOS
is a functional map (see Figure 9) which in addition to setting out the
specific marketing and marketing-related activities, also includes more
general knowledge and skills such as management and finance (a
comparable and mirror set of standards for social marketing including
social research has also been developed in the UK).
Source: UK Commission for Employment and Skills
Figure 9 – UK’s National Occupational Standards: marketing standards

functional map

Within each of the eight elements of the functional map there are a
multitude of units (around 100 of them) each describing a part of the
function and identifying performance criteria, behaviours and knowledge
and understanding, which individuals involved in that activity should
display. An example of one of these units – for collecting market research
data is shown overleaf. As you can see there is nothing revolutionary
here; someone involved in collecting market research data should be able
to select the right research environment, use the research techniques,
have good communications skills, and understand the legal environment,
etc. The value of NOS is that they are a ready-made toolkit of some key
standard functions that exist within marketing and market research. In
the UK the NOS are used to develop vocational qualifications, i.e. any
government accredited qualifications must clearly map against the
relevant NOS. On a more practical day-to-day basis, the NOS can be used
by employers to assess and benchmark staff performance against a set of
established criteria, help to identify skills and knowledge gaps that need
to be filled and can be used in appraisal schemes and developing job
roles. They also provide managers with a tool for workforce management
and quality control, can assist with setting individual and team objectives,
and can assist with designing and developing in-house training
programmes (see Figure 10 for an example of how it can be applied in
practice).

Figure 10 – How National Occupational Standards can be applied in

practice

An example of a National Occupational Standard for research – collect

market research data:
1.2.4 : Collect market research data
What this unit is about
This is about collecting data using relevant market research

methodologies and tools.
Who this unit is for
This unit is recommended for those involving in collecting market

research data. Such researchers may be employed directly within
the organisation commissioning the research, or for an
agency/consultancy. February 2006 (Level 2/3)
Outcomes of effective performance
You must be able to do the following:
1. Ensure that respondents understand what is required in the

data collection process, including, where appropriate, the
purpose and duration of the process
2. Establish an environment in which respondents feel confident
and comfortable in providing information
3. Select from a range of techniques those which are most
appropriate for the research project
4. Where relevant, use research materials correctly and in
accordance with the aims of the research plan
5. Conduct the collection of data in line with the aims of the
research plan
6. Record findings accurately and fully, in line with the
requirements of the research plan
Behaviours which underpin effective performance
1. You adopt communication styles appropriate to respondents

and the research situation
2. You present information clearly, concisely and accurately
3. You listen actively, clarifying points to check mutual
understanding
4. You comply with legal requirements, industry regulations,
organisational policies and professional codes

Investors in People
Knowledge and understanding
You need to know and understand the following:
General knowledge and understanding
1. The role of data collection in relation to a given research

project, and the processes required to fulfil this role
2. The ethical and legal frameworks which underpin data
collection
3. The processes involved in the effective collection of data
4. How the aims of the project might affect an individual’s
willingness to participate as a respondent, and how to respond
accordingly
5. How your style of communication can affect an individual’s
willingness to respond
6. How to select the communication techniques which are the
most appropriate for different respondents, and are in keeping
with the requirements of the research project
7. The importance of accurate and appropriate recording of data
8. How to record data appropriately and accurately in relation to
a given research project
Industry/sector specific knowledge and understanding
1. Legal, regulatory and ethical requirements affecting the

collection of market research data
Context specific knowledge and understanding
1. The purpose and aims of the data collection

2. Where applicable, the organization, and its products/services
appropriate to the research being undertaken
Investors in People
The IiP standard was developed in 1990 by a group of employees and
employers, and was launched nationally in the UK in 1991. In October
1993, Investors in People UK was established as the body responsible for
the standard. Since its launch IiP has been adopted in countries
worldwide in Africa, Europe, North America, the Middle East and
Australasia (see Investors in People International for more details); for the
purposes of this chapter the UK publication, as the originator of IiP, will
be used.

When Investors in People was launched the then UK Secretary of State

for Employment, Michael Howard, said ‘Investors in People is a standard
designed by business, which will be met by business, because it is in the
interests of business.’ Across all types of business the uptake of IiP has
been considerable; about 30 per cent of the UK’s workforce is employed
either by organizations that are recognized as IiP employers or
organizations working towards achieving recognition status.53 The
Institute of Employment Studies has found that IiP-recognized companies
generate higher gross profits per employee than non-IiP companies.54 At
the time of writing at least 40 UK research companies have implemented
IiP.55
The standard is based on three fundamental principles:
• Plan – Develop strategies to improve the performance of an

organization
• Do – Take action to improve the performance of an
organization
• Review – Evaluate the impact on the performance of an
organization
Overlaying the three principles are ten indicators:
• Plan: business strategy, learning and development strategy,

people management strategy and leadership management
strategy
• Do: management effectiveness, recognition and reward,
involvement and empowerment, and learning and development
• Review: performance measurement and continuous
improvement.
The standard has become more flexible, with businesses now able to
concentrate on those parts of the IiP framework which are most relevant
to its business.
Commitment requires companies to have in place written business plans

which, alongside financial and other goals, take into account learning
and development needs. Such plans can be in any format and may be
spread over several documents that are part of more general ones (e.g.
Board minutes); IiP requires some documentation, but it is not at all
53
Bourne, M and Franco-Santos, M; Investors in People, Managerial Capabilities and
Performance – A Study by the Centre for Business Performance, Cranfield School of
Management (2010).
54
Institute for Employment Studies, Does IIP add value to business? IES Report (2008).
55
Based upon organizations listed in MRS publication, the Research Buyers Guide 2011 and
accredited with IiP.

Investors in People
prescriptive about the form this takes. IiP focuses on outcomes; what
needs to be achieved rather than how to achieve it. Commitment also
implies some longer-term vision or purpose of where the company is
going and it is essential that this is effectively communicated to all staff
(including the contribution they will make to realizing it). The vision, of
course, does not have to be overambitious; it is perfectly satisfactory if a
company wishes to stay the same size. Commitment to growth is not
essential (indeed can be fatal if an organization falls into the trap of over
trading) and staying in business has to be core to any business ambition.
The indicators related to this part of the standard, such as mission
statements, core values, business strategies, key performance indicators,
etc., cover visible evidence of the commitment with staff involvement in
development. When it comes to assessment, what staff know, believe and
feel is as important as any documentation.
The requirements to regularly review learning and development needs,

means a company must have mechanisms in place to discuss these with
each individual. The learning and development needs should be related
to both individual and corporate needs, and managers need to also
consider what resources will be needed to deliver the required learning
and development. It should not be assumed, however, that IiP necessarily
involves spending any more on training or other forms of external
professional development; the same budget may just be spent more
effectively. Responsibilities for implementation of training also need to
be defined and understood – who will make sure it all happens? The
assessment indicators required are evidence of the above in various sorts
of records as well as the direct involvement and understanding by the
staff involved. The standard requires a link between personal learning
and development needs and the corporate vision and mission i.e. you
need to be able to identify how personal learning and development
contribute to the corporate goals. Taking action to train and develop
staff has to cover both new recruits and existing workers, and continuous
learning should be at the heart of the organization. Staff members
should take responsibility for their own learning and development (in
conjunction with their line manager). Indicators in this case should show
what happens and could include, for example, records of professional
development plans and learning logs, evidence of training courses, and
the evaluation of learning opportunities.
The final part of the standard concerns reviewing investment in people,

looking at what has been invested, the impact of the investment, and
planning future learning and development. Reviews should cover both
the company and the individual level (and possibly departments in
between). The senior management of the company is required to come
to an overall judgement of the programme and draw implications for the
future. Assessment indicators cover various types of evidence –
documentary or otherwise – to show that this has been done.

As a document the IiP standard is relatively short and user friendly. There
are no difficult terms and the language is easy and non-technical. There
is no need for any sort of translation to relate the standard to the
practices and structure of any particular industry. A fair criticism of the IiP
standard, however, is that it can appear to repeat itself in several places,
but this is quite easily untangled.
Implementing IiP is not particularly difficult for a company following

something approaching the ‘better model’ of learning and development
as previously described. Providing there is commitment, planning, action
and review and providing there is full staff involvement, little if any
additional input will be needed to satisfy the IiP standard. Documents
such as business plans and professional development records will be
needed, but these can be in any form and do not need to measure up to
any format requirements. Any company seriously committed to planned
learning and development will almost certainly maintain such records in
some retrievable and assessable form.
IiP is non-prescriptive and very flexible. However, IiP is not in any sense
an alternative to quality assurance systems. It addresses quality but from
a different and complementary approach. There is no requirement for an
BS EN ISO 9001- or BS ISO 20252-accredited company to seek IiP or vice
versa but they hang together well. If a formal quality system, such as
those found in the ISO standards, is in place, it is sensible to integrate the
documentation and records of learning and development into the formal
system. If the documents and records are needed they should be well
kept and a formal quality system is, among other things, an effective way
of ensuring this.
While IiP is flexible and non-bureaucratic, the need for commitment

cannot be overemphasized. Establishing true commitment and going
behind documentary paper trails is very much at the heart of the
assessment process.
A timetable for implementation of an IiP programme is needed and

realistically this takes at least one complete planning cycle; for most
companies (depending on size and complexity) between 12 and 18
months is required to implement IiP. At the start of the year learning and
development is planned as part of the overall business plan. Meetings
with staff as well as management time will also be necessary. Over the
year the plan is put into action and towards the end of the period a
review is completed ready for the cycle to start again. Only at this point
is the company ready to consider assessment. There is an online business
support tool available on the IiP website, which features a simple
20-question diagnostic questionnaire56 which indicates how well an
56
Investors in people, http://www.investorsinpeople.co.uk/interactive/Pages/default.aspx
[accessed 12 April 2012].

Assessment to IiP
organization is performing against the standard. This tool also offers a

step-by-step tour through the standard to give an impression of what IiP
is about.
Assessment to IiP
Assessment is carried out by independent consultants from local Investors
in People Centres. Normally the process will begin with a meeting to
discuss the business priorities, what is involved in assessment and to
agree how the process will be conducted. An important part of this
discussion is agreeing the scope of the assessment and discussing an
organization’s business priorities, performance targets, etc.
The form of the assessment, but not the content, is similar to

BS EN ISO 9001 assessment. In this case the off-site ‘desk investigation’
involves examining a portfolio of evidence put together by the company
seeking assessment. In theory at least, the portfolio should contain little
material prepared specially for the assessment and is instead mainly
copies of documents produced in planning, delivering and evaluating
learning and development. Examples of material that might be contained
include:
• a brief background on the company and its business (a sales brochure

could well meet this need),
• staff and responsibility structure (e.g. a company organigram),
• the business plan; learning and development plans; the vision or
mission statement,
• copies of other quality related documents,
• a summary of any review carried out by senior management of the
effectiveness of the learning and development completed (this might
be an extract of Board minutes),
• samples taken from records of staff appraisals or learning and
development planning meetings and the details of specific learning
and development (such as training).
In summary, it should be whatever gives a picture of how learning and

development works in a company. Unless it is very deficient, the portfolio
of evidence is not regarded as evidence of compliance or otherwise with
the standard (as is BS EN ISO 9001 documentation). Rather it is used by
the assessor to structure the on-site assessment.
The on-site assessment may take several days for a medium-sized research
company and rather longer for the largest or those with several sites.
Small companies may be completed in a day or two. Most of the time
during the assessment is taken up with discussions and meetings and
involves as many staff as possible. IiP assessments are entirely
interview-based. The paperwork can provide practical examples and
evidence, but the assessment will be based on the discussions with staff.

Compared with BS EN ISO 9001, there is a larger subjective element with

the assessor seeking to establish that the company is ‘for real’ – that the
plans are not just documents but lead to action, that managers ensure
things happen and staff members are aware and understand their
involvement in both training and realizing the company’s vision. Evidence
of positive attitudes by the workforce to learning and development and
the business as a whole contributes to a successful assessment but a
complete lack of dissonance between workers and management is not
demanded. Realistically it is recognized that not everyone can be happy
with their own or general employment conditions all of the time. During
the year it may have been necessary (or thought to be necessary) for the
management to take decisions which were not welcomed by sections of
the staff.
On completion and following a short period of reflection, the IiP assessor

will provide immediate feedback, giving an initial response to what they
have seen. In most cases the assessor will indicate at this point if the
assessment has been successful. A final comprehensive report is provided
afterwards, and this can include ideas and suggestions for improvement.
Joint or Parallel assessments can also be carried out whereby assessment
for IiP is undertaken with other assessments, an BS EN ISO 9001
assessment for example (this would be less likely with specialist BSs such
as BS ISO 20252).
IiP: In summary
Being a recognized Investor has some kudos and may impress clients or
even potential staff to some extent. However, it is very doubtful that
work will be gained just on the strength of the plaque. If IiP is worth
having, it must be because the processes necessary to meet the standard
produce benefits quite unrelated to assessment or recognition. If a
company is following anything approaching the suggested ‘better model’
of professional development it will be doing nearly everything needed to
gain IiP recognition. And the commitment to learning and development,
implicit in the model, is in any case essential to delivering quality and
pursuing business success. Therefore, why not seek IiP recognition? There
will be some assessment costs but the status that is gained – and this
includes internally as well as externally – will be at least commensurate.
Indeed, based on the evidence of numerous research studies into the
impact and effectiveness of IiP (e.g. Bourne and Franco-Santos, 2010),
there is good evidence to suggest that organizations that have IiP
perform better financially and thus any investment in the cost of IiP is
more than recouped in financial growth.57 In addition, the process of
seeking IiP is itself a spur to action. Learning and development are of
57
Of course it is possible that the companies that perform better are more likely to be
receptive to investing in IiP.

IiP: In summary
course essential but the prospect of assessment gives that extra reason
for doing things right and doing them when they should be done.
At present the penetration of IiP is still relatively modest when you

consider the total sector (e.g. among UK research companies listed in
MRS’ Research Buyers Guide IiP accredited organizations represent
around 10 per cent). One of the problems for research companies, except
the very largest, is their capacity to invest in the short-term in something
that will not pay dividends until later (although it should be noted that
those that are IiP-accredited come from across the size spectrum so size is
not specifically a determining factor). Many small and medium research
companies are dependent on ad hoc research projects and these by their
nature can make it difficult for organizations of this size to release staff
time to devote to ‘non-research’ activities. Although management time
will certainly be taken up in planning and delivering the required
investment in processes and learning and development to gain IiP, if this
cannot be spared it could be argued that there is little long-term hope to
improve quality or for that matter to develop the business at all.

11 Quality system development,

implementation and assessment
This final chapter is a practical guide to implementing and being assessed

to standards such as BS ISO 20252. Topics covered include; the resources
and commitment required by a company, preparing the required
documented quality system, internal auditing and follow-on action to
establish successful implementation and preparing for assessment and
certification. Discussion of a documented quality system is illustrated by a
case study and at the end of the chapter a specimen example of a
complete quality system is provided.
If a research company wants to be recognized as complying with

BS ISO 20252, BS EN ISO 9001, or a similar standard, it needs to develop a
documented quality system which at least covers the requirements of the
standard, implement this system – and ensure it works effectively – and
then be assessed and certified appropriately. This chapter provides
guidance on how to do this. BS ISO 20252 is the particular standard
discussed but the approach is not any different for BS ISO 26362,
BS EN ISO 9001, or national market research standards. The suggested
approach is suitable for any small or medium research company and is, in
principle, applicable to a larger business although inevitably it would be
a larger and more complex project. In many cases the boundaries of the
organization implementing the standard will be clear and obvious, i.e.
the whole research company. Where a research provider is part of a
wider structure, the decision can be made to restrict implementation to
the particular part as long it is clear what is covered and what is not and
that any claims of meeting the standard are suitably qualified.
Commitment and resources

Developing and implementing a quality system is a significant project for
a company, costing time, effort, and money. Before anything else,
commitment is needed; commitment not just to start the project but to
see it through to a successful conclusion. Most market research
companies have businesses based on ad hoc commissions with short
notice of projects needing to be completed to tight timetables and with
inevitable periods of slack and frantic activity. It is easy to be committed

11 Quality system development, implementation and assessment
to developing a quality system in a quiet period, but will it be forgotten

once work capacity is stretched, bearing in mind that the project is likely
to take at least a few months?
A commitment to developing a quality system needs a realistic timetable

but one that is flexible enough to accommodate the business peaks and
troughs. Companies, in the abstract, do not have commitment of course;
it is individuals and groups who have it and in the first instance this
commitment needs to come from the most senior members of the
company: the directors, the board, partners, etc. Ideally one senior
member, perhaps the CEO, needs to be the champion for a quality system
and BS ISO 20252 and needs to be committed to seeing the project move
forward. This champion does not have to be the same as the hands-on
project leader, responsible for doing or getting done the required work
and this role is discussed shortly.
Another aspect of the commitment needed is an obligation and openness

to the changes that will be required in company practice to meet the
requirements of BS ISO 20252. As will be discussed shortly, these changes
will probably be neither as many nor as daunting as may first appear but
they cannot be ignored. Until some preliminary work has been done – an
initial audit, see below – the magnitude of change is unknown. To start
with, then, commitment to the project may be limited to this initial step.
As mentioned, a project leader is needed to develop and implement a

quality system and often the same person will take on the on-going role
of quality manager once the system is up and running, although this
does not have to be the case. What qualifications are needed to be the
project leader?
The first is time available for the role. This will vary with the size of
company and the extent to which that expertise is bought-in. The role
will not need to be full-time, except in a large company, but some diary
space is essential, including in the busy business periods. Whoever takes
on the role needs the backing of the quality champion and other senior
staff to make time available. The project leader does not have to be a
hands-on professional researcher but does need to understand, in at least
broad outline, what is involved in the research processes. He or she also
needs to know the structure of the company and who does what; in a
smaller company this is not likely to be a problem. Related to knowing
the company are people-skills – in communicating, persuading, and
obtaining information from colleagues (this should not be too hard if the
project leader is a researcher!).
Finally, there is the need for specific expertise about quality systems and
BS ISO 20252. Sometimes this may have been gained in another company
but this would be fortuitous. The necessary knowledge can come from
various sources such as courses (in the UK, relevant courses and
workshops are offered from time to time) and reading…and not least

Commitment and resources
reading this book. Another solution to a lack of expertise is to buy it in

from an independent consultant or short term recruit.
If the right consultant is brought in, not only will expertise be provided
but some of the internal workload of the project can be reduced,
including drafting quality system documentation. Furthermore, with,
hopefully, practical previous experience as well as expertise and possibly
less distraction from day-to-day work, a consultant will enable the whole
project to be completed more quickly than by someone learning as they
go along. Also, the consultant may reinforce the commitment; he or she
is costing the company money and the benefits need to be reaped.
However, there are drawbacks to using consultants (apart from costs),
and the biggest is that the project and resulting quality system is
something brought from outside, bolted on and therefore in danger of
never really being ‘owned’ by the company itself. A poor consultant may
also try to impose standard solutions without taking account of how the
company functions. As will be discussed, the documented quality system
should be unique to a particular company and off-the-shelf manuals will
not work. The question of whether or not to use an external consultant
is, therefore, something to weigh up and debate. In most cases it is
perfectly possible to do it all in-house, accepting that the project may
take longer.
If a company decides to use outside help there is the question of how to

find a good consultant. Experience with developing quality systems is the
key requirement and, for preference, experience of quality systems to
meet BS ISO 20252 specifically, but since the market research industry is
relatively small, the number of such consultants is correspondingly
limited. However, many more consultants have BS EN ISO 9001 experience
and some of these may well be effective. In general it is best to seek
prior experience in service businesses in some way similar to market
research rather than manufacturing industry only (where BS EN ISO 9001
has had the most impact). In drawing up a short list of potential
consultants’ entries in, for example, MRS’ Research Buyers Guide, the
recommendations of other research companies which have gone through
the process and certification bodies58 (see later) are all useful sources. The
decision of which consultant to choose needs to take into account fees
charged, timetable proposed, claimed experience and not least personal
chemistry.
Apart from people and their knowledge, other resources needed, beyond
normal office facilities, are few. One to mention though, and it should be
obvious, is a copy of BS ISO 20252. The UK version of this publication,
BS ISO 20252, is published by the British Standards Institution (BSI). In
other countries the national standards body is likely to be the source. To
58
In the UK, accredited certification bodies are not allowed to act as consultants. They can
and do offer training related to quality systems and may be willing to recommend
individual consultants.

implement BS ISO 20252, it is essential for at least the project leader to

read and understand the standard. The cost in the UK is about £150 and
if this cannot be afforded there is no point going any further; other costs
entailed will be far greater.
The cost of implementing and being assessed to BS ISO 20252 will vary
from company to company depending on size, level of internal expertise
and the degree of change in working practice required to meet the
standard. The main budget headings include consultants’ fees (if used),
the cost of internal staff time (salary and opportunity costs), any extra
process costs (e.g. interviewer validation) and the charges made by
certification bodies for assessment. Quotations can be sought from
consultants and certification bodies but internal costs need to be
estimated, usually after carrying out an initial audit.
What changes are needed: the initial audit

To comply with BS ISO 20252 a company needs to meet all requirements
set out in the standard except where the particular process is not carried
out by the company. For example, if a research company only carries out
qualitative research, none of Clause 6 of the standard – data processing
and management – will be relevant. Meeting requirements also means
consistently meeting them, not just when project pressure is not too
great. Given that the requirements of BS ISO 20252 cover around 30
pages of dense text this may seem daunting. However, as a general rule,
most research companies will find that they already meet the large
majority of the requirements, most of the time. This is not surprising
since the standard, by and large, sets down common and recognized
existing good practice of the international research industry. The first real
step of the project is, therefore, to analyze in detail processes,
documentation, etc., and identify which requirements are already met
and, correspondingly, where changes will be needed in working practice
– in other words to carry out an initial audit.
The initial audit can be carried out by a consultant who will (and should)
be familiar with the standard and its requirements. He or she, however,
will not know the company and its staff and so will need to work closely
with the project leader. Let us assume, however, that the initial audit is
carried out by the internal project leader without external help; the first
task will be a close study of the text of BS ISO 20252 and an
understanding of its requirements. The project leader can also decide at
an early stage, perhaps after some discussion with other staff, which
clauses and requirements do not apply because the processes are not
relevant to the work of the company. It is then a matter of going
through all remaining clauses and parts of clauses and establishing
whether current working practice is or is not in line with the standard.

What changes are needed: the initial audit
The requirements of the standard are set out in Clauses 3 to 7 and these
can be covered in order,59 in each case establishing whether the
requirement is currently being met (or not). For example, Subclause
4.1.3.8 of BS ISO 20252 requires proposals to describe whether individual
steps are to be subcontracted. Does the company subcontract any
processes and if so is this stated in proposals? However, a simple yes or
no response is often not adequate. Firstly, even small companies vary in
how a particular process is carried out between departments or between
individual members of staff. One research director, for example, may
mention subcontracting when drafting proposals whereas another may
not. Similarly, subcontracting of data collection may be included but data
entry subcontracting not. Also, a particular requirement may be
considered good practice and generally met but not in every project.
Clearly, therefore, in carrying out the initial audit, the project leader
needs to do more than sit at a desk and think; establishing current
practice requires discussions and questions across the company. We
suggest that conformity to each requirement is shown on a simple scale
of at least three points: the requirement is currently met in nearly all
cases, met in about half the cases and met rarely or not at all, plus,
where the requirement is not relevant to the processes carried out.
Obviously this needs recording e.g. ‘pencil’ notes on a copy of the
standard or on a specially designed checklist.
When all clauses of the standard have been adequately audited, the
result can be reviewed. Many of the requirements will be found to be
met wholly or at least generally. Some will not be met and changes in
existing working practice will be needed for conformity. What is involved
and the implications of making the necessary changes will vary. In some
cases the change will not be hard to make and entail no real costs; e.g.
Subclause 4.1 lists what should be included in proposals (where relevant
to the project) and adding some of these to the current format should
not be difficult. Other changes may involve significant extra costs. If
phone interviewing, for example, is carried out but without interviewer
validation to the level required (Subclause 5.4.3) then making this change
will have cost implications. The senior staff of the company must,
therefore, consider carefully the feasibility of making the required
changes and on this basis decide whether to go to the next stage or not.
For most companies the changes to be made will not be too daunting or
costly.
59
Clauses 4 to 7 roughly follow the logic of the research process from proposals through to
reporting. Clause 3, though, is more concerned with managing the quality system itself and
it is unlikely that many of these will be already met and they may be a bit more difficult to
understand.

Management system documentation

Requirements of the standard
It is not enough to just meet the process requirements of BS ISO 20252
(i.e. as per Clauses 4 to 7); a documented ‘research process management
system’ has to be in place. This is specified in Subclause 3.1.1:
The research service provider shall apply a research process

management system which covers all the requirements of this
International Standard, including scope of the system in relation to
the research service provider’s business.
In other words the system has to cover all requirements relevant to the
processes undertaken by the organization.
The research process management system shall be a documented

system …
Procedures, instructions and methods required for completing the

different tasks in accordance with the requirements of this
International Standard shall be documented, implemented,
monitored, maintained and auditable.
The documented system needs to include all the process steps relevant to
BS ISO 20252 requirements. Additional processes not covered by the
standard can also be usefully included in the quality system. The need to
send clients invoices, for example, may be sensibly included. However, as
we discuss shortly, making the system over-detailed and covering
additional areas produces its own problems.
For a company implementing BS ISO 20252 for the first time, the
documentation should describe current working practice except where
changes are required to meet BS ISO 20252 requirements, as identified in
the initial audit. The requirement that the system is ‘auditable’ means
that it should be possible to establish, by means of some sort of record,
that the system and specific parts of it are followed by the company after
the event e.g. if a client is required to ‘sign off’ a questionnaire, can we
establish at a later point that this has actually been done? The need for
an auditable system needs to be constantly considered when drafting
documents but there are dangers of ending up with a complex system of
records whose only purpose is to satisfy the quality system, with no other
practical use. Such a system will be a burden to all and in the end will be
a failure. Ideally, any record of following the system should also provide
practical benefits in management. Wherever possible the record should
also be part of documents that are produced as part of the research
processes themselves. A questionnaire, for example, is its own record;
there is no need for a record to show a questionnaire has been
produced.

Designing a documented system: an example

Describing what a documented system should cover and look like is
difficult in the abstract. As an example, at the end of this chapter there is
a complete set of quality procedures,60 and these will be used to discuss
and illustrate how to prepare appropriate documentation. These are for
an imaginary company – ARSP – although it is in fact a composite of
several real companies that one of the authors has advised and is, in this
sense, real. It will be used to discuss and illustrate what is required in
documenting a quality system but should be read with the following
caveats. The system is appropriate to the unique company and reflects
what that company does (processes carried out) and how it works.
Another company will need a different and similarly unique system. The
style and format is adequate for the company’s use and adequate to
satisfy independent assessors, but a quite different style and format could
be equally appropriate. The documentation could be more extensive and
detailed (given the activities of the company it is about the minimum
required) but there is much to be said for keeping the documentation as
simple and as short as possible. Remember that procedures are things
that actually have to be done and done all the time. For context, the
imaginary ARSP is fairly small (about 20 professional research staff) and
carries out ‘full service’ research, from brief to final reporting, but with
in-house quantitative data collection limited to phone interviewing.
Qualitative research is also carried out. (See also Appendix 2 of the
specimen procedures.)
Example of an introduction
Like all good documents, ARSP procedures have an introduction. It is less
than a page but is more than just formal significance; it covers some
requirements of BS ISO 20252.
Subclause 3.1.2 of BS ISO 20252:2012 states that the senior management

of the organization shall take responsibility for:
…committing to quality of client service (including a statement of

quality policy) appropriate to the purpose of the organization.
The first paragraph of the introduction of ARSP procedures is such a

statement of policy. Of course the policy could be a much a longer and
wordy statement but the company considers that this is adequate (if the
company considers it adequate the assessors cannot really disagree!).
The next three paragraphs of the introduction are formal statements

referring to BS ISO 20252:2012 (and meeting its requirements) and a list
60
From here on ‘procedures’ will be used for a documented quality system; other terms such
as quality manual can equally be used.

of the main sections of the document. The next paragraph confirms

overall responsibility for the system including the role of the quality
manager. The stated need for all staff to work to procedures implies that
they are not mere guidance; a commitment is implied.
The final paragraph of the introduction relates to the need, set out in
BS ISO 20252:2012 (Subclause 3.3.2), for control of key documents:
Documents shall be subject to a version control procedure that allows

the current version to be clearly identified.
If the procedures are to be followed by all staff, it is important that

everyone uses the same version of the document (procedures can and will
be revised over time). In the case of ARSP the only current version is the
one available on the computer network which is a ‘read-only’ file that
cannot be easily changed. Each page of the document also has a ‘footer’
including the version number and date of issue and this sort of control is
appropriate for all documents, including those produced in the research
process.
The point of this laboured analysis of the short introduction to ARSP

procedures is to show how some of the requirements of BS ISO 20252 can
be covered in very limited documentation.
Example of procedure format

After the introduction, the document goes to actual procedures relating
to the processes of the company. Procedures 1 to 7 concern the research
processes carried out from initial client enquiry to final reporting and,
more or less, follow the sequence of a project. Again it should be
emphasized that these uniquely reflect how ARSP works and are not a
model that can be simply used by another company. We discuss in some
detail just the first set of procedures, which concerns receiving enquiries
from clients and responding with a proposal; in other words, getting
business.
Like the introduction, the first procedure is short, with minimum

wording. The processes up to delivering a proposal to the client are
described in one page (although with reference to an appendix) under
four sub-headings and, within each sub-heading, separate ‘bullets’. The
numbering system of the procedure is to two levels but another
approach could also be acceptable. The procedure meets the
requirements of Subclause 4.1 of BS ISO 20252 that are relevant to ARSP
(e.g. ARSP only prepares ‘proposals’ and not ‘quotations’ so the latter is

not mentioned – this is also covered in Appendix 2). Some general points
about the procedures include:
• Responsibilities for the processes are defined as well as the key

process steps e.g. research directors’ review enquiries including to
check the type of work is within the capabilities of ARSP, although
specific tasks may be delegated.
• Compliance with the procedure can be checked by records
mentioned including the ‘network project sheet’ and appropriate
filing of the proposal. Notice the reference to ‘established file
naming conventions’ – these may be defined or documented
elsewhere but it may equally be the case that they are followed
consistently without an apparent need for this.
• Document control is evident; the proposal includes the date it is
written and the author and reviewer are identified. Any revisions are
numbered.
• Some process steps are in addition to BS ISO 20252 requirements and
in particular the review of proposals by a research director, which is
not an explicit requirement of the standard but was already a
well-established practice of ARSP. In Procedure 2, similarly, invoicing
is covered which, although not an BS ISO 20252 requirement, is a
vital process step for a commercial business.
• The procedures are not just recommended good practice; they are
mandatory (although having a proposal prepared by one research
director and then reviewed by another is an exception to this rule;
making this mandatory might bruise some important egos!).
• It is implicitly assumed that procedures will be followed by staff
members who know how to do the tasks; procedures are not training
manuals although they can provide a useful framework for a new
member of staff.
The ‘network project sheet’ is one of the important ways in which

Procedure 1 can be said to be auditable. This is a spread sheet available
to all relevant staff. It is used to both determine and show the job
number allocated (the main tool for project identification, used
throughout processes), the client, staff responsible and the value of the
proposal. If the job goes ahead, the project sheet is updated as specified
in Procedure 2 and 7. The same information could of course be recorded
in other ways but ARSP found this way effective (the network sheet, in
some form, pre-dated implementation of the standard) and an important
business tool apart from meeting the requirements of BS ISO 20252. At
any one time it shows the value of proposals to date (an indicator of
future business levels), which proposals are outstanding, the value of
commissioned work (which leads to turnover levels), the work allocation
of staff and future demand on the phone unit – all vital if the business is
to be effectively managed. The network sheet is, therefore, not just a
record to get or keep BS ISO 20252. The sheet could of course go into
much more detail up to requiring a ‘sign-off’ for each process and

procedure step and such records are used by some companies

implementing BS ISO 20252. However, be warned, record sheets are only
of use if they are updated and completed and the more detail, the less
likely this is to happen or, even worse, filled in at the end of the project
just to satisfy the quality system.
In other procedures, other records are referenced and these also meet
the need for auditable procedures but in nearly all cases in procedures up
to 7, the records are either produced by the research process (e.g.
questionnaires) or serve a more general business need. Some records may
be no more than emails to colleagues, clients or subcontractors and these
are quite adequate providing they are easily locatable and retrievable.
Since the default with emails is generally filing in personal folders (which
are hard to find and access by other than the person concerned), copying
them to the job file is specified (Procedure 2.4). Incidentally, the ‘job file’
is of course a network folder and hard copies of records or physical files
are generally not needed.
As already mentioned, ARSP procedures, up to 7, more or less follow the

sequential logic of a project. Procedures 8, 9 and 10 cover what can be
called support processes: dealing with subcontractors, managing
interviewers and developing other staff, all of which are included in
BS ISO 20252 requirements.
Subcontracting is an important requirement of BS ISO 20252 – see

Subclause 3.5. It is all very well for a company to have a quality system,
meeting all relevant requirements of the standard, effectively
implemented, but if important processes are passed to subcontractors,
who do not meet the standard, how can it be truthfully claimed that the
project is conducted in accordance with BS ISO 20252? A company should
have control over the selection of subcontractors and ensure that the
relevant requirements of the standard are followed by subcontractors in
at least the projects carried out for the company (e.g. in the case of data
collection this would include validation of interviewers’ work to the level
specified in the standard). Procedure 8 describes how these requirements
of the standard are met by ARSP and in a way which is considered
effective and appropriate. In this area, the quality manager has the main
role in controlling the list of approved subcontractors (in consultation
with others) and ensuring the subcontractors meet the standard. As the
BS ISO 20252 ‘expert’ the quality manager will know what a particular
subcontractor needs to do to comply with the standard. Incidentally,
subcontractors who are assessed to BS ISO 20252 would likely be
favoured – all other considerations being equal – as suppliers to ARSP for
obvious reasons.
The only method of quantitative data collection carried out in-house by

ARSP is telephone interviewing. Procedure 9 is concerned with the
management of ARSP telephone unit interviewers, including their
recruitment, validation of work by monitoring and regular appraisal. The

procedure may reflect practice that was well established by ARSP before
implementing BS ISO 20252 and covers the relevant requirements for
telephone interviewers as per Clause 5 of the standard. However, this is
an area where some companies may need to make significant changes to
established practices and quite possibly changes with major cost
implications.
BS ISO 20252 has requirements for staff competence and training

(Subclause 3.4). Nearly all companies will have some sort of staff training
and development programme. Procedure 10.3 ensures the training and
development needs of each member of staff (apart from telephone unit
interviewers which are covered by the previous procedure) are appraised
in an annual cycle and that agreed training is delivered, with records
kept. Procedure 10 requires staff to work to codes of conduct and to be
aware of other relevant issues such as data protection requirements and
understand the quality system and how it impacts their own work.
ARSP procedures up to this point (Procedure 10) cover processes that are
likely to be familiar to all research companies regardless of whether or
not there is a formal quality system. Procedure 11, however, only makes
sense as part of a quality system and meets requirements of Clause 3 of
BS ISO 20252 that are unlikely to reflect established practice. At its heart
is the need for making process improvements and dealing with identified
problems. If Procedure 11.2 is followed with commitment, rather than
just going through the motions to meet the standard, this alone can
justify the cost and effort of implementing a formal quality system. The
philosophy reflected is that of honestly facing that mistakes and
problems happen and then seeking to understand their causes and
devising solutions to lessen the chances of reoccurrence; the aim is
preventive as well as corrective action. If a problem occurs in a project,
for example one interviewer’s work is found to be defective at the end
of the fieldwork period, something needs to be done about it there and
then (perhaps allocate the quota to another interviewer and possibly
extend the fieldwork period) and this is appropriate corrective action.
However, if nothing else is done, there is a good chance the same
problem will happen on another project. A longer term, preventive
action (e.g. re-training the interviewer) is needed as well. Procedure 11.2
provides mechanisms for addressing problems and process improvement.
As in other procedures, responsibilities are assigned and records are kept.
Again, though, we must emphasize that Procedure 11.2 is just one way of
meeting the requirements of the standard. Other ways of working could
be equally effective, meet the requirements and better fit how another
company works.
The ‘find problems and fix them’ philosophy also includes deliberately
seeking them out. Internal auditing of the quality system is a tool to do
just this and is a specific requirement of BS ISO 20252 (Subclause 3.6.3).
Procedure 11.1 is to ensure effective internal audits are carried out. More

is said about internal auditing and how to do it when we discuss

implementing the quality system. The final part of Procedure 11 is to
enable the documented quality system to be changed, if required, but in
a controlled way to ensure that there is only one current version in place
at any one time. Again, changing the system is discussed later.
Example of appendices
The final parts of the specimen procedures are two appendices. Appendix
1 is an adjunct to Procedures 1 and 7 and respectively provides checklists
of the content of proposal and reporting coverage. In each case, the
items listed are as per the BS ISO 20252 requirements in this area that are
relevant to the types of project carried out by ARSP. These lists are
needed but equally could be included as part of the two procedures and
having them here is just a matter of convenience; proposal and reporting
drafters may use the appendix as a stand-alone prompt card. Appendix 2
is not strictly a requirement of the standard. However, it serves two
purposes: to guide outside assessors and to check that the system meets
BS ISO 20252. If certification is sought (see below) the assessors will find
it useful to have a brief summary of the company and its implementation
of the standard.
The first part of Appendix 2 provides this and references an organization

chart61 of ARSP. The table following shows how the clauses of
BS ISO 20252 are covered by the procedures including which clauses are
not relevant to the company’s processes. The assessors may or may not
agree that the cross-referenced procedures meet each clause but they at
least know where to look. The table is also a very useful discipline for the
drafters of the procedures in ensuring that all requirements of the
standard are covered; the time taken in preparing it is well spent.
Producing the quality system

This section of the chapter has hopefully provided some useful guidance
on what quality system documentation should cover. But how is it to be
produced? Who is going to draft it and who is going to review and check
it? Putting pen to paper (or fingers to keyboard) is likely to be the job of
the project leader or, if used, an outside consultant. The format, style and
wording can be their responsibility, but the content needs to be based on
much wider input across the company. The initial audit will have shown
that many of the requirements of BS ISO 20252 are met by current
practice and the quality system should, where this is the case, simply
61
An organization chart is a requirement of BS ISO 20252:2012 and the document should be
controlled in the same way as procedures. Given that organizations often change it may be
convenient to have the chart separately rather than having to revise a fuller procedure
document.

Implementation: getting it all working
describe this established way of working, so that the difficulties of

implementation are minimized. It may be that in thinking about current
practice, better ways of working are suggested and incorporated into the
system. However, the best advice is to make these sorts of changes later,
once the system is working well; generally it is best not to make the
project harder than it needs to be. The system will also need to include
those changes identified through the initial audit as essential to meet the
standard. There is more than one way of doing this though; required
changes can be made and described in different ways and procedures
and which is best needs the input of those who are involved in the
processes. Drafting the quality system, therefore, requires consultation
with staff involved in all relevant processes.
How to organize cross-company consultation depends on the structure

and size of the organization. In a small company, one-to-one meetings
between the project leader or consultant and members of staff may well
be adequate. Everyone does not have to be involved (most senior staff
should be), although there are dangers in limiting it only to the
management level; often what they think is done and what is actually
done in day-to-day work are not the same. In larger companies a more
formal process might be required such as working groups meeting with
the project leader. Whatever the manner of consultation, the process
needs to be iterative with draft procedures circulated, discussed and
refined. Finally a more or less final draft will be arrived at which can be
used in implementing the quality system.

With a well drafted set of procedures, developed after cross-company
consultation, surely all that should be needed to implement the system is
to agree a start date and make sure everyone has access to the procedure
manual. It will all work perfectly from there, won’t it? No, it will not. The
procedures will be just another document, possibly glanced at, but not
acted on. More proactive implementation is essential. Do not assume that
staff, however well educated, will read and internalize quality system
procedures; face it, they may reflect drafting genius but they are not an
exciting read. Instead all staff members need to have the procedures, and
why they need to follow them, explained, with the emphasis on those
which reflect changes from established practice. As part of this, the
expected benefits of a quality system meeting BS ISO 20252 can be
explained or recapped and doubts discussed and resolved. One element
in the story can be that the system can be changed on the basis of
experience. It may become apparent that specific procedures do not work
as intended or unexpected problems are thrown up and this can be
solved by document revision. However, the message should equally be
that until they are changed, in a formal way, the procedures are to be
followed to the letter as far as practically possible. The project leader will

probably be most involved in this system launch but senior and line
managers need to be seen to be committed as well. It is not unusual for
senior staff to find personal difficulties in following a system but ‘do as I
say not what I do’ is no good.
Auditing the system

Once the system has been assumed to be working for a few weeks, the
project leader (who is now perhaps re-titled as quality manager) needs to
establish that this is, in fact, the case, and the method to use is internal
auditing. Internal auditing, as already mentioned, is a requirement of
BS ISO 20252 but even if it was not, a quality system is unlikely to
function effectively without it in some form. At its simplest, internal
auditing is no more than finding evidence that each element of the
quality system is actually followed. For example, commissions are
required (as per procedures) to be logged to the network project sheet. Is
this what happens? Is the sheet up to date? Have any projects been
started that are not on the sheet?
In a small company, the job of internal auditing may well have to be

carried out by the quality manager, though this is not ideal. Wherever
possible an internal auditor should be independent of the area of audit.
In other words auditing your own work is to be discouraged and the
reason is not so much that you may hide problems but you will not see
them at all. It is similar to why books need independent editing and
proof-reading. Even the quality manager may miss issues about his or her
role in the system. Realistically, finding auditors who are not involved at
all in the processes is often difficult. An auditor does not need to be an
expert in the processes audited (it is usually better not to be) so does not
have to be a researcher. It may be that a member of staff in areas such as
finance can carry out effective audits. Alternatively there is cross-auditing,
e.g. a small team of auditors and each audits a process they are not
involved in day-to-day. Thus a ‘research executive’ audits the telephone
and data processing departments and someone from these areas audits
proposals, planning, reporting etc. Whoever audits, however, will need to
make time available but this should not be too demanding; each audit
session should only take an hour or so at most. Auditors need some
training in the work. There are formal courses in quality auditing often
run by certification bodies (which may offer a deal as part of an
assessment programme) or if an outside consultant has been involved in
developing the system they can usually provide the training. Failing such
formal training, for a small company, the quality manager and auditors
(if separate) should be able to self-learn.

Audits need to be planned. Procedures need (of course) to cover auditing

and should indicate at least minimal frequency. The specimen procedures
include (Procedure 11.1):
The quality manager arranges internal audits of the quality system

and prepares a documented audit plan to ensure the whole system
and all processes are audited at least annually.
Auditing the whole system and all processes could be done together but
it is better to do a bit at a time, particularly when auditors and the
quality manager have to fit the time required around their ‘day jobs’.
The quality manager, therefore, needs a plan of when each element will
be audited and this should be documented in some form (outside
assessors will want to see the plan). While annual internal audits of the
whole system may be adequate once a quality system is well embedded,
this will not be sufficient to start with.
First audits need to be carried out soon after the launch date and
repeated until the system appears to be working effectively; the results
of initial audits will determine this. During early audits project records
may be limited because the projects have not gone through all or most
processes since the quality system start date, but this is not a reason to
delay auditing for too long. At the individual audit level (i.e. a session)
the auditors also need to plan what in detail they are going to cover,
where and who they will need to see. Some sort of checklist for each
audit, based on the relevant procedures, is needed. Appointments may
need to be made with the ‘auditees’ to ensure availability; unexpected
visits may not be practical (staff may be away from the office) or
desirable.
Although the auditing role can be wider, the main objective should be
comparing what the procedures say should happen with evidence of
actual practice. For example specimen Procedure 9 states:
New interviewers are monitored when carrying out their first

interviews with feedback given and documented.
Is there evidence that this happens in the phone unit? For the most
recently recruited interviewers, can the documentary evidence be
produced? Checking this will involve a meeting with the phone unit
manager or a supervisor and requesting retrieval of these records. To
some extent the evidence may be verbal but good procedures must be
auditable via the review of records of some sort and not dependent on
hearsay alone. In the example, it may be the case that some new
interviewers were monitored but others not, so there is a need for at
least a sample to be taken. This is even more the case when the evidence
sought is in project records. Practice is required to be consistent across
projects so establishing whether, for example, questionnaires are
‘signed-off’ requires looking at a number of job files and the sample size

may be determined to some extent by initial findings. Conformity to

procedures in, say, the first three or four project records examined may
give adequate confidence, but if there is a problem with one of the first
three, more should be examined to determine the level of
‘non-conformity’.
The audit role is strictly one of fact finding; non-compliance with

procedures may be found but deciding how to deal with the problems is
a separate stage and usually one for the quality manager. Sometimes this
may be difficult, particularly where the auditor and quality manager is
the same person. Audits and any problems found need to be recorded as
a written report and the main headings should include who carried out
the audit and when, procedures and parts of the company or staff
audited, project records examined where relevant (job numbers), and
what the audit found out. The convention is to describe the results under
‘findings’ and ‘observations’. Findings are cases of non-compliance or
‘non-conformities’ with the procedures, e.g. ‘no monitoring records for
three interviewers on first assignment’.
Audits are not meant to blame anyone but if a reason is given for
non-compliance this can usefully be noted as well; such comments may
well suggest the solution, e.g. ‘I did not know I was meant to monitor
first job’. Observations are other notes the auditor may think it useful to
make – perhaps problems less than non-conformities, e.g. it took a long
time to find the monitoring records. The audit report can then be
discussed with the quality manager who should have the responsibility, in
consultation with other staff including those directly affected, of finding
solutions to any problems. The emphasis needs to be longer term; finding
a solution so that the problem is less likely to occur again. Incidentally, if
the non-compliance found is a failure to complete a required record,
making the record retrospectively serves no purpose at all.
Revising procedures
The two main ways of solving non-compliance with procedures is training
and procedure revision. Particularly when the quality system is first
implemented, the most common reason for staff not following
procedures is simply that they did not know what they were supposed to
do and this will be found regardless of whatever effort is made in
launching and communicating the system. Often the fact that the
non-compliance has been identified, through an audit, effectively solves
the problem; the staff members concerned now know what they should
be doing. Sometimes, though, a more formal training session will be
needed and as a last resort the line manager will need to be pushed to
ensure future conformity.
Less commonly, the reason for non-compliance may be that the particular
procedure does not work well or is ambiguous. In this case the solution is

a revision of the procedure and it needs to be agreed in consultation

with staff members who are particularly affected. Of course, the
requirements of BS ISO 20252 must still be met so there are constraints
on how procedures can be changed. The quality manager, or perhaps an
outside consultant, should be able to decide what changes are possible
within the standard. The action taken as a result of the findings from
internal audits need to be recorded; perhaps as part of the audit report.
As will be mentioned later, audit reports are regarded as important
records by outside assessors. It is also important to establish that action
taken is effective and the problem has been solved. Often this requires a
re-audit after time has been allowed for the action to take effect.
In the first few months after implementing the quality system, the need
to revise some procedures should be expected, a matter of learning
through experience. Changes in the documented quality system, however,
need to be made in a formal way to ensure that at any one time there is
only one current version to be followed. The specimen procedures
include (in 11.3) a procedure for revision, including authority to make the
change (the procedure could just as well allow the quality manager to
make the revision without reference to anyone else) and a record of
what was changed (a copy of the superseded procedure). All staff
members need to be made aware that procedures have been revised; do
not just hope they will find this out otherwise.
An organization may also decide to revise the quality system apart from
dealing with problems found in internal audits. BS ISO 20252 requires
problem solving and a quality improvement process far wider than
reacting to internal audits. There is a need to consider process-related
problems (e.g. interviewer error rates), issues raised by clients (complaints
or as feedback from monitoring client satisfaction) or deficiencies in the
work of subcontractors. All positive suggestions for improvement should
also be considered. A process is needed for this as well, such as 11.2 of
the specimen procedures.
A final comment on implementation is that the quality manager has an

important role in ensuring the system works. The specific responsibilities
of the quality manager can vary. Organizing internal audits and seeing
that problems arising are dealt with, safekeeping the documented quality
system and liaising with any outside assessors are all core roles. Tasks such
as controlling sub-contractors and ensuring staff understand the quality
system may be the responsibility of the quality manager or others. The
quality manager needs some expertise and if, as is common, he or she
was also the project manager for developing and implementing the
system this will have been acquired on the job with or without outside
assistance. Formal training courses are also possible. Except in the largest
companies, the quality manager’s role is likely to be part time and
fitted-in with other responsibilities. The job cannot be neglected, though;
the system will not work without active management.


A company can implement a quality system to meet BS ISO 20252
without being independently assessed. In fact, ISO rules state that, like
any ISO standard, compliance with BS ISO 20252 can be established by
first, second or third-party assessment. First-party assessment is by
self-certification, i.e. simply stating ‘ARSP meets all relevant requirements
of BS ISO 20252’. Second-party assessment is by a customer or client and
we will not discuss this further, although this may be relevant for a
research company ensuring compliance of its subcontractors. Third-party
or independent assessment is what is usually meant when a company
‘has’ BS ISO 20252 or a similar standard. Having gone to the trouble of
implementing the system, companies usually wish there to be no
question about compliance and this is provided by independent
assessment; self-certification may be all very well, but will others believe
it?
Choosing the assessor

For product standards and specifications, self-certification is actually very
common; it is asserted that the product conforms to a standard and the
buyer can probably confirm this by examining or physically testing it. For
a quality management or process system such as BS ISO 20252 or
BS EN ISO 9001 this is not the case; reports prepared by BS ISO 20252
companies do not look any different. Also it has to be said that without
the discipline of visits by outside assessors (which are charged), the
system can be neglected and all the work developing it wasted.
If outside and independent assessors are to be used, their assessment has

to be reputable, authoritative and credible to clients. Anyone can set up
as assessors of BS ISO 20252 and perhaps market themselves as an
‘easy-pass’. Therefore, ideally, assessors who are themselves assessed in
some way should be used. In the UK and elsewhere there are
accreditation schemes for assessment bodies and accreditation62
specifically for them to carry out BS ISO 20252 assessment and
certification. Currently two such UK bodies (MQA and SGS) are accredited
by the United Kingdom Accreditation Scheme (UKAS) for BS ISO 20252.
This accreditation is given after UKAS determines, through its own
assessment, that the body conforms both to general standards for
assessment and certification and to a specification for BS ISO 20252
assessment developed by the Market Research Quality Standards
62
To clear up some confusion of terminology, it is assessment bodies which are accredited
and they assess companies by auditing to standards such as BS ISO 20252:2012 and then
certify or register the companies. Although it is a commonly-used phrase, companies are
not accredited; they are certified.

Association (MRQSA).63 The costs of accreditation are met by the

assessment bodies and no doubt passed on to companies using their
services. Obviously, the two accredited UK companies are experienced in
working with market research companies but in other countries finding
an assessment body with appropriate experience may be harder.
A company seeking assessment to BS ISO 20252 should contact the

assessment bodies at a fairly early stage; before the quality system is
implemented or even when the project is being considered. The bodies
will be happy to have an initial meeting and although they cannot
provide consultancy, useful advice is likely to be given. Assessment bodies
of course charge for their services and having at least an estimate may be
part of deciding whether to implement the standard. In any case, there is
no harm in having alternate quotations before deciding which body to
choose although other factors should be considered as well and especially
judgements on the service to be offered.
Assessment bodies may be awarding or withholding certification but they

are suppliers of a business service. Before providing a quotation, the
assessment body will obviously need information about company size and
structure, location of offices and the range of market research services
offered in-house or subcontracted, which will determine which
requirements of the standard apply. Before assessment, the company is
required to have implemented its quality system for a minimum period –
agreed with the assessment body – to allow time for an adequate body
of evidence to be available and for research projects to have been carried
to the quality system.
The assessment process

The initial assessment (there are subsequent ones – see below) is in two
parts. Firstly, the assessors will check the quality system documentation to
determine whether it covers the requirements of BS ISO 20252. This may
be done at the research company’s offices or at a distance. The suggested
addition of a documented cross reference between standard and
procedures, as per Appendix 2 of the specimen procedures, makes this
task easier. As a result of this stage the assessment body may raise issues.
It may be that some requirements are not considered to be fully or
adequately met and changes may be needed. If the development of the
quality system has been thorough, such queries should be fairly easy to
resolve including by minor revision of procedures. Also, whether or not a
particular requirement of the standard is adequately met is not always a
black and white matter and assessment bodies will be willing to argue
and debate a point, although their greater knowledge of standards and
assessment puts them at an advantage.
63
The general standards for assessment are BS ISO/IEC 17021:2011, and ISO/IEC Guide
28:2004.

Once the quality system documentation is assessed to be adequate, the

second and more major part of the process is an assessment audit.
Assessment auditing is no different, in principle, to internal auditing;
evidence, and particularly documentary evidence, of compliance with the
quality system and BS ISO 20252 is sought. The whole quality system and
all requirements of the standard and all parts of the company involved in
relevant processes are included in the audit. For UK assessments, some
guidelines on the time to be spent auditing and the depth of sampling of
project records are defined in the assessment and certification
specification, and should be much the same regardless of which
assessment body does the work. While documentary records are the main
type of evidence, assessors expect to talk to staff in their own working
areas and they may be questioned on how they work and, therefore,
their knowledge of quality system procedures (reciting them
parrot-fashion, though, is not expected). As part of the audit, assessors
will want to establish that internal auditing has been carried out and
that problems found in this way have been addressed; it is not a problem
if internal auditing has raised issues providing they have been followed
through to a positive solution.
Assessment bodies have a positive attitude. They are looking for evidence
of compliance so that certification can be awarded. They are not there to
find problems. However, in a full assessment audit, it is likely that
instances of non-compliance with the company’s own system and
BS ISO 20252 will be found. No one and no company is perfect.
Non-compliances are classed, by assessors, as either ‘major’ or ‘minor’
non-conformities. A major non-conformity is where a significant part of
the quality system and standard is not met (e.g. interviewer validation is
not done to the required level) and where this is the case the assessment
body will be unable to certify the company until the problem is solved
and this will normally require an additional audit visit (with extra
charges). Really, no company should be ‘failed’ in this way; adequate
internal auditing should have identified the problem and action should
have been taken before outside assessment. Minor non-conformities are,
for example, specific omissions in following procedures in isolated cases,
and in an assessment a number of such may well be found. Unless such
minor non-conformities are so numerous as to be judged collectively as a
major non-conformity, the assessors will still recommend certification
providing the company agrees to solve the problems before the next
routine assessment visit. This normally involves the formality of having a
‘corrective action request’ document raised by the assessor and at the
next visit, resolution of this will be the first thing checked. Give or take
any minor non-conformities and corrective action requests, the assessor
will at the end of the audit indicate whether or not certification will be
recommended; in the positive case the actual certificate will be issued
shortly by a senior officer of the assessment body.

Assessment to other standards
As already intimated, assessment is not a once-off ordeal (like the driving

test) but is, effectively, an annual process. Within a year of the initial
assessment, there will be a ‘surveillance’ visit; in principle carried out in
just the same way but taking less time (about half) and probably only
covering some parts of the system and processes. Less frequently (e.g.
every three years) a fuller assessment will be carried out; more or less in
the same depth as the initial one. Companies are expected to keep their
quality system effective and any non-conformity will be identified and
corrective action requests will be raised. As a last resort certification can
be withdrawn.
Assessment to other standards

This chapter has been about BS ISO 20252 implementation and
assessment. However, the same approach works equally for other
standards relevant to market research. A few final words are appropriate
about BS ISO 26362 (access panels) and BS EN ISO 9001, since companies
already assessed to BS ISO 20252 can extend their certification to also
cover either or both of these standards. Note, though, we are not saying
the reverse; companies assessed to BS ISO 26362 or BS EN ISO 9001 but
not BS ISO 20252 may well have to make significant changes to include
the latter.64 Reasons for wishing to extend certification may include
greater market awareness of these other standards; BS EN ISO 9001, for
example, is recognized throughout the business world and not just
amongst market research buyers and users. Extending the assessment and
certification to these additional standards can be done at the initial
assessment, but at least for BS EN ISO 9001 it is probably more sensible to
take it in stages and have BS ISO 20252 certification in place and then
think about extending it.
Extending BS ISO 20252 certification to BS ISO 26362 is the easier case.

BS ISO 20252 includes all the substantive requirements of BS ISO 26362
and (if access panel research is carried out) assessment to BS ISO 20252
will have included all requirements of BS ISO 26362. It should, therefore,
only be a matter of requesting the assessment body to include the latter
in the scope of assessment and certification providing the assessment
body is capable and willing to certify to BS ISO 26362.
On first reading, BS EN ISO 9001 may seem to have some different

requirements to BS ISO 20252. However this is more appearance than
substance. At least for the day-to-day research processes, extending
certification to include BS EN ISO 9001 will have little or no impact. Some
changes may be needed in procedures managing the quality system and
possibly requiring a little more formality. Internal auditing, for example,
64
BS ISO 20252:2012 includes many requirements which are specific to market research
processes, whereas BS EN ISO 9001 is a far more general standard which is applicable to
almost any organization regardless of their business.

may need to be covered by more detailed procedures and internal

auditors may need rather more formal training. Other quality system
documentation may also need revising, including a more formal
statement of quality policy and objectives and monitoring whether these
are met. While assessment bodies are strictly not allowed to offer
consultancy, in some countries including the UK they will be willing to
offer advice to some level and probably include an audit of existing
quality system documentation to identify where changes are needed to
meet BS EN ISO 9001. The changes needed are unlikely to involve much
cost and extending the assessment to cover BS EN ISO 9001 should not
increase assessment charges by very much.

Specimen procedures
ARSP Ltd. quality system procedures
Introduction
ARSP is a UK-based research company committed to carrying out

market research to high and recognized standards including the
requirements of the market research standard BS ISO 20252, MRS
Code of Conduct and established professional practice.
This document defines the quality system procedures of ARSP.
The quality system meets the requirements of: BS ISO 20252 – see
Appendix 2.
The main headings of the procedures are:
1. Enquiries and proposals

2. Commission of projects
3. Project planning
4. Qualitative research
5. Data collection
6. Data processing
7. Reporting and project completion
8. Sub-contracting
9. Management of fieldworkers
10 Staff development
11 Quality system management
Appendix 1: Proposal and reporting checklists
Appendix 2: Implementation of BS ISO 20252
The managing director of ARSP has overall responsibility for the

implementation of the quality system and appoints a quality
manager with day-to-day responsibility for administrating the
system. Other staff have responsibilities as defined in procedures or
as delegated to them by the managing director.
All staff are required to follow the procedures relevant to the work
they carry out.

Specimen procedures
The latest and up-to-date version of these procedures is available as

a read-only file on the company computer network. No printed
copy can be assumed to be current.
1. Enquiries and proposals
1.1 Responsibilities for Enquiries
• Research directors, to whom enquiries for work are channelled,

are responsible for reviewing all enquiries and ensuring that
resources are available to undertake the work, and that an
appropriate response is provided to the client within the
required time period.
• Responsibility for progressing the enquiry may be delegated to
another research director or other member of staff who will be
appropriately briefed (including response timing).
• Meetings may be arranged with the client to discuss the
proposed work and expand any written briefing received. The
research approach may also be reviewed at internal meetings.
• If the research director considers that the enquiry cannot be
met by the Company, the client is informed.
1.2 Enquiry logging
• The research director responsible ensures the enquiry is given

the next job number and logged to the network project sheet;
this also shows the member of staff responsible and the value
of the project.
• If the agreed response time cannot be met, the client is advised
and a new delivery date agreed.
1.3 Proposal drafting
• The member of staff allocated the task drafts a proposal. The

proposal may be in various formats but meets the coverage
requirements as per the proposal checklist – see Appendix 1.
• The proposal is reviewed, before it is dispatched, by a research
director – if a research director is the author, a review by a
second senior member of staff is not mandatory but is good
practice.
• The proposal document includes the job number, the date it is
written and the names/initials of the author and the reviewer.
• A network job folder is opened and the proposal filed in it
together with any associated material. The established file
naming conventions are used.
• The member of staff with primary responsibility for the enquiry
chases-up the proposal as appropriate.

Specimen procedures
1.4 Proposal revision
• Prior to dispatch to the client, the proposal document is

over-written to incorporate changes from internal editing and
review.
• If a client requests revisions to the proposal, a revised
document is prepared and reviewed as above. The revision
status is shown on the document (revision 1, etc.).
2. Commission of projects
2.1 Meeting the proposal specification
• When a go-ahead is received from a client, the research

director responsible reviews the project and ensures that the
work can be carried out as per the proposal, including timings.
Any issues arising are discussed with the client and resolved.
• Any variations of the project from the proposal, at this point or
subsequently, including delays in timetable, are documented to
the client.
2.2 The research team
• The research director responsible allocates staff as the research

team (the team is identified on the project sheet).
• Although the research director has ultimate responsibility,
another member of staff may be the team leader responsible
for day-to-day project management.
2.3 Confirmation of commission
• The research director sends a confirmation of commission to

the client unless the client has confirmed the go-ahead in
writing (including by email). The client is also sent relevant
details of the project, as below, unless the information has
already been provided e.g. as part of the proposal:
— Timetable, including fieldwork dates, the date when the
client will be asked to approve the questionnaire, any
agreed review meetings with the client and reporting
dates.
— Any material, respondent incentives or information to be
provided by the client and, where relevant, arrangements
for storage, safekeeping and disposal of these at the end
of the project.
— The identity of any sub-contractors to be used and the
parts of the process they will undertake.
• The team leader raises the initial invoice after the confirmation
of commission is sent to the client.

Specimen procedures
• A set-up meeting with the client may be arranged and if held,

key points are agreed and documented.
• The project sheet is updated by the team leader or research
director to show commission date, initial invoice date, planned
data collection dates and planned project completion dates.
2.4 Project records
• All documentation produced as part of the project (including

copies of documents sent to clients, subcontractors, etc.), are
filed to the job folder and files are named as per the
conventions in use. Key emails concerning the project (including
internal) are copied to the job folder.
• All key documentation including questionnaires, discussion
guides and related material (e.g. prompt material), instructions
to sub-contractors and client reporting includes, as part of the
document, the job number, a version number (which is
increased as the document is revised) and date of production.
2.5 Project confidentiality
• All project details (including that the work is being carried out,
the methodology and results) are confidential to the client and
are not given to third parties without the written agreement of
the client.
• The identity of the client is not revealed to research
respondents unless this is agreed in writing with the client
and/or there is a legal obligation to do so.
• All project records are kept securely and available to ARSP staff
only as required to carry out work.
3. Project planning
3.1 Phone unit and Data Processing (DP) requirements
• The research team prepares a job requirement document for

the telephone unit and may also hold meetings to discuss the
interviewing process. The number of interviews to be treated as
pilots (see 5.1) are specified.
• A job requirement is also prepared for the DP department
(usually later than for the phone unit) including output
required and any special editing.
3.2 Sub-contractor’s specification
• Where any research process is to be sub-contracted (e.g. group

recruitment, field interviewing, on-line data collection) a
specification is prepared for the sub-contractor (see 8).

Specimen procedures
3.3 Questionnaire design
• The research team draft questionnaires, recruitment

questionnaires and discussion guides as required for any project
or review drafted by subcontractors.
• Questionnaires include standard respondent reassurances.
• Discussion guides define the use of any qualitative techniques
required and cover assurances to respondents on the use of
recordings (see 4.2).
• The team leader reviews the drafts (if he or she is not the
drafter).
• The drafts are sent to the client for approval, the client
representative is pressed for a decision if necessary and
revisions made as required.
• Questionnaires and discussion guides show the identity of the
document’s drafter and reviewer and the client representative
and date of the latter’s approval being given.
• Questionnaires and discussion guides are subject to document
control as per 2.4 including; job numbers, version numbers and
dating.
3.4 CATI scripts
• The telephone unit prepares a CATI script from the draft

questionnaire.
• The script, including routings, is reviewed and tested by the
research team and this is recorded on the script.
3.5 Interviewing samples
• The research team source the respondent sample for fieldwork

(this may be from the client as per 2.3) and otherwise plan
sampling.
• The sample is reviewed to identify any problems requiring
addressing, including relating to data protection issues, before
it is used by the phone unit or sub-contractor.
• Full records of sampling sources and selection including
respondent quotas are made and retained as project records
(see 7.1 & 7.4).
4. Qualitative research
4.1 Qualitative research planning
• Members of the research team act as group or depth

moderators and discuss the issues to be covered with other
members of the team before preparing discussion guides (see
3.3) and carrying out any fieldwork.

Specimen procedures
• Sub-contractors recruit groups, depths and book venues and

the research team prepare a full written specification of
requirements as per 3.2 and 8.
• The research team either draft the recruitment questionnaire or
approve it if drafted by the sub-contractor (see also 3.3 re:
client approval).
• The recruitment material is checked to assess that appropriate
information is supplied about arrangements for the group
including any client observation, recording, incentives, etc.
• The sub-contractor provides the moderators with written details
of the groups or depths.
• The research team advise client representatives of group details
if it has been agreed that they shall observe the groups.
4.2 Conducting groups or depths
• During the group or depth the moderator checks respondents’

identity and that they meet relevant recruitment criteria.
• Groups or depths are recorded (audio, digital and/or video) and
the recordings are labelled.
• Group or depth respondents are made aware at the start of the
group or interview that recordings are to made, the intended
use of these recordings and level of anonymity.
• Clients are asked to confirm in writing that their observations
of groups or listening to recordings (where relevant) will be
used for research purposes only.
• Moderators provide written feedback on recruitment to the
sub-contractor including any problems in matching recruitment
specification requirements and send a copy to the quality
manager.
• The research team analyse qualitative research data from
recordings. An analysis plan is prepared and documented and
working sheets and analysis notes are retained as part of
project records.
5. Data collection
5.1 Data collection by telephone
• The telephone unit prepare the CATI script, which is checked by

the research team (see 3.4).
• The telephone unit manager briefs supervisors on the
requirements for the job and members of the research team
may participate in this briefing.
• Supervisors are present in the telephone unit when
interviewing is in-process.

Specimen procedures
• A telephone project file is opened and includes all details of

the job including interviewers allocated to it and validation of
their work.
• Supervisors book telephone interviewers as required.
• All members of the allocated interviewing team are briefed by
a supervisor. There is a written record of the coverage of the
briefing.
• Supervisors monitor progress of the interviews and make
reports to other members of staff as requested.
• Initial interviews (the numbers are agreed with the research
team) are treated as pilots and the completed scripts are
reviewed by the phone unit manager and/or members of the
research team. The CATI scripts may be revised as a result of
this pilot including pre-coding open-ended questions.
• Supervisors select interviewers for monitoring; except for small
projects, the aim is to monitor some interviews of each
interviewer allocated to the project.
• Supervisors monitor interviews by listening-in and on-screen. At
least 5 per cent of all completed interviews per job are
monitored with at least 75 per cent of each interview listened
to.
• Any issues identified by the supervisor during monitoring are
discussed with interviewers and further training is arranged if
required.
• The telephone unit project file shows the number/proportion of
the total of interviews monitored, interviewers monitored,
issues identified and feedback and further training given to
interviewers.
5.2 Other data collection
• Data collection, other than qualitative or by the telephone unit,

is sub-contracted. A specification is prepared for the
subcontractor as per 3.2 and 8 and this includes, where
relevant, requirements for coding and data entry (these
processes may be carried out by separate sub-contractors).
• Where questionnaires, output specification or other key process
documents are drafted by the sub-contractor, these are
reviewed by the research team and the client (see 3.3).
• The research team reviews work carried out by sub-contractors
(see 8.4).

Specimen procedures
6. Data processing
6.1 Datafile editing
• Datafiles of CATI interviews, or as prepared by a sub-contractor,

are checked to ensure they include the correct number of
records (e.g. the number of completed interviews).
• Where required, an edit program is prepared and tested before
use.
• Records are made so that any changes made in datafiles
through editing are fully traceable and transparent; this
includes where changes are made on a case by case (manual)
basis.
• Datafiles and the edit programmes are labelled to identify the
project and the nature of the file (e.g. original and cleaned
data).
6.2 Data analysis and output
• Data Processing (DP) staff prepare an analysis programme to

meet the required specification provided by the research team.
• The output is checked by both DP staff and the research team
and revisions made as required. The checks made include:
— That all questionnaires are included;
— Source questions or other variables are identified/defined;
— Appropriate single/multi data entry by question;
— Correct filtering;
— Correct bases and sub bases and that bases are shown –
base numbers checked against interviewing records;
— ’Do not know’ and ’no response’ results are differentiated;
— If weighting is carried out, both unweighted and weighted
bases are shown correctly;
— Cross breaks are as per the specification;
— Mean scores (if used) are correctly applied and the values
used are shown;
— Labelling and titling, including abbreviations, are accurate,
meaningful and spelling is correct;
— Respondent identity is secure in tables – identification is
not shown implicitly or explicitly;
— Any statistical methods, including weighting, are described
and defined.
• Re-runs or extra analysis output are checked as above.
• All files created in data analysis are labelled by project number
and show their status (e.g. first run, re-run etc). All analysis
processes are traceable and transparent.

Specimen procedures
7. Reporting and project completion
7.1 Reporting
• The research team draft client reports and presentation

materials are prepared in the format specified in the proposal.
• All reports include: the research objectives, at least a summary
of the methodology used and a statement of compliance with
MRS Code of Conduct and BS ISO 20252.
• Where unattributed verbatims are included in reports, or in any
other way, respondents’ anonymity is safeguarded.
• The client is given copies of all questionnaires and discussion
guides (normally at an earlier stage of the project).
• The full methodology and other content, as per the reporting
checklist (Appendix 1), is documented and if not passed to the
client with the main report or presentation material, its
availability to the client is confirmed in writing (it may be
referenced in other documents, e.g. the proposal).
• Before delivery to the client, reports and presentation material
are reviewed by the research director including against the
specification for the project as per the proposal or in additional
documents.
• Reporting documents are subject to document control (job
number, version etc) and identify the authors and research
director (by initial or in full).
• Where data are passed to clients as computer files (e.g. DP
output tables), the client’s format requirements are established
and the files are labelled appropriately. Successful
transmission/delivery is confirmed.
7.2 Final invoice
• The team leader raises the final invoice.
7.3 Client satisfaction
• After completion, the team leader sends the client a

satisfaction questionnaire relating to the particular project.
• The managing director receives and reviews all completed client
satisfaction questionnaires and if required, considers any action
to be taken (see 11.2).
• Any complaint received from a client (other than via as above)
is reported to the managing director who takes appropriate
action (see 11.2).

Specimen procedures
7.4 Project records
• The project sheet is updated by the team leader to show the

reporting date and that a client satisfaction questionnaire has
been sent.
• Hard copy project records, including recordings, which are not
held as computer files, are retained for a period of 12 months.
• On disposal, hard copy records are shredded or similarly
destroyed by a contractor and this is confirmed in writing.
• Computer files relating to a project are retained on the
network file servers for a minimum of 12 months and then as
archived data for a minimum of five years.
7.5 IT security
• All files on the network servers are backed up daily with copies
retained off-site.
• Full and continually updated virus protection and firewall
programmes are in place – all files in the network are scanned
for viruses.
• The computer network, including hardware and software, is
supported and maintained and these arrangements are
documented.
8. Sub-contracting
8.1 Preferred sub-contractors
• In consultation with research directors, the quality manager

prepares and makes available a list of preferred sub-contractors
for all research processes including group recruitment, field
interviewing, on-line data collection and data preparation and
data entry.
• A preferred sub-contractor may be removed from the list by the
quality manager, in consultation with research directors, and at
least annually the quality manager and research director review
the list.
• Only preferred sub-contractors are used except where the client
specifically nominates another sub-contractor.
8.2 Compliance of sub-contractors with BS ISO 20252
• If certified to BS ISO 20252, preferred sub-contractors provide

to the quality manager written evidence of such certification
and its continuing validity.
• If not certified as above, preferred sub-contractors confirm in
writing to the quality manager that all relevant processes will
be carried out in compliance with requirements of

Specimen procedures
BS ISO 20252, including all required validation of interviews,

interviewer training and verification of data preparation and
data entry plus any legal assurances required (e.g. data
protection requirements).
• The above confirmation will be in sufficient detail to ensure all
specific requirements of the standard are to be met.
8.3 Sub-contractor specification
• For each project the sub-contractor is given a specification in

sufficient detail to carry out the work (see 3.2).
• Before starting work for a project, the sub-contractor is
required to confirm in writing that the work will be carried out
to the specification and if not certified to BS ISO 20252, that all
processes will be carried out in conformity with this standard
and that related records (e.g. of validation) will be made
available to the research team on request.
8.4 Review of sub-contractor project work
• The research team will, on completion, review the work for the
project of the sub-contractor.
• The team leader decides whether to request access to the
sub-contractor’s project records to confirm compliance with
BS ISO 20252.
• Any significant problems with the sub-contractor’s work are
reported in writing to the quality manager. The project director
considers the need for any immediate action.
9. Management of telephone unit interviewers
• Potential telephone interviewers complete an application form.

References may be requested and taken and these are
documented.
• All new telephone interviewers receive a minimum training of
six hours duration, using standard material, and delivered by
the telephone unit manager or a supervisor.
• A training evaluation form is completed by the trainer and
agreed with the interviewer.
• Telephone interviewers have access to a telephone interviewers’
manual, which is updated as required.
• New interviewers are monitored when carrying out their first
interviews with feedback given and documented (see 5.1).
• Telephone interviewers are appraised by a supervisor twice per
year. This appraisal includes additional monitoring of interviews
and review of routine monitoring. An appraisal report is
prepared by the supervisor and signed by the interviewer.
• Full records of the recruitment, training, projects allocated and
appraisal processes for each interviewer working for the

Specimen procedures
company are maintained and up-dated. These records may

include both hard copy and computer file documents.
10. Staff development
10.1 Codes of conduct
• Up-to-date copies of the MRS Code of Conduct and relevant

guidelines to the MRS Code are available to all staff on the
company network.
• Research directors or other senior managers are responsible for
ensuring that staff understand the requirements of and their
responsibilities to the above Code and of data protection
legislation applicable to their work.
10.2 Quality system requirements
• Research directors/other senior managers are responsible for

ensuring staff understand the requirements of the quality
system and the need to work to quality system procedures as
relevant to their roles.
10.3 Staff training
• The training and development needs of all staff other than

interviewers (see 9) and including telephone unit supervisors
are appraised on appointment and, thereafter at least annually,
by research directors or other senior managers. Records are
made of these appraisals including whether previously agreed
training has been delivered.
• An individual training plan is agreed and documented and
research directors/other managers are responsible for ensuring
that agreed training is delivered within defined periods.
• Training and development may be delivered in various ways
including internal and outside courses, coaching and
mentoring.
11. Quality system management
11.1 Internal auditing of the quality system
• The quality manager selects members of staff to carry out

audits and ensures they are trained for the role.
• The quality manager arranges internal audits of the quality
system and prepares a documented audit plan to ensure the
whole system and all processes are audited at least annually.
• The results of the above audits are reported in writing and
discussed with the quality manager.

Specimen procedures
11.2 Dealing with problems and quality improvement
• As per procedures, various checks, validations and verifications

are carried out as part of the research process. Any problems
identified and actions taken are noted as part of project
records.
• Where, as a result of the above, data or output is removed
from further processing its status is identified in the file or on
the documents.
• At meetings of senior staff, held at least every quarter and
chaired by the managing director, general quality problems or
opportunities for improvement are discussed and these may be
identified from:
— The analysis of outcomes of checks, validations and
verifications as above;
— Internal audits as per 11.1;
— External audits;
— Customer satisfaction questionnaires or complaints as per
7.3 and 7.4;
— From reviews of sub-contractors (see 8.4);
— From other company meetings;
— On the initiative of any member of staff.
• Responsibility for implementing agreed improvement action is
assigned and the member of staff given the responsibility
reports progress.
• Action may include staff training, changes to processes and
revision of quality system procedures. The aim is to achieve
long-term improvement rather than just immediate correction
of such as faulty work.
• The discussion of the above and proposed action is minuted.
• The quality manager reviews the above minutes and reports of
action as above and ensures that any action taken is effective
including through additional auditing.
11.3 Revision of procedures and other quality system documents
• The managing director authorizes any changes to quality

system procedures.
• Changes are made to the read-only file of procedures (see
introduction) and the version numbering and date of issue
amended by the quality manager.
• As a record of revisions, superseded versions of procedures are
retained as a separate and clearly identified file.
• Staff are advised of the changes made to procedures and
encouraged to read the new version.
• Telephone unit department manuals are also revised in a
controlled way with version numbers and the date of issue
shown and the current version file clearly identified.

Specimen procedures
• The quality manager ensures that the copies of the MRS Code
of Conduct and associated guidelines, accessible on the
network, are up to date.
Appendix 1: Proposal and reporting checklists
Proposal coverage checklist – Procedure 1.3
Proposals shall include where relevant the following:
a) The research objectives and background as agreed with the

client;
b) Main areas of data to be obtained – topic area of the
questionnaires or discussion guide;
c) Quantitative methodologies to be used including:
— Types of respondents,
— Sampling sources and selection methods,
— Level of data accuracy expected,
— Approximate length of interviews,
— Data collection method – e.g. CATI telephone unit,
— Data processing methods including weighting or other
statistical techniques;
d) Qualitative methodologies to be used including:
— Types of respondents, method of recruitment and
incentives to be offered,
— Number and make-up of groups or depth interviews,
— Planned duration of groups or depths,
— Types of venue to be used – e.g. studio, in-home and
recording arrangements;
e) Which process steps, if any, are to be sub-contracted;
f) Materials or services to be provided by the client;
g) Progress meetings and participation of the client – e.g. review
of questionnaires;
h) Type of reporting and reporting documentation to be provided
including availability of full methodology;
i) Project timings;
j) Project costs including fixed and any variable charges and VAT
rate;
k) Research director responsible and other members of the
research team;
l) Compliance of all work with the MRS Code of Conduct,
requirements of data protection legislation and BS ISO 20252;
m) ARSP standard commercial terms and conditions of contract.

Specimen procedures
Reporting checklist – Procedure 7.1
A full and complete record will be prepared of the research in

sufficient detail to allow the project to be replicated at a later date.
This record shall include where relevant the following information:
a) The research objectives as per the proposal and whether these

have been met in full;
b) Processes carried out by ARSP or by identified sub-contractors;
c) Copies of all questionnaires or discussion guides and associated
material;
d) Types of respondents covered;
e) Source and methods of sampling, respondent selection and
recruitment;
f) Data collection method including mode (telephone etc), dates,
locations or venues of interviews or groups and incentives given
to respondents;
g) Interview validation;
h) Data processing methods including weighting;
i) Achieved sample profile, variations from the planned sample
and response or strike rates from respondent contacts;
j) Levels of accuracy of the data and any other limitations on
their use.
Appendix 2: Implementation of BS ISO 20252
ARSP Ltd has implemented a quality system meeting the

requirements of BS ISO 20252: 2012. These requirements are met
through following the quality system procedures. The managing
director has overall responsibility for compliance with the quality
system.
ARSP operates from offices in London and Birmingham.
All research projects are the responsibility of a research director.

The managing director may act as a project director. Other qualified
members of staff are allocated to the research team for a project.
See also the current organization chart.
ARSP carries out full service research projects for clients. Most
projects are quantitative in scope with data collection by telephone
interviews. Qualitative research is also carried out but with
respondent recruitment sub-contracted. Quantitative data
collection, other than by telephone interviewing, is also
sub-contracted.

Specimen procedures
Comments on specific requirements of BS ISO 20252 and their

implementation by ARSP are as below.
Comments on BS ISO 20252:2012 clauses and cross-references to

ARSP Quality System (QS) Procedures
BS ISO 20252 Comment

Clause
3.1.1/2 See QS Procedures: Introduction, 10,
11 and Appendix 2
3.1.3 See QS Procedures: Introduction
and Appendix 2
3.2 See QS Procedure: 2.5
3.3 See QS Procedures: Introduction, 2.4
and others
3.4 See QS Procedures: 9 and 10
3.5 See QS Procedure: 8
4.1 ARSP prepares proposals rather
than quotations. See QS Procedure:
1
4.3 See QS Procedures: 2 and 3
4.4 See QS Procedures: 3.3, 4.1, 5.1 and
8. Note: To date projects have not
required translation of
questionnaires.
4.5 See QS Procedures: 1.3 and 3.5
4.6 See QS Procedures: all
4.7 See QS Procedures 2.3 and 2.5
4.8 See QS Procedures: 7.1 and
Appendix 1. Oral presentations are
carried out by relevant research
teams. To date, research has not
been published.
4.9 See QS Procedures: 7.1 and 7.4
5.1/5.2 See QS Procedures: 5, 9 and10.
Note: In-house data collection is by
phone and qualitative research is
carried out by the research teams.
Other types of data collection are
sub-contracted.

Specimen procedures
5.3 See QS Procedures: 3.3 and 5.1.

Notes: 1. All telephone interviews
are by CATI with automatic
recording of interviewer, date and
duration. 2. Interviewer training
covers contact with children and
other vulnerable respondents. If an
interview sample is expected to
include a significant proportion of
such respondents, requirements are
also covered in the project briefing.
5.4 See QS Procedure: 5.1. Note:
Validation of telephone interviews
is by monitoring; no other form of
interviewing is carried out in-house
5.5 See QS Procedure: 4. Note:
Qualitative recruitment is
sub-contracted: see QS Procedure 8
5.6 Any self-completion data collection
(including on-line) is
sub-contracted; see QS Procedure 8.
5.7/8 To date ARSP has not carried out
research covered by these clauses.
5.9 See QS Procedures: 4, 5.1, 7.1, 7.4
and Appendix 1
6.2 Any hard copy data entry is carried
out by sub-contractors. See QS
Procedure 8.
6.3 Data entry is by CATI with proven
software. See QS Procedure 6.1
6.4 In-house CATI interviewing normally
is restricted to closed-response
interviewing after the pilot stage –
QS Procedure 5.1. Sub-contractors
may carry out coding – QS
Procedures 3.2 and 8
6.6/7 See QS Procedure: 6
6.9 See QS Procedure: 6 and 7.4
7 See QS Procedures: 7.1 and
Appendix 1

Bibliography and sources
Also listed are other sources consulted but which are not referenced.
Alt, M and Brighton, M (1981) ‘Analysing data: or telling stories?’, Journal

of the Market Research Society 23(4)
American Association for Public Opinion Research (2010) AAPOR Report

on Online Panels, AAPOR, Deerfield
Belson, A (1986) Validity in Survey Research, Gower, London
Bendell, T, Kelly, J, Merry, T and Sims, F (1993) Quality: Measuring and

Monitoring, Century Business, London
Bourne, M and Franco-Santos, M (2010) Investors in People, Managerial

Capabilities and Performance, A Study Conducted by the Centre for
Business Performance, Cranfield School of Management, Cranfield
Brace, I (2008) Questionnaire Design: How to Plan, Structure and Write

Survey Material for Effective Market Research (Market Research in
Practice), Kogan Page Limited, London
British Quality Foundation (2010), The EFQM Excellence Model 2010, BQF,
London
Brown, S (1995) ‘Postmodern marketing research’, Journal of the Market

Research Society 37(3)
BS 7911:2003, Specification for organizations conducting market research
BS EN ISO 9000:2005, Quality management systems – Fundamentals and

vocabulary
BS EN ISO 9001:2008, Quality management systems – Requirements
BS EN ISO 9004:2009, Managing for the sustained success of an

organization – A quality management approach
BS EN ISO 19011:2011, Guidelines for auditing management systems
BS ISO 20252:2012, Market, opinion and social research – Vocabulary and

service requirements

BS ISO 26362:2009, Access panels in market, opinion and social research –

Vocabulary and service requirements
BS ISO/IEC 17021:2011, Conformity assessment – Requirements for bodies

providing audit and certification of management systems
BS ISO/IEC 27001:2005, Information technology – Security techniques

Information security management systems – Requirements
Callingham, M and Smith, G (1994) ‘Quality comes to the market research

world’, Journal of the Market Research Society, 36(4)
Callingham, M and Smith, G (1994) ‘Quality comes to the market research

world’, The Journal of the Market Research Society, 36(4)
Carroll, G and Boutall, T (2010) Guide to Developing National

Occupational Standards, UK Commission for Employment and Skills,
Wath-Upon-Dearne
Collins, M (1989) ‘Concepts of accuracy in market research’, Seminal

proceedings, Market Research Society, London
Collins, M (1989) ‘Concepts of accuracy in market research’, Seminar

proceedings, Market Research Society, London
Colwell, J (1990) ‘Qualitative research’, Journal of the Market Research

Society 32(1)
Comley, P (2008), ‘Online Research Communities’, International Journal of

Market Research, 50 (5)
Crosby, P (1979) Quality Is Free, McGraw-Hill, New York
Deming, W E (1989) Out of Crisis, MIT Centre, Cambridge, MA
Dexter A (2007) ‘The charm of magpies’, MRS Conference Papers 2007
Dexter, A and Page, A (2008) ‘The business of insight: the price of

everything, and the value of nothing?’, Conference proceedings, Market
Research Society
EFAMRO (2010) Guidance Note: Data Protection in the EU, EFAMRO, The
Hague
EFAMRO (2010) Guidance Note: Privacy and Electronic Communications in

Research, EFAMRO, The Hague
ESOMAR (2007) ICC/ESOMAR International Code on Market and Social

Research, ESOMAR, Amsterdam

ESOMAR (2008) 26 Questions to Help Research Buyers of Online Samples,

ESOMAR, Amsterdam
ESOMAR (2011) Global Market Research 2011: ESOMAR Industry Report,

ESOMAR, Amsterdam
Friedman, A and Mason, J (2004) The Professionalisation of UK

Professional Associations: Governance, Management and Member
Relations, PARN, Bristol
Gabriel, C (1990) ‘The validity of qualitative research’, Journal of the

Market Research Society, 32(4)
Gehling, R (2008), ‘The power of co-creation – how the changing role of

the consumer impacts market research’, Conference proceedings,
ESOMAR
Hammersley, M (1993) Social Research – Philosophy and Practice, Sage

Publications, London
Illich, I et al (1977) Disabling Professions, Marion Boyars, London
Inside Research (2009) ‘U.S. Online MR Gains Drop’ 20(1), 11-134.
Institute for Employment Studies (2008), ‘Does IIP add value to business?’,
Employment Studies newsletter No 7, IES, Brighton, UK
Investors in People (2011) It’s Your Choice: the Standards and Beyond –
The Investors in People Framework, The Stationary Office, London
Investors in People Unit (2006) Investors in People Standard, The

Stationery Office, London
IQCS (2009), Interviewer Quality Control Scheme: Minimum Standards for

Market Research Data Collection, IQCS, London
Ishikawa, K (1985) What Is Quality control?, Prentice-Hall, Englewood

Cliffs, NJ
ISO 10004:2010, Quality management – Customer satisfaction – Guidelines

for monitoring and measuring
ISO/EC Guide 28:2004, Conformity assessment – Guidance on a third-party

certification system for products
ISO/EC Guide 65:1996, General requirements for bodies operating product

certification systems
Jackson, P and Ashton, D (1993) Implementing Quality through BS 5750,

Kogan Page, London

Jackson, P and Ashton, D (1995a) Achieving BS EN ISO 9000, Kogan Page,

London
Jackson, P and Ashton, D (1995b) Managing a Quality System Using

BS EN ISO 9000, Kogan Page, London
Juran, J M (1989) Juran on Leadership for Quality, Free Press, New York
Kearon, J and Earls, M (2009) ‘Me-to-we research’, Conference

proceedings, ESOMAR
Kiecker, P and Nelson, J E (1996) ‘Do interviewers follow telephone survey

instructions?’, Journal of the Market Research Society, 38(2)
King Taylor, L (1992) Total Customer Service, Century Business, London
Koehn (1994) The Ground of Professional Ethics, Routledge, London
Lawrence, R J (1982) ‘To hypothesize or not to hypothesize’, Journal of

the Market Research Society, 24(4)
Marketing and Sales Standards Setting Body (2006) ‘National

Occupational Standards for Marketing/Marketing Communications and
Sales: a world class benchmark for professional practice’ CD (no longer in
production)
May, J (2006) ‘Ladders, Stars and Triangles: Old and New Theory for the
Practice of Public Participation’, International Journal of Market Research,
48(3)
Mc Givern, Y (2009), The Practice of Market Research – Third Edition,

Pearson Education Limited, Harlow
Moore, W E (1970) The Professions: Roles and Rules, Russell Sage, New
York
MRS (1983) ‘Guide to Good Coding Practice’, MRS, London
MRS (1996) ‘Qualitative Recruitment. Report of the Industry Working

Party’, Journal of the Market Research Society, 38(2)
MRS (2003) The Data Protection Act 1998 & Market Research: Guidance
for MRS Members, MRS, London
MRS (2010) Code of Conduct, MRS, London
MRS (2011) Regulations for Using Research Techniques for Non-Research

Purposes, MRS, London
MRS (2011) The Annual Survey of the UK Market Research Industry 2010,
MRS, London

MRS (2011) The Research Buyer’s Guide 2011, MRS, London
O’Brien, J (1987) ‘Two answers are better than one’, Journal of the
Ohm, P (2010) ‘Broken Promises of Privacy: Responding to the Surprising

Failure of Anonymization’, UCLA Law Review, 57
O’Shaughnessy, J and Holbrook, M B (1988) ‘Understanding consumer

behaviour: the linguistic turn’, Journal of the Market Research Society,
30(2)
Owen, D (1991) ‘Every coding is another encoding’, Journal of the Market

Research Society, 33(4)
Page, B (2005) ‘Viewpoint: Quality Control’, International Journal of

Market Research, 47(4)
PARN (2011) Financial & Operating Ratios for Professional Bodies –

Summary Report: 2009/2010, PARN, Bristol
Phillips, A (2005), ‘Viewpoint: Maintaining Research Standards’,

International Journal of Market Research 47(5)
Pirsig R (1999) Zen and the art of motorcycle maintenance – 25th

Anniversary Edition, Vintage, London
Popper, K (1959) The Logic of Scientific Discovery, Hutchinson and Co,

London
Poynter, R (2010) ‘A Taxonomy of New MR’, Conference proceedings,

Market Research Society
Poynter, R and Lawrence, G (2007) ‘Insight 2.0: new media, new rules,
new insight’, Conference proceedings, ESOMAR
Rittenhurg, T L and Murdock, G W (1994) ‘The pros and cons of certifying

marketing researchers’, Marketing Research, 6(2)
Robson, S and Hedges, A (1993) ‘Analysis and interpretation of

qualitative findings’, Journal of the Market Research Society, 35(1)
Rothman, J and Mitchell, D (1989) ‘Statisticians can be creative too’,

Journal of the Market Research Society, 31(4)
Rust, R T and Cooil, B (1994) ‘Reliability measures for qualitative data’,

Journal of Marketing Research, 31
Sayer, A (1992) Method in Social Science – A Realist Approach, Routledge,

London

Smith, D and Dexter, A (1994) ‘Quality in market research: hard

frameworks soft problems’, Journal of the Market Research Society, 36(2)
Sykes, W (1990) ‘Validity and reliability in market research’, Journal of the

TNS Opinion and Social (2011), Special Eurobarometer 359 Report –

Attitudes on Data Protection and Electronic Identity in the European
Union (June 2011), TNS Opinion and Social, Brussels


213
Licensed Copy: Mr. Universiti Teknologi Malaysia User, Universiti Teknologi Malaysia, 14/08/2012 02:51, Uncontrolled Copy, (c) The Br
If you found this book useful, you may also want to buy:
Complaints Management: Turning Negatives into Positives

Michael Hill
Complaints should be an easy way for your customers to express an opinion about
your products and services. This book has been designed to help any company or
organization that wants to improve the effectiveness of customer complaint
processes and systems and also adhere to the principles of the complaints
management standard, ISO 10002:2004. The book draws on case studies from a
variety of sectors, such as financial services, healthcare, not-for-profit, utilities, and
travel, to present the reader with opportunities to reflect on good practice, while
realizing the underlying benefits of the standard. There are quotes and statistics that
can support an internal business case, as well as practical checklists for handling
complaints, analysing root causes, and conducting system reviews. The book also
references the benefits that can be delivered from external dispute resolution
schemes and how ISO 10003:2007 delivers a standard for improving the
effectiveness of such schemes.
The book is available in two formats – text alone and text with standards.
A5 paperback · ISBN 978 0 580 71876 2/ (with standards) 978 0 580 76553 7
· 247/344pp · £40.00 or £160 with standards (member)/ £288 (non-member)
· BSI order reference: BIP 2211/2222
For more details see http://shop.bsigroup.com/bip2211
Forthcoming
Raising the Standard for Collaboration
David Hawkins
The aim of the book is to provide an introduction to the concepts and value of
collaboration and the standard BS 11000-1:2010, Collaborative business
relationships. A framework specification. It focuses on providing a first step to
adoption and implementation and is a low cost introduction before venturing into the
purchase of the standard and implementation investment.
A5 paperback · ISBN 978 0 580 78737 9 · 256pp · £55.00

· BSI order reference: BIP 2225
For more details see http://shop.bsigroup.com/bip2225

Bip 2206-2012

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Bip 2206-2012

Uploaded by

Copyright:

Available Formats

Licensed Copy: Mr.

From Theory to Practice

From Theory to Practice

Debrah Harding and Peter Jackson

This second edition published in the UK in 2012

© The British Standards Institution 2012

First edition published by Kogan Page in 1997

Typeset in Frutiger by Letterpart Limited, www.letterpart.com

2 Market research as an industry 11

3 Quality and truth 27

4 Quality and the profession 43

5 Quality and business process 63

Quality in Market Research v

6 Quality and legislation 81

7 Quality management and BS EN ISO 9001 99

8 Process standards for interviewers 115

9 International quality standards for research 131

10 People quality – Investors in People 149

11 Quality system development, implementation and

Specimen procedures 189

Bibliography and sources 207

vi Quality in Market Research

Exactly when the first sample survey occurred is a matter on which

Whilst quantitative survey research has its theoretical basis in statistics

The importance of the process quality around these components has

It is a timely addition to the professional survey researchers tool box and

Quality in Market Research vii

In this introduction we discuss the continuing importance of quality issues

Quality has always been a concern of market research,1 although its

The research sector has a long history of supporting and devising

Quality in Market Research 1

Interviewer Quality Control Scheme in the UK), national and now

We hope the book will assist those involved in research to understand

Market research today

• Research is in demand. Since the turn of the 21st century research

2 Quality in Market Research

Market research today

of which 34 per cent post opinions about products and brands.5

Quality in Market Research 3

The quality movement

Relative quality, then, is a matter of meeting requirements with an

In the literature of quality and in all manner of quality programmes, it is

4 Quality in Market Research

aspirations and supply ability of brand owners. Quality as excellence

Quality as excellence also has a dynamic dimension. Relative quality is all

Professionalism is also built around notions of quality as excellence. The

Standards are a well established means of ensuring that relevant needs

Quality in Market Research 5

The point of product standards for technical and manufacturing processes

While nearly all published technical standards relate to physical products,

The increasing globalization of research, and the need for consistent

6 Quality in Market Research

research – vocabulary and service requirements. Chapter 9 will look at the

There are also process or capability standards, developed to improve

Once standards are established within a field of activity the question of

A solution potentially attractive to both sides is, therefore, third-party or

Quality in Market Research 7

testing etc) to what lies behind the attainment of consistent quality

With an increasing amount of legislation impacting on the practice of

Purpose is about commitment to quality within the organization and this

8 Quality in Market Research

emphasis on commitment and how to truly achieve it. It is the process

Process concerns systems and methodologies to deliver quality