See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/278847816

The evaluation of social media marketing campaigns: an exploratory study

Conference Paper · July 2012

CITATIONS READS

0 5,059

3 authors:

Brendan James Keegan Jennifer Rowley

National University of Ireland, Maynooth Manchester Metropolitan University
39 PUBLICATIONS   257 CITATIONS    402 PUBLICATIONS   24,710 CITATIONS   

SEE PROFILE SEE PROFILE

Jane Tonge
Manchester Metropolitan University
12 PUBLICATIONS   163 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

PhD: Digital Placemaking - Health & Wellbeing - Green & Blue Space View project

Artificial Intelligence and Machine Learning in Marketing View project

All content following this page was uploaded by Brendan James Keegan on 21 June 2015.

The user has requested enhancement of the downloaded file.

 The evaluation of social media marketing campaigns: an exploratory study

Everything that can be counted does not necessarily count

and everything that counts, cannot necessarily be counted.
(Attributed to Albert Einstein: Theoretical Physicist)

Introduction
This paper reports an exploratory small-scale study into the approaches adopted by digital marketing
agencies to evaluate the effect or impact of social media (SM) marketing campaigns. The Web 2.0
digital arena offers many possibilities for the evaluation of marketing campaigns, since it is relatively
easy to generate a wide range of ‘performance statistics’ through web analytical software. However,
the researcher’s experience in the sector suggests that there is variation in practice and that in some
cases, whilst evaluation reports are created, the data is not used to inform decision making regarding
further campaigns. In addition, whilst there are a number of premium, subscription based automated
software systems (e.g. Radian6, Brandwatch) which attempt to measure and evaluate SM marketing
campaigns, widespread dissatisfaction amongst practitioners and commercial commentators (E-
Consultancy, 2010) and excessive licence fees (that are a significant barrier to smaller agencies)
suggest that the industry would benefit from further investigation into SM evaluation methods.

Aims and Objectives

It is the aim of this study to explore and understand the approaches to the evaluation of SM marketing
employed by digital marketing agencies. In order to gain an understanding into approaches of this
nature, the research objectives for this project are as follows:
1. To profile the evaluation processes and techniques used by digital marketing agencies to
evaluate SM campaigns
2. To gain an understanding of the drivers that affect the selection and design of these processes
3. To gather practitioners’ perspectives on the relative merits and limitations of the approaches
that they have adopted
4. To explore the relationship between agency size and the adopted evaluation approach
5. To offer recommendations to enhance practice regarding the evaluation of SM campaigns

This paper next reviews the previous research and theoretical context for this study. This is followed
by an outline of the research methods, and, an indicative summary of potential outcomes from the
research.

Literature Review:
The development of SM platforms and their inherent user interaction was comprehensively explored
by Boyd & Ellison (2008) who concluded that consumers of SM can “construct a public or semi‐
public profile within a bounded system, articulate a list of other users with whom they share a
connection, and view and traverse their list of connections and those made by others within the
system” (p.1). Seen to be a derivative of what is classed as the second generation of the internet,
commonly referred to as Web2.0 (O'Reilly & Constantinides, 2009), SM is rapidly asserting itself as
an important modern cultural facet. Wide availability of high speed broadband provision coupled with
a multitude of web-enabled mobile devices means that the average UK web user spends more time on
Facebook than any other site with 16.2% of all time spent online (ComScore, 2011)

Since their customers and audiences are on SM, marketers have developed techniques to actively
promote or raise awareness of their brands on these platforms, extending relationship marketing into
this new arena (Constantinides & Fountain, 2008).

Despite this growing interest in SM, in general, and amongst marketers, in particular, and SM’s
potential as an effective communications medium, research in this area is still limited. Such research
as has been conducted mostly takes a consumer or societal perspective, covering topics such as the

1
inclusion of SM in the classroom (Granitz & Koernig, 2011), and the influence of SM on teenagers
(Kelly, Kerr, & Drennan 2010). With a few notable exceptions, there is limited work on the digital
marketing practitioners’ approach to SM campaigns.

The study of the evaluation of SM campaigns fits within the wider arena of the measurement and
evaluation of the effectiveness of digital marketing and advertising. It is commonplace for digital
practitioners to quantify metrics for campaign performance, typically via a penultimate action of the
campaign cycle. Within the digital sphere, search engine advertising systems (e.g. Google Analytics)
in conjunction with html tracking can manipulate immense volumes of data. However, SM platforms
generate data of a distinctly different format to that associated with display advertising; highly
semantic data is produced by the activity on SM platforms generally bound loosely by the term
‘engagement’ (Mangold & Faulds, 2009).

Two key works on SM marketing evaluation are those of Murdough (2010) who looked at SM
measurement as a construct, and Cruz & Fill (2008) who explored the evaluation of viral marketing
through practitioner enquiry. Both inform the research approach adopted in this study. Murdough’s
(2010) model of the social media measurement process (Figure 1) embed measurement in a strategic
model of SM marketing. Cruz & Fill’s (2008) methodological approach in their work on viral
marketing has also helped to shape the approach adopted in this study; they identified performance
metrics from the academic literature and employing semi-structured interviews with selected industry
practitioners. Finally, Broom et al (2011) concluding their investigation into work on the relative
return on investment of SM, asserts that research covering practitioner approaches to SM marketing is
sparse.

Figure 1: Social Media Measurement Process

Source: (Murdough, 2010)

Research Design
This paper presents a small-scale exploratory study in an area in which there has previously been
limited research. Accordingly, a qualitative approach has been adopted using semi-structured
interviews (Bryman & Bell, 2011) in order to explore practitioner attitudes, experiences and opinions
when dealing with SM. The questions were open-ended thus not limiting the respondents/interviewees
choice of answers (Gubrium & Holstein, 2002). The purpose was to provide a setting where the
interviewer and interviewees could discuss the topic in depth (Creswell, 2007). Questions covered the
following four main areas:

2
 1. Key informants professional profile, including: role, career, involvement with SM campaigns
and their evaluation.
2. Agency profile, including: size, range of clients and campaigns, profile of experience with
SM campaigns (both stand alone and multi-channel)
3. Profile of two or three key recent SM campaigns, including: client, sector, objectives,
message, media, audience, budget, and negotiation, development and design processes.
4. SM campaign evaluation, including: the methods of evaluation used, and the decision
processes that determine the measurement approach and tools.

All questions were supplemented with a number of prompts, which the interviewer used as necessary
to ensure coverage of key themes. In addition, the opportunity was left open to explore unanticipated
topics and themes.

Participants were all practitioners in marketing agencies involved with SM campaigns. This was
consistent with the broad aim of this study to understand practitioner’s decisions and approaches to
evaluation. One practitioner was selected as a key informant for each of a number of agencies.
Agencies were selected on the basis of their involvement in SM campaigns, and their accessibility to
the research. Within each agency a key informant was identified who had either practical or
managerial involvement with SM campaigns. Many of these agencies are relatively small, so it is
often the case that there is typically one practitioner who is well-versed in the SM area. Identification
of agencies and contact with key informants was facilitated by the researcher’s existing industry
relationships through a local trade association, although it should be noted that agencies and
practitioners alike will be anonymised in subsequent iterations of the research.

Interviews were conducted either in person in the participants work place, or via Skype alongside its
partner recording software, Pamela. Given the deadline dominated working nature of the participants,
it was important to be flexible as to the interview arrangements to suit the needs of the participants. It
is estimated that between thirty minutes to an hour shall be needed for interviews.

Potential outcomes
It is anticipated that by the time of the AM Conference the data for this study will have been collected
and analysis completed, and that it will be possible to present some key findings. The data collected
will reveal the range of different approaches adopted by the case study digital marketing agencies in
their evaluation of SM campaigns, and provide some context and rationale for the approaches
adopted. It is anticipated that practices will range from very basic evaluation, to complex systematised
approaches to evaluation, and that the involvement of clients in specifying evaluation will also vary.
The interviews will also offer insights into the impact of client base, involvement with clients, the
campaign process, decision making processes and the availability of software reporting tools.

3
 References

Boyd, D. M., & Ellison, N. B. (2008). Social Network Sites: Definition, History, and Scholarship.
Journal of Computer-Mediated Communication, 13(1), 210-230.

Broom, D., McCann, M., Bromby, M., & Barlow, A. (2011). Return on Investment: What Literature
Exists on the Use of Social Media and ROI? Journal of Database Marketing.

Bryman, A., & Bell, E. (2011). Business Research Methods (3rd ed.). New York, New York, USA:
Oxford University Press.

ComScore Data Mine. (2011). More time spent on Facebook than any other UK site.[Online]
Accessed January, 12/1/12, from http://www.comscoredatamine.com/2011/07/more-time-is-
spent-on-facebook-than-any-other-uk-site/

Constantinides, E. (2009). Social Media/Web 2.0 as Marketing Parameter: An Introduction.

Proceedings of 8th International Congress Marketing Trends.

Constantinides, Efthymios, & Fountain, S. J. (2008). Web 2.0: Conceptual foundations and marketing
issues. Journal of Direct, Data and Digital Marketing Practice, 9(3), 231–244. Nature
Publishing Group.

Creswell, J. W. (2007). Qualitative inquiry & research design : choosing among five approaches, 2nd
ed. London: SAGE.

Cruz, D., & Fill, C. (2008). Evaluating viral marketing: isolating the key criteria. Marketing
Intelligence & Planning, 26(7), 743-758.

E-Consultancy. (2010). Social Media and Online Brand Monitoring Trends Briefing Social Media
and Online Brand Monitoring Trends Briefing. Statistics. [Online] Accessed January, 12/1/12
http://econsultancy.com/uk/reports/social-media-and-online-brand-monitoring-trends-briefing

Granitz, N & Koernig, S. K. (2011). Web 2.0 and Marketing Education: Explanations and
Experiential Applications. Journal of Marketing Education. 33(1), 57-72.

Gubrium, J. H. J. (2002). The handbook of interview research: context and method. Sage Publications
Ltd.

Kelly, L., Kerr, G., & Drennan, J. (2010). Avoidance of Advertising in Social Networking Sites: The
Teenage Perspective. Journal of Interactive Advertising, 10(2), 16-28.

Mangold, W. G., & Faulds, D. J. (2009). Social media: The new hybrid element of the promotion mix.
Business Horizons, 52(4), 357-365.

Murdough, C. (2010). SOCIAL MEDIA MEASUREMENT: IT’S NOT IMPOSSIBLE. Journal of

Interactive Advertising, 10(1), 94-100.

View publication stats