Industrial Marketing Management 86 (2020) 16–29

Contents lists available at ScienceDirect

Industrial Marketing Management

journal homepage: www.elsevier.com/locate/indmarman

You have not been archiving emails for no reason! Using big data analytics T
to cluster B2B interest in products and services and link clusters to financial
performance
Yang Yanga, Eric W.K. See-Tob, Savvas Papagiannidisc,
⁎

a
Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong
b
Department of Computing and Decision Sciences, Faculty of Business, Lingnan University, Tuen Mun, Hong Kong
c
Business School, Newcastle University, 5 Barack Road, Newcastle upon Tyne NE1 4SE, UK

ARTICLE INFO ABSTRACT

Keywords: The potential of big data analytics when it comes to gaining business insights, such as market trends and con-
Big data sumer preferences, has captured the interest of both scholars and business practitioners. However, the extant
Analytics literature has so far provided limited empirical evidence to demonstrate how big data analytics can create
Emails business value. To address this research gap, this paper followed a novel big data analytical approach that
Internal
involved analysing email archives about product/services demand clusters in a B2B setting. We analysed 621 k
Market research
Sales
emails exchanged between 2009 and 2018. We identified a number of discussion clusters that were considered
proxies for the interest buyers expressed in the products/services on offer. These clusters and associated dis-
cussion trends were linked to the company's revenues and financial performance, showing good predictive
power. In doing this, we have demonstrated how widely available data, such as emails, which all companies
have, can be used to underpin new methods for the early identification and monitoring of product demand
trends, informing marketing strategies.

1. Introduction
For practitioners, and especially marketers, the environment has
been rapidly changing due to the unprecedented volume, velocity, and
variety of data available from consumers, competitors, and partners
(John Walker, 2014). The speed with which information is generated
requires different data management and faster market analyses than
traditional market analytics can handle. Emerging literature has aimed
at understanding the value of big data analytics in exploiting business
insights (Gordon & Mohammad, 2012; Mithas, Lee, Earley, Murugesan,
& Djavanshir, 2013) and the implications these can have. For example,
Wiersema (2013) believed that the potential power of big data tech-
niques can make current B2B models obsolete. Lilien (2016) also agreed
on the importance of data driven decisions in the B2B field, highlighting
the need to harness the potential of big data and analytics. Companies
that perform well against the competition have been found to rely more
on big data analytics (Asay, 2014). Scholars have recognised that big
data analytics can lead to sustainable competitive advantage by iden-
tifying underlying business insights (Erevelles, Fukawa, & Swayne,
2016; Lycett, 2013). Despite the above, traditional market analytical
techniques and tools are still the dominant way of gaining business
insights (Xu, Frankwick, & Ramirez, 2016). Unlike big data analytics,
which aim to analyse massive volumes of data in real time, traditional
marketing analytics focus mainly on improving key performance in-
dicators such as market share, customer relationship, and revenues
(Sathi, 2014). It is true that traditional marketing analytics can still
provide useful business insights (Rusetski, 2014). Still, they can often
be limiting when it comes to capturing information and making sense of
large and unstructured data, which may result in unreliable or biased
decision-making (Xu et al., 2016). With the increasing volume and
complexity of business information, the data-driven decision-making
process is expected to play an ever increasingly important role, with big
data analytics being the underpinning tool to support this evolution
(Gustke, 2013).
Due to the novelty of big data analytics in the B2B context, studies
assessing its effect and performance are scarce. Most of the studies so-
lely focus on the development of conceptual frameworks to understand
the impact of big data analytics on marketing activities (Erevelles et al.,
2016; Gunasekaran et al., 2017; Xu et al., 2016). Such limited de-
monstrations of big data analytics may also be due to the difficulties in

⁎
Corresponding author.
E-mail address: savvas.papagiannidis@ncl.ac.uk (S. Papagiannidis).

https://doi.org/10.1016/j.indmarman.2019.01.016
Received 30 July 2018; Received in revised form 28 December 2018; Accepted 25 January 2019
Available online 01 February 2019
0019-8501/ © 2019 Elsevier Inc. All rights reserved.
Y. Yang et al.
collecting data from diverse information sources (Fan, Lau, & Zhao,
2015). Only a few studies have presented applications of big data
techniques in the B2B context. For example, Bohanec, Borstnar, and
Robnik-Sikonja (2015) developed machine learning models to under-
stand B2B better sales forecasting. Similarly, D'Haen and Van den Poel
(2013) presented an iterative three-phased automated machine
learning model to help acquire clients in a B2B environment. It is worth
noting that typically such studies consider data acquired from outside
data providers or collected from the Internet (Bohanec et al., 2015;
D'Haen & Van den Poel, 2013; Lackman, 2007). Despite the increasing
importance and relatively higher accessibility of data generated in
companies' internal environments (McAfee, Brynjolfsson, Davenport,
Patil, & Barton, 2012), it is not often the case that such data sets are
utilised. The increasing availability of electronic information generated
by daily operations, such as emails, trading records or consumer re-
views has created an accessible data pool for management and provides
several business opportunities. Identifying an approach to extract the
underlying knowledge contained in heterogeneous information sources
is a difficult, but worthwhile, challenge for both marketing strategists
and scholars. Part of the challenge relates to dealing with information
overload (Eppler & Mengis, 2004). In the B2B context, as business and
marketing related data become larger and more inexplicable, the lim-
ited processing capacity of decision-makers causes bottle-necks in de-
ciphering and interpreting a complicated environment (Gordon &
Mohammad, 2012). This problem is widely recognised for its adverse
effect, causing loss of productivity or creating stress, in turn reducing
the performance of individuals and organisations (Eppler & Mengis,
2004). Considering the high complexity and dynamics of B2B markets,
the negative influence of information overload can be amplified by the
complexity of decision-making (Edmunds & Morris, 2000). Thus, there
is a pressing need to present information in a manageable and easy to
comprehend manner.
Given the above, the objective of this paper is threefold. First, we
provide empirical evidence in the form of a novel big data methodology
of how big data analytics can be leveraged in the B2B context to obtain
marketing insights from a new vantage point. The presented approach
can help transform data that is often considered redundant into input
for marketing and managerial decision-making. We apply our metho-
dology to the email archives of Company X, a third-party assurance firm
that specialises in testing, inspection and certification services. Second,
given that the major barrier to applying big data analytics in the B2B
environment is the great difficulty of collecting and storing data, we
considered an internal data stream to address this challenge. Instead of
acquiring costly data from the external environment, we propose uti-
lising the most conventional, but often unexploited, information source
inside a company: business emails. Finally, the identified clusters of
interest are then related to financial performance, showcasing the po-
tential that the information extracted can have when it comes to pro-
ducing tangible results for organisations. Systematising the collection,
analysis and presentation can offer marketing managers a demand/
performance forecasting tool that complements their existing arsenal.
2. Literature review
2.1. Big Data and analytics in the B2B context
Nearly 2.5 billion gigabytes of data are produced every day
(Sivarajah, Kamal, Irani, & Weerakkody, 2017), and this number is
doubling every 40 months or so (George, Haas, & Pentland, 2014).
Laney (2001) used volume, velocity and variety, known as the 3Vs, to
capture the key dimensions of big data. Volume refers to the vast
amounts of data. Velocity refers to the speed at which new data is
generated and the speed at which it moves around. Finally, variety
refers to the range of data types and sources. Big data may involve a
number of different sources such as web pages, user-generated content,
social media, and even business data like transactional records or daily
17
Industrial Marketing Management 86 (2020) 16–29
emails (McAfee et al., 2012). This suggests that companies should look
not just externally for data, but also explore making good use of the
internal data. This is especially true given that the increasingly digitised
business environment and activities are creating new streams of data
and information that could be leveraged to gain significant advantage
by enhancing productivity and competitiveness (Manyika et al., 2011).
Manyika et al. (2011) describe big data as “the next frontier for in-
novation, competition, and productivity”. Big data can change com-
petition by transforming processes, altering corporate ecosystems, and
facilitating innovation (Brown, Chui, & Manyika, 2011). Not surpris-
ingly, and given the potential benefits that big data can result in, there
is a “widespread belief that large data sets offer a higher form of in-
telligence and knowledge that can generate insights that were pre-
viously impossible, with the aura of truth, objectivity, and accuracy”
(Boyd & Crawford, 2012, p.663). Research and business applications
should be driven by the big questions, not the big data as such
(Papagiannidis, See-to, Assimakopoulos, & Yang, 2018). The ability to
analyse massive amounts of data should not be an end in itself but,
instead, analytics should focus on applying statistics for gaining prac-
tical insight (Wang, Gunasekaran, Ngai, & Papadopoulos, 2016). Big
data can be leveraged effectively to support business decisions. For
example, it has been recognised that big data analytics can be embraced
as a disruptive technology that will reshape marketing intelligence (Fan
et al., 2015; LaValle, Lesser, Shockley, Hopkins, & Kruschwitz, 2011).
For example, McAfee et al. (2012) put forward empirical evidence to
show that companies relying more on data-driven decision-making are
performing better when it comes to productivity and profitability. Si-
milarly, Davenport (2014) presented a number of cases of organisations
drawing competitive advantages from the use of data and analytics. For
instance, Wal-Mart uses dynamic big data analytics to improve its
supply chain strategy and pricing models. In addition, LaValle et al.
(2011) found that top-performing companies use analytics five times
more than lower performers, and half of the participating companies
stated that the improvement of information and analytics was a top
priority in their organisations. However, they also argue that the pro-
cess of transforming big data analytics into business value is still far
from mature.
Three challenges have been identified in relation to B2B data ana-
lytics (Martin et al., 2015). First, establishing a long-term mechanism to
continually collect data from multiple information sources could be a
costly task. This requires investment in new approaches, applications,
and frameworks for effective data management (Fan et al., 2015). Data
in the B2B context can be less voluminous and more challenging to
collect than data from consumer sources. Secondly, there is a trend in
big data analytics towards mixing first-party, reasonably verified data,
with public and third-party external data, which has largely not been
validated and checked by any standard methodology. The reliability of
any finding based on such mixed data sources may vary widely (Kaisler,
Armour, Espinosa, & Money, 2013). Considering the importance of
basing decisions on data analytics, for instance, when it comes to in-
forming marketing strategies and campaigns, this limitation can be
critical. Thirdly, changing the experience-based decision-making cul-
ture into data-driven thinking can be challenging. As McAfee et al.
(2012) stated in their business review, “exploiting vast new flows of
information can radically improve your company's performance. But
first you'll have to change your decision-making culture”. This is par-
ticularly true when data is expensive to obtain or is in an unstructured
format. To support a data-driven culture, data analytic professionals
need to map the available data and devise a plan as to how to effec-
tively communicate this knowledge to the domain experts (Chen,
Chiang, & Storey, 2012). Additionally, using big data as essential
components of business decision-making requires new capabilities,
which includes IT infrastructure, as well as organisational and cultural
changes (Bughin, Chui, & Manyika, 2010). As such, even when com-
panies can access large data sets, they still cannot analyse them in
meaningful ways, resulting in sub-optimal outputs (Griffin et al., 2013;
Y. Yang et al.
Lilien, 2016). Tackling the challenges mentioned above can form the
main pillars of a strategy to address common issues related to big data
in B2B settings and maximise returns. The business value of big data
analytics stems from its ability to create a data-driven decision culture
in B2B companies, helping them gain competitive advantages
(Morabito, 2015; Wedel & Kannan, 2016).
2.2. Using emails for B2B marketing insights
A possible way to alleviate the data collection difficulty for big data
analytics is to reuse the data generated in the internal environment of
the companies, such as business emails. Email is the most prevalent
channel and collaboration tool in both business and social environ-
ments. Since the mid-90s, when its popularity exploded, email has
continued to be a quick and user-friendly communication and decision-
making tool within organisations (Lahiri, Mihalcea, & Lai, 2017).
Compared to other communication channels, email is cost effective
(Whittaker, Bellotti, & Moody, 2005) and provides a traceable record
via quoted messages (Tedmori, Jackson, & Bouchlaghem, 2006). A
survey conducted in 2017 (The Radicati Group, 2017) estimated that
there are over 6.3 billion email accounts. This figure is predicted to
reach 7.7 billion by 2021, i.e. a growth of > 22%. When it comes to
business communications, email continues to play a dominant role,
bringing tangible results. > 60% of consumers would prefer to be
contacted by brands via email, with 55% of companies generating >
10% of their sales by email (Bawm & Nath, 2014).
Given the importance of emails in the business environment, there is
a rich body of literature studying the impact of emails at the organi-
sational or individual level. When it comes to the business use of emails,
three main literature streams exist. The first one examines the knowl-
edge source identification (or “expert location”) (Jackson, Tedmori,
Hinde, & Bani-Hani, 2012; Paul, 2016; Tedmori et al., 2006), and aims
to identify users who hold the knowledge in an organisation. The
second type is spam email detection by using text-mining techniques
(Ajaz, Nafis, & Sharma, 2017; Basavaraju & Prabhakar, 2010). The final
one considers a variety of algorithms that have been developed to ex-
tract keywords from emails (Lahiri et al., 2017; Rose, Engel, Cramer, &
Cowley, 2010; Shah, Perez-Iratxeta, Bork, & Andrade, 2003). However,
among the studies in these streams, only a few of them attempted to use
big data analytics on emails to create practical business value in real-
world cases. For example, Li, Sen, and Zaman (2015) attempted to re-
trieve keywords from business emails, concluding that they could be
utilised for business management solutions. Although this study is re-
latively close to business information extraction, they did not demon-
strate how the extracted information can support decision-making.
From a marketing communications perspective, marketers consistently
ranked email as the single-most-effective tactic for awareness, acqui-
sition, conversion, and retention (Deal, 2014). Email can provide sev-
eral important, often unexploited, opportunities for knowledge-finding,
and the knowledge extracted from emails can be accessed and reused
directly (Janine, Ton, & Van Joolingen Wouter, 2004). However, as
organisations adopt email as the primary method of communication,
they often neglect the fact that email content contains information
about business decisions, actions and transactions (Jackson & Tedmori,
2004). These business emails become documents and records with legal
requirements and restrictions. The challenge is that due to the un-
structured format of emails and the large volumes often accumulated,
the knowledge that they contain is difficult to extract (Jackson &
Tedmori, 2004).
The business information contained in emails is critical because it
can be used for several purposes including a market trend identification
(Kok & Yih, 2009). Losing or failing to optimally exploit the underlying
business information in these unstructured texts can lead to a sig-
nificant loss of business opportunity (Duff, 1996). More importantly,
companies rarely recognise email as an essential information source to
support big data analytics (Jackson & Tedmori, 2004). In the previous
18
Industrial Marketing Management 86 (2020) 16–29
section, we have suggested a different way of addressing the difficulties
in collecting data (Chen et al., 2012; Kaisler et al., 2013). Given that
email is profoundly embedded in the daily operation and communica-
tion of a company, it can be continuously generated and added to an
ever growing and traceable repository (Campbell, Maglio, Cozzi, &
Dom, 2003). Additionally, companies do not need to invest resources to
collect email data. The automatic archiving feature of emails solves the
data collection problem in applying big data analytics and enables
dynamic and real-time analysis (Alrashed, Awadallah, & Dumais,
2018). In view of the cost-effectiveness and content richness, we sug-
gest that email should be deemed as an important data source for big
data analytics in B2B companies. A possible limitation of using email as
the data source is privacy. For example, mining emails, even when
results are displayed in aggregated forms, might expose private in-
formation (Jackson, Yates, & Orlikowski, 2007). Pre-processing to dis-
tinguish personal emails and business emails may be necessary
(Goodpaster, 2015).
Given the above and to contribute towards addressing the chal-
lenges outlined, this study will explore adopting big data techniques to
extract business information from a corporate email archive containing
B2B communications. In doing so it will showcase how to utilise the
contained business information to support management decision-
making.
3. Methodology
3.1. The data
In this study, we obtained an email archive from Company X.
Company X is a third-party assurance firm that specialises in testing,
inspection and certification services. This company has several teams of
experts to meet the testing and inspection needs of manufacturers,
traders, and buyers around the world. Over the past few years, the
number of professionals in Company X has been growing, reaching 300
testing staff with a qualification certificate, while its service scope and
service sectors are expanding rapidly (7 divisions to provide > 200
testing services). In 2017, the revenue of Company X reached 105
million HK dollars and served > 1200 customers worldwide. To meet
the increasingly diversified testing requirements of customers,
Company X has established a team of experts. This team consists of
multiple senior testing staff from different departments and is re-
sponsible for providing customers with professional and timely testing
recommendations. According to the instruction from the CEO, the ex-
pert team represents the best that Company X can offer and as such
their professional opinion is highly valued by both customers and
Company X management. In this study, the expert team provided the
necessary information about the company's products and services. They
also suggested possible topics that staff may discuss in daily operations,
based on their experience with customer interactions. Their suggested
topics were used as a valuable reference for our analysis, as it offered a
benchmark for the methodology's findings.
Company X provided an archive contained 18 GB (originally 292GB
of Microsoft PST format files that included attachments) of text emails
generated or received by employees of the Food and Pharmaceutical (F
&P) department. The archive covered a period of about 10 years (from
June 2007 to March 2018). The raw dataset contained 621,090 unique
messages (i.e. considering only the new messages sent and not previous
replies that may have been contained in the same email), all in English
and UTF8-coded. Each data point had four features: (a) sender, (b)
receiver, (c) email subject and (d) email body. As we extracted data
from the inbox of Outlook software, the emails collected from each
employee were actually the emails received by them. On average, each
employee received 15,925 emails with a standard deviation of 24,702
emails. Fig. 1 below shows the distribution of emails among the 39
employees. It should be noted that a few employees have a small
number of emails received or available (e.g., employee ID 7, 14, 18, 19,
Y. Yang et al.
Fig. 1. Distribution of incoming emails by employee.
20). This is because these employees joined the company after July
2017, thus having fewer emails compared to other colleagues. The
difference among individuals does not affect the analysis as we con-
sidered the entire department as a whole.
3.2. Email cleaning and discussion thread Identification
Before exploring the latent business topics in this archive, we ap-
plied a number of pre-processing techniques to eliminate noise and
unnecessary information. Firstly, we removed all non-business emails
from the dataset. The non-business email extraction was based on de-
tection of keywords. A “non-business dictionary” was manually estab-
lished to support a rule-based classifier (Androutsopoulos, Koutsias,
Chandrinos, & Spyropoulos, 2000). An example of the simple rule is: IF
“word in the dictionary appears in the subject” OR “word in the dic-
tionary appears in email body” THEN “the email is a non-business
email”. The dictionary contained 27 words or phrases like “fax notifi-
cation”, “automatic reply”, “alert email”, and “sick leave”. The entries
added to this dictionary were provided by the supervisors of the IT and
human resource departments. In total, we removed 76,881 non-busi-
ness emails (544,209 remained).
We then extracted the body text from each email and deleted all the
greetings, closings, sign-off, and personal information as these do not
contribute to our study's objectives. In the next step, we combined
emails with the same subject. For instance, if an employee had issued a
test inquiry and received a reply from an expert, these two emails were
combined as one data point under a subject generated by the employee.
One issue was that it was possible to have different threads sharing the
same subject title, e.g. “test enquiry”. To address this issue, we checked
the raw data and consulted with the department's expert team. As a
professional certification institution, this firm has strict regulations for
business email writing, and the use of ambiguous language in email
subjects is forbidden. General subject titles like “sales enquiry” or “test
enquiry” are seldom used in this email archive. Although we still found
a small number of emails with general subjects such as “enquiry” and
“result”, they were not expected to significantly influence our analysis.
After the above steps we ended up with a total 544,209 emails. We
then combined emails according to their subjects as planned. After this
step, the dataset contained 61,435 threads, with 8.8 emails per subject
on average (the standard deviation was 12.3).
19
Industrial Marketing Management 86 (2020) 16–29
3.3. Content pre-processing
The content pre-processing phase converted the original email texts
into a data-mining-ready structure. Firstly, we removed stop-words
from the subject line and body, based on a combined dictionary. This
870 word dictionary was built based on the complementary set of
several English stop-words lists, which involved the NLTK stop-words
list (Madnani, 2007), Fox stop-words list (Fox, 1989), GoogleStopWords
(Rose et al., 2010) and MySQL stop-word list (Moh & Bhagvat, 2012).
Following the application of stop-words, all the names (based on the
NLTK name corpus), numbers, symbols, punctuation marks and mea-
surement units were removed. As a last step, in order to avoid the data
sparsity problem of the topic (Hong & Davison, 2010), we removed
emails that had a length shorter than five words. The refined dataset
contained 43,137 threads.
3.4. Topic modelling
In extant studies of topic modelling, four techniques are well re-
cognised: Latent Semantic Analysis (LSA), Probabilistic Latent Semantic
Analysis (pLSA), Explicit Semantic Analysis (ESA) and Latent Dirichlet
Allocation (LDA). LSA is a classic technique to cluster documents and
extract latent topics. The core idea of LSA is to generate a documents-
terms matrix and use singular value decomposition (SVD) to decompose
the matrix into a separate document-topic matrix and a topic-term
matrix. The basic rationale behind LSA is that documents which share
frequently co-occurring terms will have a similar representation in the
vector space. LSA is a simple and easy to use approach, but it does have
a primary limitation: low efficiency. The size of the documents-terms
matrix can significantly increase with a large dataset. Accordingly, the
high computational complexity for the SVD process can make an ana-
lysis unfeasible. To address the efficiency problem of LSA, Hofmann
(2001) proposed pLSA use a probabilistic generative process to infer
hidden topics using posterior inference. However, one shortcoming of
pLSA is that it contains many free parameters and the number of
parameters grows linearly with the size of training documents (Hoff-
mann, 2011). Again, for a large document set, pLSA can become very
complex. LDA extends PLSA to deal with the weakness of free para-
meters. LDA uses Dirichlet priors for the document-topic and word-
topic distributions, lending itself to better generalization and much
Y. Yang et al.
fewer parameters (Chien & Wu, 2008, Chang & Chien, 2009). Finally,
ESA is not considered in our study as this technique cannot discover
latent topics. Instead it can produce labelled concepts based on an ex-
isting knowledge base (e.g., Wikipedia). Thus, ESA cannot address our
research objective: to discover the underlying information that is not
identified in the business emails.
Given the above, we employed LDA to identify the latent topics in
this dataset. LDA is a popular topic modelling technique for exploring
document collections (Blei, Ng, & Jordan, 2003). LDA topics are for-
mally a multinomial distribution over words, and by convention the top
ten words are used to identify the subject area or give an interpretation
of a topic. The generative probabilistic model of LDA can be described
as a three-step process: first, LDA assigns a set of topics (t) for each
document (d) with Dirichlet distribution, i.e., P(t| d). Second, each topic
defines some unique multinomial probability over the words (w) in the
dictionary, i.e., P(w| t). Third, the product of P(t| d) and P(w| t) is cal-
culated to find the probability of a word i existing in a given document
d and the word i belonging to the mixture word set of topic t. This
process can be formalised as:
T
P (w | d ) = P (w | t = k ) P (t = k | d )
k=1
Where P(w| t = k) is the probability of word i belonging to the word
mixture of topic k, and P(t = k| d) is the probability of selecting word i
from topic k in the document d. These two probabilities are estimated
from the training document sets using Dirichlet priors and a fixed
number of topics. Blei et al. (2003) chose to use the Gibbs sampling
approach (Darling, 2018) to iterate multiple times over each word i in
document d, and sample a new topic k for the word based on the
probability P(w| d), until the LDA model parameters converge. The
outcomes of the LDA model are a document-topic matrix and a topic-
word matrix. Leveraging on these two matrices, we can cluster docu-
ments into semantically-meaningful groupings and understand the
content. This technique matches our research purpose as the main aim
of using LDA is to explain the nature of data by producing interpretable
clusters. For example, Griffiths and Steyvers (2004) used LDA to iden-
tify scientific topics in a large set of academic documents. Similarly, we
would like to identify business activities in the email archive. Compared
to traditional topic modelling techniques like Latent Semantic Analysis
or Probabilistic Latent Semantic Analysis, the number of parameters
that LDA has to estimate only scales with the number of topics, making
it much better-suited to working with large datasets (Tirunillai & Tellis,
2014). We coded a Python (a high-level programming language for
general-purpose programming) script to implement LDA on the target
dataset. We leveraged the well-developed topic modelling package,
genism, to apply LDA on the refined dataset.
Table 1
. Topics (B2B products and services) suggested by the expert team.
Topic ID Topic label
1 Hokla enquiry, information and feedback
2 Microbiological testing
3 Heavy metal testing
4 Acid migration testing
5 Promotional events and business development
6 Fabric and textile material testing
7 Food webinars and training
8 Food inspection, audit and certification
9 Food contact material and food grade testing
10 Proprietary Chinese medicines testing and pharmaceutical services
11 Invoices and quotations
12 Customer service enquiry
13 Genetically modified organisms and genetically modified food testing
14 Shelf life determination
15 Others
20
Industrial Marketing Management 86 (2020) 16–29
4. Results and discussion
Before embarking on the analysis, several parameters for the LDA
model needed to be determined. First, the number of topics had to be
specified. We consulted with the expert team. They provided us with a
list showing the 15 topics they commonly discussed in the F&P de-
partment (Table 1). The topic selection was based on the service in-
troductory brochure of Company X (9 types of testing services). We also
asked the expert team to recall any other possible topics they commonly
discussed in daily operations (6 topics). These 15 topics were then
considered as a reference for our analysis. If our algorithm could extract
topics that matched the majority of the manually assigned labels, then
we would have had a way of validating the topic extraction accuracy of
LDA algorithm and its relevance to Company X business operations. In
this case, a perfect match between the known topics and the generated
ones may have not been desirable either, as if such a match was to be
achieved then no new business insights could be generated. Given the
above, to maintain the comparison between expert suggested topics and
machine-extracted topics from daily emails, we also fixed 15 as the
number of topics for our LDA model. This setting provided us with an
opportunity to compare the topics generated from emails vs. the topics
given by the firm experts, which may generate more insights from the
latent business topics in the emails.
Secondly, we used asymmetric Dirichlet priors in the LDA estima-
tion with α = “auto” and β = 0.01, which are common settings in the
literature (Wei & Croft, 2006). Fixing α = “auto” enables the model to
learn an asymmetric prior over words directly from the training data
(Řehůřek & Sojka, 2010). Setting asymmetric Dirichlet prior over the
document–topic distributions has substantial advantages over a sym-
metric prior, while an asymmetric prior over the topic–word distribu-
tions provides no real benefit (Wallach, Mimno, & McCallum, 2009).
Then, following the instructions of Blei et al. (2003), the learning
iterations for the training model is the number of processes that find the
optimising values of the variational parameters for each document. In
this case, we fixed the iterations at 4000, in order to ensure that LDA
model parameters can converge (the log file of gensim confirmed that
all the parameters converge within 3700 iterations). Table 2 presents
the result generated using the aforementioned settings. LDA requires
manual labelling, according to the top words distribution. Thus, we
generated labels together with the company to understand the business
nature of each email cluster, namely how emails related to the products
and services offered by Company X.
To assess the performance of the LDA model, we compared the B2B
product and service topics given by the expert team and the topics
generated by LDA. Table 3 shows the results of topic matching. We used
the statistical measures precision and recall evaluating the accuracy of
LDA. Out of 15 topics identified, 12 were true positives, that is, they
exactly match 12 of the manually assigned topics. There were three
false positives in the set of LDA topics, resulting in a precision of 80%.
Comparing the 12 true positives to the total 15 manually assigned to-
pics, the recall was also 80%. As an unsupervised technique without
any domain knowledge being needed, LDA still reached 80% accuracy,
showing an acceptable performance. Additionally, LDA identified three
underlying topics that may have been neglected by the company, which
were “promotion and marketing emails”, “application emails” and “test
advisory”. These three topics, especially promotion emails, can still
contribute to a better understanding of the business condition of the
firm. This was particularly true for the “other” cluster as in our ap-
proach a potentially “black box” is opened up for areas of interest that
at a given moment in time are not necessarily of significant interest.
Still Company X could explore opportunities among them for potential
future growth.
Below are the word-cloud visualizations of four topics (See Fig 2).
Such a visualisation technique can help the practitioners easily identify
the business information for each cluster of emails. The weight of words
Y. Yang et al. Industrial Marketing Management 86 (2020) 16–29

Table 2
. Top ten words of the 15 clusters of B2B products and services on offer.
Cluster ID Cluster label Top 10 words

1 Heavy metal tests Metal, rsi, heavy, content, cadmium, phthalate, accessory, jewelry, hair, application
2 Water and microbiology tests Water, method, aatcc, microbiology, srv, cushion, standard, legionella, iso, textile
3 Fabric related tests Fabric, plate, light, green, textile, quotation, advise, swab, black, composition
4 Application emails Application, contact, question, report, provide, advise, item, office, origin, letter
5 Invoice related emails Invoice, payment, account, industry, bank, create, pdf, term, outstand, webinar
6 Webinars and trainings Webinar, message, training, seafood, instrument, contain, transmission, industry, premium, icp
7 Customer service Client, contact, iaq, measure, provide, report, air, application, schedule, advise
8 Hokla related emails Hokla, method, compliance, apc, coli, bag, cfu, detect, special, intern
9 Others Ice, pesticide, tpc, zone, branch, item, provide, dose, opinion, addition
10 Acid migration tests Migration, acid, acet, oil, article, phthalate, ethanol, metal, heavy, simulation
11 Test advisory Feedback, order, regulation, advise, express, leachable, style, proceed, Europe, cordial
12 Marketing and promotions Invite, market, astm, promotion, course, provide, event, university, cost, nutrition
13 Audit related emails Audit, document, schedule, record, author, contact, preparation, rate, hospital, suggest
14 Material related tests Material, plastic, silicon, steel, quotation, fda, advise, stainless, coat, lid
15 Pharmaceutical related tests Pharmacopoeia, volume, bottle, tablet, republic, apc, vitamin, coli, regulation, capsule

Table 3
. Topic matching.
Matching status 15 topics given by the expert team 15 topics generated by LDA

Topics that matched Hokla enquiry, information and feedback Hokla related emails
Microbiological testing Water and microbiology tests
Heavy metal testing Heavy metal tests
Acid migration testing Acid migration tests
Fabric and textile material testing Fabric related tests
Food webinars and training Webinars and training
Food inspection, audit and certification Audit related emails
Food contact material and food grade testing Material related tests
Proprietary Chinese medicines testing and pharmaceutical services Pharmaceutical related tests
Invoices and quotations Invoice related emails
Customer service enquiry Customer service
Others Others
Topics that could not be matched Allergen identification Promotion and marketing emails
Genetically modified organisms and genetically modified food testing Application emails
Shelf life determination Test advisory

Fig. 2. Word-cloud visualizations of four topics.

in the diagrams below depends on the P(w| t), the probability of word w
belonging to topic t. This probability is given by the topic-word matrix
of the LDA model.
Leveraging on the matrix of top words, we clustered 43,137 threads
to 15 business clusters with an average 2876 emails in each. Below is
the thread distribution diagram for each cluster (See Fig 3).
4.1. Trend analysis
We extracted the sending time of the first email in each thread and
labelled it as the “starting time stamp” for the corresponding thread. If
we deem each email thread as an information stream about the firm
activity, then this time stamp can be used to record when this inner
activity started. Given that we had the time stamps and business labels

21
Y. Yang et al. Industrial Marketing Management 86 (2020) 16–29

Fig. 3. Cluster distribution.

Fig. 4. Monthly distribution of threads.

of threads, we conducted a longitudinal analysis. For each business
cluster, we counted the cumulative number of threads under each topic
from 2009 January to 2018 March. According to the monthly dis-
tribution statistics for the refined dataset (Fig. 4), for the trend analysis,
we used the data points after Jan 2009 as the data points during 2007 to
2008 were too sparse to show a trend. By plotting the thread count for
each cluster by month, we can see a business trend for this cluster.
Below is an example diagram (Fig. 5) generated based on cluster 1,
heavy metal related tests.
We employed Moving Average (MA) to smooth out short-term
fluctuations and to highlight longer-term trends and the time-window
of the averaging process is set to 6 months. This technique is widely
used in economics and signal processing, for instance, for a trend
monitor or low-pass filter. According to the chronological diagram of
heavy metal test emails, we can see that the department experienced a
significant peak season from Jan, 2016 to Jan, 2017. Conversely, for
topic 5 (Fig. 6), the invoice related tests show a continuously upward
trend from year 2014. These kinds of business trends may provide
management with a roadmap or an overview of the interest in a par-
ticular type of product/service and provide valuable insights into their
demand and serve as a proxy for their potential in the financial per-
formance of the firm.
4.2. Predictive power of business trend on revenues
As we had the department monthly revenue data from 2007 to
2017, we tested if there was any relationship between the topic trends
and revenues. Given that email is now one of the primary business

22
Y. Yang et al. Industrial Marketing Management 86 (2020) 16–29

Fig. 5. Business trend figure of topic 1 (heavy metal tests).

Fig. 6. Business trend figure of topic 5 (invoice related emails).

Table 4 productivity applications and is considered as the most frequently used

. The results of granger causality tests (1-month lag). communication tool in a firm (Ciccio & Mecella, 2013), it can serve
indirectly as an overview map for the inner activities of a firm, ranging
Cluster id Cluster label F-statistics P-value
from knowledge transfer to business development process. As we suc-
12 Marketing and promotions 32.6818 < .0001*** cessfully clustered emails, we were able further to explore which one
7 Customer service 26.5975 < .0001*** causes the most significant impact on a firm's business success (reflected
10 Acid migration tests 13.8779 .0003***
in monthly revenues).
2 Water and microbiology tests 13.0549 .0005***
3 Fabric related tests 12.9089 .0005*** The focal dependent variable “monthly revenue” was a dynamic
5 Invoice related emails 11.7687 .0009*** time-series. We tested which business trend improved short-term or
4 Application emails 10.601 .0015*** long-term forecasts of future movements in monthly revenue, and vice
14 Material related tests 10.2468 .0018*** versa. To tackle this question, we employed Granger causality. This
13 Audit related emails 9.0333 .0033***
9 Others 8.3826 .0046***
method is a well-known test for bivariate causality, and involves esti-
11 Test advisory 7.7159 .0065*** mating a linear reduced-form vector auto regression (VAR). This can be
15 Pharmaceutical related tests 7.6113 .0068*** expressed as:
8 Hokla related emails 7.331 .0079***
6 Webinars and trainings 7.0751 .012** yt = L ( )iyt 1 + L ( ) jx t 1 + t1
1 Heavy metal tests 2.7251 .1017
x t = L ( )ix t 1 + L ( ) jyt 1 + t2
Notes: **p < .05, ***p < .01.
In the maintained structure, x and y are jointly determined

23
Y. Yang et al.
Fig. 7. Sales and business trend of topic 12 (marketing and promotions) – 3, 6 and 9 months moving average comparison.
endogenous variables, and εt1and εt2 are assumed to be iid (0, σ2). L(∙)i
denotes the polynomial lag operator of order i, for instance, L
(β)iyt−1 = β1yt−1 + β2yt−2 + … + βiyt−i. To examine Granger caus-
ality between x and y, the following hypotheses are tested:
H0 : L ( ) j = 0
H1: L ( )i 0 Here we set the time-window of the averaging process to three, six and
If H0 is rejected, then we can say x provides statistically significant
information about future values of y (Cromwell, Hannan, Labys, &
Terraza, 1994). The F-statistics can also be used to rank the explanatory
power of x on y.
In the current study, the business trend of cluster j was oper-
ationalised by the dynamic number of emails of cluster j in month t,
denoted as Njt. Revenue of month i was denoted as Rt, thus we had the
VAR model to test:
Rt = L ( )iRt + L ( ) kNj, t + marketing and promotion activities of this firm were successful and
1 1 t
The polynomial lag operator was fixed at 5 at this stage, in order to
test the impact of the business trend in a maximal time window of
5 months. Below is the result of the Granger causality test (Table. 4).
The above results were based on a 1-month lag operator setting. We
rejected lag operator 2 to 5 as their F-statistics were all lower than 1-
month's. Thus, we adopted the 1-month lag as the best operator. From
the above table several insights were obtained.
Firstly, at the 0.01 significance level, most of the business clusters
showed significant predictive power on monthly sales (except heavy
metal tests and training). This finding confirms that the internal
24
Industrial Marketing Management 86 (2020) 16–29
activities, or the business model of a firm can influence the financial
performance (revenue generation). Leveraging on the big data techni-
ques, we provide empirical evidence to support this.
Secondly, we found that marketing and promotion activities (topic
12, rank 1) were very important in maintaining revenues in this B2B
context. The predictive power of topic 12 can be seen in Fig. 7 below.
nine months separately and compare the synchronism of sales and topic
trend under different time window settings. The trend diagram of this
topic indeed showed synchronism with monthly revenues. From a
short-term view (3-month moving average) to a long-term view (9-
month moving average), the trend of topic 12 illustrates synchroniza-
tion with monthly sales, showing its good predictive power and ro-
bustness across different settings of time-window values.
The F-statistics for this topic were much higher than other test-re-
lated emails, showing its influence on revenues. It also showed that the
effective. The management may consider investing more resources in
this part to stimulate further growth in revenues.
Third, general customer service (topic 7, rank 2) was much more
important than specific testing enquires. Similar to topic 12, the pre-
dictive power of topic 7 can also be seen in Fig. 8.
By checking the top word list of topic 7, we found the word “report”,
which is the final product provided to customers. This finding implies
that service quality, whether pre-sale or after-sale, was vital for the
business success of the firm, especially given the B2B context. After
discussing this finding with the expert team, we identified an alter-
native explanation for this insight, that is, that customers who had more
Y. Yang et al.
Fig. 8. Sales and business trend of topic 7 (customer service) – 3, 6 and 9 months moving average comparison.
general enquiries were not as knowledgeable and price sensitive as
customers who had specific testing enquires. Thus, transactions with
such customers were more likely to be successful.
Fourth, acid migration tests, water and microbiology tests and fabric
related tests caused relatively more impact on revenues. This finding
implies that the pre-communications of these tests were more easily
successful. In other words, the manpower and resource investment in
these tests can obtain higher ROI. A possible explanation for this finding
is that the “acid migration tests” and “water and microbiology tests” are
two market‑leading services provided by Company X. Therefore, these
two business activities are relatively more trusted by customers com-
pared to other services. We are surprised to find that the “fabric related
tests” is also identified as an influential activity. This service is not one
of the major services provided by this department, but its profitability is
significant. Thus, its commercial potential is worth further in-
vestigating.
Conversely, not all topics have good synchronism with monthly
sales. As shown in Table 4, Topic 1 (heavy metal test related emails)
causes an insignificant impact on sales (p-value = .1017). In Fig. 9
below, we can observe that the business trend of heavy metal related
tests cannot accurately predict the future monthly sales of the F&P
department. This finding implies the low effectiveness of communica-
tion about this service. This could be considered as a warning sign for
the firm. That is, resources spent on this service may not contribute to
revenue increase. The management needs to further investigates the
reason why this is the case. (See Tables 5 and 6.)
Fifth, it was surprising to find that training and webinar activities
(topic 6, rank 14) were relatively less important than other clusters.
Given that the firm is a professional certification company, this was
expected to be a core element of competitiveness. An explanation for
this finding could be that the training of employees is a considerably
25
Industrial Marketing Management 86 (2020) 16–29
long-term investment. A 1-month scrutiny window is much shorter for
us to observe its benefit. As Bartel (1995) remarked, the outcomes of
the first-year training (e.g., performance or productivity improvement)
will not appear until the second year.
To quantify the impact of business clusters, we redid the VAR
analysis following the model proposed previously:
Rt = L ( )iRt + L ( ) kNj, t +
1 1 t
Table 5 shows the result of this VAR model.
Here we ranked the clusters by the standardised beta, showing their
impact strength in descending order. As shown in the above table, the
rank of clusters was similar to the Granger causality test. Given the
coefficient beta, we can quantitatively interpret the impact of business
clusters on revenues. Taking topic 12 as an example, and assuming the
other variables (e.g., sales of last month) held constant, with an increase
of one standard deviation in marketing and promotions related emails,
revenues rose 0.466 standard deviations. More generally, in terms of
unstandardised beta, with one email increase of the marketing and
promotions cluster, the revenue will increase by $4280 HK dollars. Note
that we could not compare unstandardised betas directly to rank their
importance, as the email scales for each cluster are different.
Trend analysis can provide strategists and marketers with a more
intuitive understanding of the internal operation situation and emer-
ging market trends. As emails are dynamically updated through the
daily operation of firms, it is possible and feasible for us to capture the
short-term information about market movements, which can be valu-
able to support future decision making. For example, if we observe that
the number of emails about “fabric related tests” is rapidly increasing,
then management should consider whether it is necessary to invest
more resources (staff or facilities) to meet the possible surge in the
capability needs of fabric testing. Also, the result of the proposed LDA
Y. Yang et al. Industrial Marketing Management 86 (2020) 16–29

Fig. 9. Sales and business trend of topic 1 (heavy metal related tests) – 3, 6 and 9 months moving average comparison.

Table 5
. The results of vector auto regression model (1-month lag).
delta adjusted R2 Beta standardised beta Significance

Topic 12 Marketing and promotions 0.121 0.428 0.466 < .001***

Topic 7 Customer service 0.102 0.23 0.438 < .001***
Topic 10 Acid migration tests 0.057 0.529 0.327 < .001***
Topic 2 Water and microbiology tests 0.054 0.362 0.324 < .001***
Topic 3 Fabric related tests 0.053 0.752 0.314 < .001***
Topic 4 Application emails 0.044 0.288 0.314 .002***
Topic 5 Invoice related emails 0.048 0.026 0.313 .001***
Topic 14 Material related tests 0.042 0.238 0.293 .002***
Topic 13 Audit related emails 0.037 0.178 0.276 .003***
Topic 9 Others 0.034 0.376 0.257 < .001***
Topic 8 Hokla related emails 0.03 0.339 0.252 < .001***
Topic 6 Webinars and training 0.028 0.44 0.23 .009***
Topic 15 Pharmaceutical related tests 0.031 0.695 0.23 .007***
Topic 11 Test advisory 0.031 0.153 0.227 .006***
Topic 1 Heavy metal tests 0.009 0.136 0.127 .102

Notes: ***p < .01.

approach highlights three more business activities that may be ne-
glected by the expert team (promotion and marketing emails, applica-
tion emails and test advisory emails), showing that the big data ana-
lytics can indeed complement the experience-based decision process.
This finding supports the statement of McAfee et al. (2012) that “data-
driven decisions tend to be better decisions”, and we suggest that
companies should consider leveraging on both domain expertise to-
gether with data science when making final business decisions. Finally,
the impact rank of business trends on monthly sales provides a quan-
titative basis for the firm's resource distribution plan. Given the limited
resources of a firm, management should investigate the business ac-
tivities that cannot contribute to revenue increase, understanding the
reasons for their low effectiveness.
Finally, in the above analysis we have consistently considered the
period 2009–2018. Still as in the period between 2009 and 2012 the
volume of emails was significantly less than the volume of emails be-
tween 2013 and 2018, we could have potentially focused our analytical
effort on this period. The analysis (Tables 6 and 7) returns a significant
performance improvement for the VAR model, if we only use the data
after 2013. Taking topics 12 and 13 as examples, the delta adjusted R2

26
Y. Yang et al. Industrial Marketing Management 86 (2020) 16–29

Table 6 data techniques can solve the information overload problem and the
. The results of Granger causality tests (1-month lag) (2013–2018). knowledge extracted can support marketing decisions (Morabito, 2015;
Cluster id Cluster label F-statistics P-value Wedel & Kannan, 2016). To address the above, we conducted an em-
pirical study to showcase how a big data approach can transform
12 Marketing and promotions 45.4364 < .001*** mundane emails into valuable information using data analytics prac-
7 Customer service 34.736 < .001***
tices. Leveraging an advanced big data analytic technique, we identified
3 Fabric related tests 11.206 .0014***
10 Acid migration tests 10.2637 .0022***
a number of thematic clusters. These could be used to monitor the
2 Water and microbiology tests 10.1039 .0024*** demand trend in a regular manner. The business information extracted
5 Invoice related emails 9.2015 .0036*** was then compared to revenues, showing good predictive power for
14 Material related tests 6.9465 .0107** explaining the performance of a company.
4 Application emails 5.7972 .0192**
Given the evolving nature of the volume of data and the increasing
13 Audit related emails 4.9 .0307**
11 Test advisory 3.9476 .0516* importance of data-driven strategies, this study contributes to the ex-
9 Others 3.9469 .0515* tant body of literature in several ways. We have showcased a detailed
15 Pharmaceutical related tests 3.8939 .0532* process of using a big data NLP method to achieve effective knowledge
8 Hokla related emails 3.4545 .0681*
extraction. Our approach demonstrates the transformation process from
6 Webinars and training 3.1738 .0755*
1 Heavy metal tests 0.9374 .3369
data to business value, producing a better understanding of the appli-
cation of big data analytics and the knowledge discovery process.
Notes: *p < .1, **p < .05, ***p < .01. Additionally, the proposed approach using an internal and cost-effec-
tive information source (e.g., daily business emails) to achieve business
of topic 12 increase from 0.121 to 0.296 and the delta adjusted R2 of trend analysis of companies can help address the significant problem of
topic 13 increase from 0.037 to 0.043. Such a difference meant a dif- data acquisition and management in the B2B context (Fan et al., 2015).
ferent ranking of the clusters when sorting them by the delta adjusted Given that email is profoundly embedded in the daily operation and
R2. All clusters that were found to be statistically significant in the full- communication of a company, it can be sustainably generated and
period analysis remained significant in the 2013–2018 period. Heavy added to a massive and traceable repository. Our study showcases how
metal test results were found to be insignificant in both cases. email can be deemed an important data source in firms and how the
underlying business information in emails can offer valuable insights.
Also, as emails can be automatically updated and kept in dynamic form,
5. Conclusion the chronological analysis of the short-term trend of these business
activities can generate profound insights that would have been difficult
The underlying business information, especially the market trend to obtain otherwise. We suggest that companies should consider
and consumer preferences, has captured the interest of scholars and leveraging on both domain expertise together with data science when
business practitioners in the last few decades. Gaining such information making business decisions.
in advance can provide companies with considerable competitive ad-
vantage compared to peers. The era of web 2.0 has brought both op-
portunity and challenges for business practitioners. The data revolution 6. Practical implications
underway demands a broader appreciation of the variety of emerging
data sources and types, and a more comprehensive set of skills, in- This study also has several practical implications for B2B strategists,
cluding those being developed in the digital humanities, as well as basic marketers, as well as IT professionals. First, practitioners should realise
coding and simulation. Such a situation results in something of a that the growth of the digital universe continues to outpace the growth
paradox in that, despite the emerging data deluge, information ex- of human analytic ability. The ability to utilise the ever-increasing in-
tracted from the data is still highly limited. As McAfee et al. (2012) formation is critical to seize the opportunity in competition, and the
describe, despite the increasing importance and relatively higher ac- data utilisation relies on big data analytics. Using the case in this study
cessibility of data generated in companies' internal environments (e.g., as an example, we identified the most “profitable” business activity in
emails, transaction records) (McAfee et al., 2012), seldom do compa- this company as marketing and promotion events, while “heavy-metal
nies try to extract information from it. To cope with this paradox, the related testing service” is not related to revenue growth. Such in-
concept “big data analytics” has been recognised by scholars in the formation can be critical for business strategists as they now know how
business and IT domains. They believe that the proper leverage of big to distribute company resources and how to estimate return-on-

Table 7
. The results of vector auto regression model (1-month lag) (2013–2018).
s Delta adjusted R2 Beta Standardised beta Significance

Topic 12 Marketing and promotions 0.296 0.575 0.728 < .01***

Topic 7 Customer service 0.251 0.317 0.702 < .01***
Topic 3 Fabric related tests 0.102 0.884 0.44 < .01***
Topic 10 Acid migration tests 0.094 0.601 0.425 < .01***
Topic 2 Water and microbiology tests 0.092 0.435 0.441 < .01***
Topic 5 Invoice related emails 0.085 0.324 0.438 < .01***
Topic 14 Material related tests 0.063 0.261 0.371 .012**
Topic 4 Application emails 0.052 0.29 0.362 .017**
Topic 13 Audit related emails 0.043 0.185 0.318 .028**
Topic 6 Webinars and training 0.035 0.415 0.27 .042**
Topic 9 Others 0.033 0.355 0.27 .047**
Topic 15 Pharmaceutical related tests 0.033 0.593 0.244 .049**
Topic 11 Test advisory 0.033 0.129 0.24 .048**
Topic 8 Hokla related emails 0.028 0.331 0.277 .063*
Topic 1 Heavy metal tests −0.001 0.087 0.113 .307

Notes: *p < .1, **p < .05, ***p < .01.

27
Y. Yang et al.
investment more holistically. Second, there is a growing consensus in
the business domain that using big data techniques will take hard work
and significant investment. This is true when data is expensive to obtain
or manage. Nevertheless, we argue that this problem can be alleviated
by using daily emails as an analytic information source. As the majority
of companies neglects the fact that email content contains information
about business decisions, actions, and transactions (Jackson & Tedmori,
2004), the email archive has the potential to become a cost-effective
tool to support big data techniques. It holds out the prospect of gaining
detailed knowledge about company business activities not captured by
expensive business intelligence software.
7. Future research
This study has some limitations that can be leveraged to inform
future research. First, owing to data and time restrictions, we were
unable to capture email data from other companies/industries, espe-
cially those which are outside of the B2B context. Due to the diverse
business nature and company culture, the predictive power of the
identified business trends may change on a cross-organisational basis.
Therefore, future research could apply our big data NLP methodology
to other industries and compare the results against different contextual
settings. Second, although we address the importance of the internal
data, it may still be possible to include other useful information sources
that could be obtained from the external environment, such as the web-
pages of competitors. Mining the business information from their
websites may obtain some insights in mapping the entire market of a
specific industry. Also, having internal sales by product type would
make it possible to test against the corresponding clusters. Third, in
terms of the validation of the LDA method, we utilise the manual labels
as the benchmark to calculate the precision and support of this ap-
proach. Future work may consider incorporating a more solid valida-
tion process to increase the confidence in the use of LDA for the iden-
tification of business clusters. Fourth, for the Granger causality test in
this paper, we fixed the polynomial lag operator at 5 to test the impact
of the business trend in a maximal time window of 5 months. This
setting may result in the relatively lower significance of some long-term
orientated activities (e.g., training and webinars for employees). There
is a possibility that these business activities have to take sufficient time
to harvest their outcomes, such as performance improvement (Bartel,
1995). To address this concern, we encourage future research to be
concerned with these activities for further exploring their potential
from the long-term perspective.
References
Ajaz, S., Nafis, M. T., & Sharma, V. (2017). Spam mail detection using hybrid secure hash
based naive classifier. International Journal of Advanced Research in Computer Science,
8(5), 5. https://doi.org/10.26483/ijarcs.v8i5.3675.
Alrashed, T., Awadallah, A. H., & Dumais, S. (2018). The lifetime of email messages: A large-
scale analysis of email revisitation. Paper presented at the Proceedings of the 2018
Conference on Human Information Interaction & Retrieval, New Brunswick, NJ, USA.
Androutsopoulos, I., Koutsias, J., Chandrinos, K. V., & Spyropoulos, C. D. (2000). An
experimental comparison of naive bayesian and keyword-based anti-spam filtering with
personal e-mail messages. (Paper presented at the Proceedings of the 23rd annual
international ACM SIGIR conference on Research and development in information
retrieval, Athens, Greece).
Asay, M. (2014). 8 reasons why big data projects fail. Retrieved from https://www.
informationweek.com/big-data/big-data-analytics/8-reasons-big-data-projects-fail/
a/d-id/1297842.
Bartel, A. P. (1995). Training, wage growth, and job performance: Evidence from a
company database. Journal of Labor Economics, 13(3), 401–425.
Basavaraju, M., & Prabhakar, D. R. (2010). A novel method of spam mail detection using
text based clustering approach. International Journal of Computer Applications, 5(4),
15–25.
Bawm, Z. L., & Nath, R. P. D. (2014). A conceptual model for effective email marketing.
22–23 Dec. 2014. Paper presented at the 2014 17th International Conference on
Computer and Information Technology (ICCIT).
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of
Machine Learning Research, 3(Jan), 993–1022.
Bohanec, M., Borstnar, M., & Robnik-Sikonja, M. (2015). Feature subset selection for B2B
28
Industrial Marketing Management 86 (2020) 16–29
sales forecasting. Proceedings of 13th International Symposium on Operational Research,
SOR, Vol. 15.
Boyd, D., & Crawford, K. (2012). Critical questions for big data. Information,
Communication & Society, 15(5), 662–679. https://doi.org/10.1080/1369118X.2012.
678878.
Brown, B., Chui, M., & Manyika, J. (2011). Are you ready for the era of ‘big data’.
Mckinsey Quarterly, 4(1), 24–35.
Bughin, J., Chui, M., & Manyika, J. (2010). Clouds, big data, and smart assets: Ten tech-
enabled business trends to watch. Mckinsey Quarterly, 56(1), 75–86.
Campbell, C. S., Maglio, P. P., Cozzi, A., & Dom, B. (2003). Expertise identification using
email communications. Paper presented at the Proceedings of the twelfth international
conference on Information and knowledge management, New Orleans, LA, USA.
Chang, Y. L., & Chien, J. T. (2009). Latent Dirichlet learning for document summarization.
In Acoustics, Speech and Signal Processing, 1689–1692 ICASSP 2009. IEEE International
Conference.
Chen, H., Chiang, R. H., & Storey, V. C. (2012). Business intelligence and analytics: From
big data to big impact. MIS Quarterly, 1165–1188.
Chien, J. T., & Wu, M. S. (2008). Adaptive Bayesian latent semantic analysis. IEEE
Transactions on Audio, Speech, and Language Processing, 16(1), 198–207.
Ciccio, C. D., & Mecella, M. (2013). Mining artful processes from knowledge workers'
emails. IEEE Internet Computing, 17(5), 10–20. https://doi.org/10.1109/MIC.
2013.60.
Cromwell, J. B., Hannan, M. J., Labys, W. C., & Terraza, M. (1994). Multivariate tests for
time series models. Thousand Oaks, CA: Sage.
Darling, W. (2018). A theoretical and practical implementation tutorial on topic modeling and
gibbs sampling.
Davenport, T. (2014). Big data at work: Dispelling the myths, uncovering the opportunities.
Harvard Business Review Press.
Deal, D. (2014). Workhorses and dark horses: Digital tactics for customer acquisition.
Gigaom Research Raport.
D'Haen, J., & Van den Poel, D. (2013). Model-supported business-to-business prospect
prediction based on an iterative customer acquisition framework. Industrial Marketing
Management, 42(4), 544–551.
Duff, B. (1996). Document management offers security and order for intranet informa-
tion. IIE Solutions, 28(12), 28–31.
Edmunds, A., & Morris, A. (2000). The problem of information overload in business or-
ganisations: A review of the literature. International Journal of Information
Management, 20(1), 17–28.
Eppler, M. J., & Mengis, J. (2004). The concept of information overload: A review of
literature from organization science, accounting, marketing, MIS, and related dis-
ciplines. The Information Society, 20(5), 325–344.
Erevelles, S., Fukawa, N., & Swayne, L. (2016). Big data consumer analytics and the
transformation of marketing. Journal of Business Research, 69(2), 897–904.
Fan, S., Lau, R. Y., & Zhao, J. L. (2015). Demystifying big data analytics for business
intelligence through the lens of marketing mix. Big Data Research, 2(1), 28–32.
Fox, C. (1989). A stop list for general text. SIGIR Forum, 24(1–2), 19–21. https://doi.org/
10.1145/378881.378888.
George, G., Haas, M. R., & Pentland, A. (2014). Big data and management. Academy of
Management Journal, 57(2), 321–326.
Goodpaster, K. E. (2015). Business ethics. Wiley Encyclopedia of Management.
GORDON, S., & MOHAMMAD, S. (2012). Perspective-taking and the attribution of ig-
norance. Journal for the Theory of Social Behaviour, 42(2), 181–200.
Griffin, A., Josephson, B. W., Lilien, G., Wiersema, F., Bayus, B., Chandy, R., & Spanjol, J.
(2013). Marketing's roles in innovation in business-to-business firms: Status, issues,
and research agenda. Marketing Letters, 24(4), 323–337.
Griffiths, T. L., & Steyvers, M. (2004). Finding scientific topics. Proceedings of the National
Academy of Sciences of the United States of America, 101(Suppl. 1), 5228–5235.
Gunasekaran, A., Papadopoulos, T., Dubey, R., Wamba, S. F., Childe, S. J., Hazen, B., &
Akter, S. (2017). Big data and predictive analytics for supply chain and organiza-
tional performance. Journal of Business Research, 70, 308–317. https://doi.org/10.
1016/j.jbusres.2016.08.004.
Gustke, C. (2013). Big data takes turn as market darling. Retrieved from https://www.
cnbc.com/id/100638376.
Hofmann, T. (2001). Unsupervised learning by probabilistic latent semantic analysis.
Machine learning, 42(1-2), 177–196.
Hong, L., & Davison, B. D. (2010). Empirical study of topic modeling in Twitter. Paper
presented at the Proceedings of the first Workshop on Social Media Analytics,
(Washington D.C., District of Columbia).
Jackson, A., Yates, J., & Orlikowski, W. (2007). Corporate blogging: building community
through persistent digital talk. Jan. 2007. Paper presented at the System Sciences,
2007. HICSS 2007. 40th Annual Hawaii International Conference on.
Jackson, T., & Tedmori, S. (2004). Capturing and managing electronic knowledge: The de-
velopment of the email knowledge extraction system.
Jackson, T. W., Tedmori, S., Hinde, C. J., & Bani-Hani, A. I. (2012). The boundaries of
natural language processing techniques in extracting knowledge from emails. Journal
of Emerging Technologies in Web Intelligence, 4(2), 119–127.
Janine, S., Ton, D. J., & Van Joolingen Wouter, R. (2004). The effects of discovery
learning and expository instruction on the acquisition of definitional and intuitive
knowledge. Journal of Computer Assisted Learning, 20(4), 225–234.
John Walker, S. (2014). Big data: A revolution that will transform how we live, work, and
think. International Journal of Advertising, 33(1), 181–183.
Kaisler, S., Armour, F., Espinosa, J. A., & Money, W. (2013). Big data: Issues and challenges
moving forward. Paper presented at the 46th Hawaii international conference on
System sciences (HICSS).
Kok, S., & Yih, W. T. (2009). Extracting product information from email receipts using markov
logic. Paper presented at the 6th Conference on Email and Anti-Spam, Mountain
Y. Yang et al.
View, California, USA.
Lackman, L. C. (2007). Forecasting sales for a B2B product category: Case of auto com-
ponent product. Journal of Business & Industrial Marketing, 22(4), 228–235.
Lahiri, S., Mihalcea, R., & Lai, P. H. (2017). Keyword extraction from emails. Natural
Language Engineering, 23(2), 295–317.
Laney, D. (2001). 3D data management: Controlling data volume, velocity and variety.
Retrieved from http://blogs.gartner.com/doug-laney/files/2012/01/ad949-3D-Data-
Management-Controlling-Data-Volume-Velocity-and-Variety.pdf.
LaValle, S., Lesser, E., Shockley, R., Hopkins, M. S., & Kruschwitz, N. (2011). Big data,
analytics and the path from insights to value. MIT Sloan Management Review,
52(2), 21.
Li, J., Sen, S., & Zaman, N. (2015). Entity extraction from business emails. International
Journal of Information Technology and Computer Science, 7(9), 15–22.
Lilien, G. L. (2016). The B2B knowledge gap. International Journal of Research in
Marketing, 33(3), 543–556.
Lycett, M. (2013). ‘Datafication’: Making sense of (big) data in a complex world. European
Journal of Information Systems, 22(4), 381–386.
Madnani, N. (2007). Getting started on natural language processing with Python.
Crossroads, 13(4), 5.
Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., & Hung Byers, A.
(2011). Big data: The next frontier for innovation, comptetition, and productivity.
Martin, B., Elin, C.-E., Dorota, K., Marek, K., John, S., & Yu, Z. (2015). Internal migration
data around the world: assessing contemporary practice. Population, Space and Place,
21(1), 1–17.
McAfee, A., Brynjolfsson, E., Davenport, T. H., Patil, D. J., & Barton, D. (2012). Big data:
The management revolution. Harvard Business Review, 90(10), 60–68.
Mithas, S., Lee, M. R., Earley, S., Murugesan, S., & Djavanshir, R. (2013). Leveraging big
data and business analytics. IT Professional, 15(6), 18–20.
Moh, T.-S., & Bhagvat, S. (2012). Clustering of technology Tweets and the impact of stop
words on clusters. Paper presented at the Proceedings of the 50th Annual Southeast
Regional Conference, Tuscaloosa, Alabama.
Morabito, V. (2015). Big data and analytics: Strategic and organizational impacts. Springer.
Papagiannidis, S., See-to, E., Assimakopoulos, D., & Yang, Y. (2018). Identifying in-
dustrial clusters with a novel big-data methodology: Are SIC codes (not) fit for pur-
pose in the internet age? Computers & Operations Research, 98(October 2018),
355–366.
Paul, S. A. (2016). Find an expert: Designing expert selection interfaces for formal help-giving.
Paper presented at the 2016 CHI Conference on Human Factors in Computing
29
Industrial Marketing Management 86 (2020) 16–29
Systems.
Řehůřek, R., & Sojka, P. (2010). Software framework for topic modelling with large corpora.
Rose, S., Engel, D., Cramer, N., & Cowley, W. (2010). Automatic keyword extraction from
individual documents. Text mining (pp. 1–20). .
Rusetski, A. (2014). Pricing by intuition: Managerial choices with limited information.
Journal of Business Research, 67(8), 1733–1743.
Sathi, A. (2014). Engaging customers using big data: How marketing analytics are transforming
business. Palgrave Macmillan.
Shah, P. K., Perez-Iratxeta, C., Bork, P., & Andrade, M. A. (2003). Information extraction
from full text scientific articles: Where are the keywords? BMC Bioinformatics, 4(1),
https://doi.org/10.1186/1471-2105-4-20 20.
Sivarajah, U., Kamal, M. M., Irani, Z., & Weerakkody, V. (2017). Critical analysis of big
data challenges and analytical methods. Journal of Business Research, 70, 263–286.
Tedmori, S., Jackson, T. W., & Bouchlaghem, D. (2006). Locating knowledge sources
through keyphrase extraction. Knowledge and Process Management, 13(2), 100–107.
The Radicati Group (2017). Email statistics report, 2017–2021. Retrieved from https://
www.radicati.com/wp/wp-content/uploads/2017/01/Email-Statistics-Report-2017-
2021-Executive-Summary.pdf.
Tirunillai, S., & Tellis, G. J. (2014). Mining marketing meaning from online chatter:
strategic brand analysis of big data using latent dirichlet allocation. Journal of
Marketing Research, 51(4), 463–479.
Wallach, H. M., Mimno, D. M., & McCallum, A. (2009). Rethinking LDA: Why priors
matter. Advances in Neural Information Processing Systems, 1973–1981.
Wang, G., Gunasekaran, A., Ngai, E. W. T., & Papadopoulos, T. (2016). Big data analytics
in logistics and supply chain management: Certain investigations for research and
applications. International Journal of Production Economics, 176, 98–110.
Wedel, M., & Kannan, P. K. (2016). Marketing analytics for data-rich environments.
Journal of Marketing, 80(6), 97–121.
Wei, X., & Croft, W. B. (2006). LDA-based document models for ad-hoc retrieval. Paper
presented at the Proceedings of the 29th annual international ACM SIGIR conference
on Research and development in information retrieval, Seattle, Washington, USA.
Whittaker, S., Bellotti, V., & Moody, P. (2005). Revisiting and reinventing email. Human
Computer Interaction, 20(1), 1–9.
Wiersema, F. (2013). The B2B agenda: The current state of B2B marketing and a look ahead.
Vol. 42.
Xu, Z., Frankwick, G. L., & Ramirez, E. (2016). Effects of big data analytics and traditional
marketing analytics on new product success: A knowledge fusion perspective. Journal
of Business Research, 69(5), 1562–1566.