Health Communication

ISSN: 1041-0236 (Print) 1532-7027 (Online) Journal homepage: https://www.tandfonline.com/loi/hhth20

A Social Networks Approach to Understanding

Vaccine Conversations on Twitter: Network
Clusters, Sentiment, and Certainty in HPV Social
Networks

Itai Himelboim, Xizhu Xiao, Danielle Ka Lai Lee, Meredith Y. Wang &
Porismita Borah

To cite this article: Itai Himelboim, Xizhu Xiao, Danielle Ka Lai Lee, Meredith Y. Wang & Porismita
Borah (2019): A Social Networks Approach to Understanding Vaccine Conversations on Twitter:
Network Clusters, Sentiment, and Certainty in HPV Social Networks, Health Communication, DOI:
10.1080/10410236.2019.1573446

To link to this article: https://doi.org/10.1080/10410236.2019.1573446

Published online: 14 Jun 2019.

Submit your article to this journal

Article views: 63

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=hhth20
HEALTH COMMUNICATION
https://doi.org/10.1080/10410236.2019.1573446

A Social Networks Approach to Understanding Vaccine Conversations on Twitter:

Network Clusters, Sentiment, and Certainty in HPV Social Networks
Itai Himelboima, Xizhu Xiaob, Danielle Ka Lai Leeb, Meredith Y. Wangc, and Porismita Borahb
a
Social media Engagement and Evaluation, Grady College of Journalism and Mass Communication; bEdward R. Murrow College of Communication,
Washington State University; cDepartment of Mass Communication, Stephen F. Austin State University

ABSTRACT
Individuals increasingly rely on the Internet, and social media in particular, for health-related information.
A recent survey reports that 80% of Internet users search for health information online. In the present study, we
employ Twitter data to understand content characteristics and the patterns of content flow of the conversa-
tions about the HPV vaccine debate. Approaching the HPV vaccine conversations on Twitter as a social
network, we can identify key self-formed subgroups—clusters of users who create “siloes” of interactions
and information flow. Combining network analysis, computer-aided content analysis, and human-coded
content analysis, we explored the communication dynamics within the groups in terms of group members’
affective and cognitive characteristics. Our findings show that positive emotion is positively correlated with
cluster density, an indicator of strong ties and rapid information flow. In the case of negative emotion, we
found that anger is a significant negative predictor for graph density. We also found a correlation between
certainty and tentativeness; both at cluster as well as at tweet level, suggesting that clusters bring together
people who are sure about the HPV vaccine and people who are exploring for answers.

Human papillomavirus (HPV) is a common sexually transmitted
infection in the United States. There are approximately 79 million
people infected with HPV, and an estimated 14 million becomes
newly infected every year (Markowitz et al., 2014). Although the
majority of these infections resolve without many serious conse-
quences, HPV is considered responsible for 90% of anal and
cervical cancers, 50% and more vaginal, vulvar, and penile cancers
(Centers for Disease Control and Prevention [CDC], 2018). HPV
vaccination can protect against strains of HPV and have become
an integral part of adolescent preventive health care (CDC, 2018).
The traditional information source, such as television and
newspapers, as well as health institutions, plays an essential
role in informing and educating the public about health. More
recently, however, the dissemination of health information
has changed as user-generated content online become easy
to produce and conveniently accessible through social media.
A recent survey reports that 80% of Internet users search for
health information online (Weaver, 2013). As the media land-
scape evolves, more and more individuals are receiving health
information from social media platforms such as Twitter. This
is particularly relevant to the younger population, who may be
less tuned to news sources and or health care providers (Fox
& Duggan, 2012) but more influenced by health information
online (Wartella, Rideout, Montague, Beaudoin-Ryan, &
Lauricella, 2015). This is important for the high public con-
troversy HPV vaccination, where health care providers’ infor-
mation may compete with the vaccine criticism movement
(Keelan, Pavri, Balakrishnan, & Wilson, 2010).
Individuals then have more control than ever of the content
that they choose to tune to and share with others. Users rely on
their social media connections to determine what content—estab-
lished as news media and health institutions or less verified user-
generated content—they see and what may go unnoticed. As the
traditional one-to-many model loses its dominance in the new
information ecosystem, a more complex approach is needed to
understand the flow of health-related information—the growing
public controversy around vaccinations, and HPV vaccination in
particular. Since an increasing number of individuals seek and
share health information on platforms such as Twitter, it is crucial
to understand how characteristics of HPV vaccine-specific con-
tent are associated with their information flow patterns. This
study applies social network analysis to identify the clusters—
siloes of information flow—that users create on Twitter when
posting and sharing content about HPV vaccines. Applying com-
puter-aided content analysis and manual coding, we explore the
communication dynamics within the groups regarding group
members’ affective and cognitive characteristics.
Importance of studying HPV vaccination
A plethora of studies has investigated public awareness of the
HPV vaccine as well as HPV vaccine-related media coverage.
In a study examining the challenges and opportunities faced
by HPV vaccination, Sherris et al. (2006) suggested that
although health providers and health policymakers have
expressed concerns regarding HPV associated diseases and

CONTACT Porismita Borah porismita@gmail.com Edward R. Murrow College of Communication, Washington State University, Pullman, WA 99163, USA.
Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/hhth.
© 2019 Taylor & Francis Group, LLC
2 I. HIMELBOIM ET AL.
Table 1. Regression analysis predicting graph density.
Graph Density
Constant 0.007 (0.001)***
Independent variables
Positive Emotion 0.002 (0.001)*
Anger −0.007 (0.003)*
Total adjusted R2 0.220**
Number of cases 42
Note: * p < .05, ** p < .01, *** p < .001 The coefficients are unstandardized with
standard errors in parentheses
Table 2. Correlation between certainty and tentativeness.
r N
Cluster Level 0.443** 42
Tweet Level 0.061*** 27,023
Note: * p < .05, ** p < .01, *** p < .001
conveyed an interest in HPV vaccination, the knowledge gap
appeared to be salient. More research has explored media
coverage of HPV specific issues since the media has been an
essential source for individuals to access medical information.
Examining news articles between 2002 and 2005, Calloway,
Jorgensen, Saraiya, and Tsui (2006) revealed that the majority
of the articles were directly derived from scientific data but
failed to emphasize the promising benefits of HPV vaccines.
Similarly, Krakow and Rogers (2016) demonstrated the lack of
detailed coverage of HPV and the HPV vaccine in newspa-
pers. Media coverage of HPV vaccination have important
implications for the public understanding of the vaccine
(Gollust, LoRusso, Nagler, & Fowler, 2016; Hilton, Hunt,
Langan, Bedford, & Petticrew, 2010) and as such research
should monitor the information available to the public.
HPV vaccination and emotions
Prior research (e.g. Christy et al., 2016; Pereira et al., 2004)
highlights the critical role that emotions play in health-related
issues, including HPV. In a study of unvaccinated college
students, Christy et al. (2016) found that anticipated regret,
referring to the regret that if one did not get vaccinated but
developed HPV infection later on, did significantly increase
the willingness of getting HPV vaccination among college
males. Shepherd and Gerend (2014) further suggested that
individuals had stigmatized feelings such as “gross-out,” “dis-
honest,” and “unwise,” when rating a patient with cervical
cancer with a specified reason such as sexually transmitted
illness.
Social media
Briones, Nan, Madden, and Waks (2012) revealed that the
majority of the HPV vaccination-related YouTube videos
were negative with more “likes” compared to the positive
ones. The study also suggested that user-generated content
consisted of a large number of negative videos, which might
mislead the publics’ perceptions. As a result, publishing cred-
ible videos about HPV and HPV vaccination became
a pressing issue for official health agencies (Briones et al.,
2012). In a recent study, Massey et al. (2016) investigated
Twitter data to examine vaccine communication on the
forum. Their findings showed that there was more positive
sentiment associated with “HPV vaccine” while name-brand
vaccines were associated with negative sentiment. Thus, there
is a continuing conversation about HPV vaccines in the media
as well as the public. Just as other health issues, HPV evokes
strong emotions in individuals.
Many studies, as discussed, examined health-related con-
versation, including HPV vaccination, at the message or the
user levels. However, on social media, content is discussed at
a group level, as it is being shared and exchanged. Each posted
message is available to a group of people, based on their social
media connectivity. Opinions or sentiment become dominant
within groups of users, and it is a product of interactions
among these users, not just “mechanical aggregation of the
expressions of isolated individuals” (Cramér, 2016, p. 19).
Twitter as a social network
A social network approach shifts the focus from the aggrega-
tion of the post or user-level attributes to group structures and
patterns of interconnections. A network is formed by nodes—
social actors and links, symbolic ties or relationships between
nodes (Himelboim, 2017). On Twitter, we consider each par-
ticipant in a conversation as a node while relationships that
are created by tagging (i.e., mentions) retweets and replies, as
connecting links. In social networks, giving the opportunity to
interact freely, people choose to interact with some more than
others, creating subgroups of self-selected subgroups, or net-
work clusters. In other words, they share connectivity-based
characteristics (Newman, 2003; Wasserman & Faust, 1994).
Within each sub-community, people are more closely inter-
connected that with users in other clusters and share similar
information sources (Carrington, Scott, & Wasserman, 2005;
Himelboim & Han, 2014).
Furthermore, individuals also tend to connect with similar
others, a phenomenon called homophily: “a contact between
similar people occurs at a higher rate than among dissimilar
people” (McPherson, Smith-Lovin, & Cook, 2001, p. 416).
Evidence of homophily was found based on demographic
characteristics such as age, sex, race, and education (as early
as Bott, 1928; Loomis, 1946) and psychological characteristics
like intelligence, attitudes, and aspirations (Almack, 1922;
Richardson, 1940). On social media, Conover, Goncalves,
Ratkiewicz, Flammini, and Menczer (2011) found similarities
within clusters of political leaning expressed in tweets while
Himelboim, McCreery, and Smith (2013) identified similarity
of hyperlinks’ political leanings on Twitter.
From a different angle, Sunstein (2005), suggested that
without caution, the group interaction would become “group-
think,” in which some individuals became reticent in order to
achieve uniformity and avoid conflicts. The “groupthink”
effect was also discussed in a panel study of health valuation
(Robinson & Bryan, 2013). The results of the study indicated
that individuals rated health states more severely after group
deliberation, which might be associated with deep thinking,
but also possibly related to “herding,” in which individuals
who perceived themselves less-informed tended to follow
others’ opinions.

Network clusters, then, make a useful unit of analysis as they
capture self-generated, partially secluded, groups of users who
are exposed to similar information sources and share similar
characteristics (Himelboim, 2017). Networks and clusters, in
particular, can differ from each other regarding their density.
In clusters with high density, individuals are highly intercon-
nected with each other, while low density indicates loose con-
nectivity among members. Density was found to be associated
with information and knowledge sharing (Carley & Wendt,
1991; Zubcsek, Chowdhury, & Katona, 2014). Therefore, den-
sity can suggest a better quality of group conversation simply
because there are more back and forth among the members.
Density is also associated with the strength of relationships
(Granovetter, 1973), and from a health standpoint, it means
a stronger bonding among members since people are getting as
well as giving feedback to each other. Either way, high density,
which indicates more interconnected relationships among
members and more frequent exchanges of opinions, has desir-
able qualities since it shows that those conversations on Twitter
may be deliberative.
Research question and hypotheses
As prior research has suggested, Twitter could be a place
where people have easy access to the most up-to-date infor-
mation regarding almost any health issue, as well as a place
where individuals get social support from each other
(Himelboim & Han, 2014; Myrick, Holton, Himelboim, &
Love, 2016). In other words, individuals may have cognitive
needs as well as emotional needs when they communicate
about health issues on social media platforms like Twitter.
Emotional support is the most prominent support people look
for through health information seeking on social media (Oh,
Lauckner, Boehmer, Fewins-Bliss, & Li, 2013). Emotional
support emphasizes closeness and bonding with others, and
cluster density may be a proxy of the level of emotional
support within each cluster. Besides, emotional support brings
personal encouragement and reassurance as well as bolster
hope and confidence (Mo & Coulson, 2008; Yoo et al., 2014).
Online support group research also shows positive emotion
like empathy and positive attitudes toward health problems
bring people with similar health concern together and closer
(Wright, Bell, Wright, & Bell, 2003; Zhang, He, & Sang, 2013).
Based on the above discussion, we propose two hypotheses
regarding how the emotion of tweets affect structure of the
network:
H1a: Cluster density is positively associated with the amount
of positive content across messages in the cluster.
H1b: Cluster density is negatively associated with the amount
of negative content.
Barabas (2004) found that even though group conversa-
tions help individuals to better understand policy being dis-
cussed, opinion strength moderates opinion change due to
deliberation. Specifically, Barabas (2004) found that people
with weak prior opinions changed the most while individuals
with strong prior opinions did not change much. Certainty
HEALTH COMMUNICATION 3
and confidence in one’s opinion would affect both the quality
and effectiveness of conversation. Kahan, Braman, Cohen,
Gastil, and Slovic (2010) showed that egalitarian communitar-
ians perceived HPV vaccine as safe, whereas hierarchical
individualists saw more risks. However, the two different
voices became more aligned after the exposure to advocates
from each side arguing in favor of or against HPV vaccines.
Unlike formal group deliberation in real life, Twitter users
start conversations with others without geographic limitation.
Often, they do not have a common goal in mind when they start
posting content and joining a conversation. In the current study,
we are interested in finding out whether Twitter users like to go
out of their comfort zone and talk to people who have different
levels of confidence and attitude certainty regarding HPV vac-
cine, or, do they stay in an echo chamber and only talk to people
who are like them. To find out we propose a research question:
RQ1: To what extent do people with strong attitudes about HPV
vaccine interact with people with uncertain attitudes about the
topic?
Methods
Data
Twitter activity related to HPV vaccination was collected in one-
week period datasets, starting 9/20/2016 for ten weeks. The time
period was selected to capture the meaningful length of con-
versation (10 weeks), with no HPV vaccine-related particular
event that can skew the conversation, which will be the closed to
the time of the execution of this study so that we can provide the
most recent data. Each dataset included self-descriptive infor-
mation about users, the content of their tweets, including hash-
tags and hyperlinks, as well as relationships among users (i.e.,
retweets, mentions, and replies). NodeXL, network analysis, and
visualization software package for Microsoft Excel, was used to
collect the data. The total number of Twitter users across the
datasets were 39,368, using the following Boolean query:
(Vaccine OR Vaccination OR Vaccines OR Immunization OR
#Vaccine OR #Vaccination OR #Vaccines OR #Immunization)
AND (HPV OR #HPV OR “Human papillomavirus infection”)
We combined network analysis using NodeXL, Linguistic
Inquiry and Word Count (LIWC) for computer-aided analysis
of the content, and traditional human-coding for more in-
depth analysis of the conversation and shared hyperlinks.
Network analysis and clusters
NodeXL, the social network analysis and visualization soft-
ware, was used to calculate metrics and visualize the networks.
A social network, then, was build based on each dataset,
where posting users are nodes (or actors) and mentions and
replies are linked (using a customized application to extract
mention and reply indicators within tweets). We identified
network clusters of relatively more connected groups of users
in the topic-networks using the Clauset–Newman–Moore
algorithm (Clauset, Newman, & Moore, 2004), which is
4 I. HIMELBOIM ET AL.
included in the NodeXL software. We selected this algorithm
for its ability to analyze large network datasets and efficiently
found subgroups. This algorithm uses edge betweenness as
a metric to identify the boundaries of communities. Each user,
then, is classified into the best-fit group (cluster), in terms of
the interconnectivity among users. The major clusters were
identified by ordering clusters by size in descending order.
The clusters identified using the algorithm above were gen-
erally very small, typically leaving very few large ones. If no
clear drop in cluster size was identified, the top three clusters
were selected, as in the majority of datasets a drop in size was
recorded after the top three clusters.
Density
NodeXL was used to calculate the density of each cluster.
Density—a measurement of users’ interconnectedness—is cal-
culated as the total number of existing links over the number
of possible links. In total, we identified 42 clusters on Twitter
and calculated densities for each (M = 0.0076, SD = 0.0048).
Linguistic Inquiry and Word Count (LIWC)
LIWC is a computerized text analysis, has been widely used to
analyze individuals’ psychological and emotional activities by
analyzing word usage (Tausczik & Pennebaker, 2010). LIWC
categorizes words based on functions, which indicate indivi-
duals’ sentimental and emotional status, and styles, which
suggest individuals’ communication patterns (Tausczik &
Pennebaker, 2010). As such, due to the very reductionist
nature of the analysis, LIWC can be utilized to examine
a broad range of text files including essays, blog posts, or
court records (Tausczik & Pennebaker, 2010). For example,
Owens and Wedeking (2011) employed ten LIWC indicators
such as tentativeness and certainty that insinuate individuals’
cognitive complexity in writing to examine the clarity of
Supreme Court records systematically. The findings revealed
that the writing clarity of justices did differ greatly under
various occasions (Owens & Wedeking, 2011). In line with
the emotion contagion revealed by LIWC, Coviello et al.
(2014) further probed the “emotion spillover” in the online
environment by looking into individuals’ text-based emo-
tional expressions on Facebook. Findings show that emotions
conveyed online not only synchronized within groups but also
had significant implications for social relations (Coviello et al.,
2014). Similarly, by looking at a collection of vocabulary use
on Facebook through the open-vocabulary approach of
LIWC, Schwartz et al. (2013) discovered that online expres-
sions considerably reflected gender, age, and personalities.
More studies have explored word use on Twitter. Bae and
Lee (2012) analyzed over 3 million messages posted by pop-
ular Twitter users and their followers by utilizing the “senti-
ment analysis” of LIWC, which was to distinguish positive
and negative opinions contained in texts. The results demon-
strated that the sentiments of popular Twitter accounts
echoed in the responses of the followers. The findings also
indicated the influence of popular Twitter accounts on indi-
viduals’ offline lives such as product preference, or election
forecast (Bae & Lee, 2012). These results were reinforced by
Tumasjan, Sprenger, Sandner, and Welpe (2011). The authors
concluded that Twitter cultivated a virtual environment for
political deliberation, in which emotional words used in
tweets about political candidates reflected individuals’ offline
political preferences and voting behaviors (Tumasjan et al.,
2011). Moreover, Stieglitz and Dang-Xuan (2012) found out
that tweets with more affective words, either negative or
positive, were more influential and widely shared. In the
current study, we used LIWC to measure emotion, certainty,
and tentativeness of each cluster as well as certainty and
tentativeness of each tweet.
One should note that computer-aided content analysis is not
perfect and may miss cases of sarcasm or parody that are often
understood only within the larger context of the conversation.
This is true for human coders as well. In more than one way the
question of possible miscoding of the content of satire, parody,
humor or sarcasm, to name a few, is part of the price that we pay
when we opt to examine “big data.” These issues are less pre-
valent in qualitative analysis, with in-depth insights of smaller
conversations. As in any decision about research methodology,
including ours, it has strength and weaknesses. That said, as
Tausczik and Pennebaker (2010) reviewed, dozens of published
articles have utilized LIWC’s sentiment and emotion analysis. It
is also worth noting that Gonçalves, Araújo, Benevenuto, and
Cha (2013) compared eight computer-aided sentiment analysis
applications and found that LIWC had the highest percentage of
agreement (73.49%).
Emotion analysis
LIWC captured the percentage of positive (e.g. love, nice,
sweet) and negative emotion words (e.g. hurt, ugly, nasty)
within tweets from each cluster. “LIWC ratings of positive
and negative emotion words correspond with human ratings
of the writing excerpts” (Tausczik & Pennebaker, 2010, p. 32).
For negative emotion, specific emotions such as anxiety,
anger, sadness were also identified. For instance, using
words “worried” or “nervous” is an indication of anxiety.
Using words like “hate”, “kill”, “annoyed” is an indication of
anger. Also, using words like crying, grief, sad is an indication
of sadness. Notably, LIWC as many other computer-aided
sentiment analysis applications (e.g., Crimson Hexagon and
IBM’s Watson), identify individual negative emotions, but
capture positive sentiment as a single construct. This general
practice in research and industry can be traced to the work of
the psychologist Paul Ekman who proposed a set of “basic
emotions” based on their distinct psychological and physiolo-
gical “signals.” This set includes a range of negative emotions
but only one positive (Ekman’s 6: anger, happiness, surprise,
disgust, sadness, and fear). In his research, he found that
individual positive emotions, such as amusement, relief, or
thrill of excitement, do not have the distinct psychological and
physiological signals that characterize the negative emotions
(Ekman, 1992). Meaning that while not all positive emotions
are the same, yet, they are not distinct enough from one
another to be clearly identified.
Our analysis included positive emotion (Min: 0.00- Max:
4.47, M = 1.39, SD = 1.03), negative emotion (Min: 0.00, Max:
3.76, M = 1.81, SD = 1.10), anger (Min: 0.00- Max: 0.75, M = 0.17,
SD = 0.22), anxiety (Min: 0.00- Max: 1.83, M = 0.48, SD = 0.42)
and sadness (Min: 0.00- Max: 2.16, M = 0.49, SD = 0.48).

Certainty and tentativeness
Certainty score counts words like “always”, “absolutely”, and
“clearly”. Certainty is theorized as how confident one is about
something. A sample tweet reads: “#Gardasil: The Decision
We Will Always Regret.” Tentativeness score counts words
such as “maybe”, “fairly”, and “perhaps”. It measures the level
of tentativeness the tweet shows. Usually, tentativeness is
theorized as how hesitant or unsure one is on a certain issue
(Owens & Wedeking, 2011). A sample tweet reads: “Maybe
#hpv vax will affect cancer rates, we don’t know yet.”
We measured certainty and tentativeness of each cluster
(certainty: Min: 0.00- Max: 2.53, M = 0.61, SD = 0.52; tenta-
tiveness: Min: 0.00- Max: 2.22, M = 0.61, SD = 0.51), as well as
certainty and tentativeness of each tweet (certainty: Min: 0.00-
Max: 20, M = 0.64, SD = 1.81; tentativeness: Min: 0.00- Max:
25, M = 0.74, SD = 2.08; N = 27023).
Finally, for further understanding of the conversations, we
conducted a manual analysis of the content by the cluster.
Firstly, we identified popular tweet content by calculating the
frequency of each tweet. Tweets that appeared six times and
above were deemed as popular and included in the subse-
quent emotion analysis. We then examined the content of
positive vs. negative emotions by clusters. Four codes were
developed: positive, negative, neutral, and conflicted. Cluster
content coded as positive praised the HPV vaccine or high-
lighted the benefits of the vaccine; cluster content coded as
negative undermined the HPV vaccine or highlighted the
safety concerns of the vaccine; cluster content coded as neu-
tral stated objective facts about the HPV vaccine; and cluster
content coded conflicted contained both positive and negative
attitudes about the vaccine. To calculate inter-coder reliability,
three clusters were analyzed by two coders. 100% agreement
was achieved without further discussion. As the reliability was
impeccable, two coders split the cluster files and coded the
rest separately.
We also examined the credibility of the links associated in
these clusters. The credibility of the links mainly depended on
the information sources of links within tweets. Three types of
Figure 1. The social network created by an exemplary network structure created by Twitter users talking about the HPV vaccination, during week of September 20th, 2016.
HEALTH COMMUNICATION 5
links were regarded as credible: 1) scientific research and
academic institutes (e.g., NIH websites, websites of academic
peer-review journals and universities); 2) mainstream media
(e.g., CNN reports for information released by the CDC;
reports from Huffington Post); 3) interviews of professionals
in the field (e.g., doctors). On the other hand, links associated
with non-mainstream media; links that did not include
expert’s points of views with only personal opinions or feel-
ings; and links connected to advertisements, were considered
non-credible.
Results
For each of the 10 datasets, Clauset–Newman–Moore algo-
rithm (Clauset et al., 2004) was applied to identify clusters.
A total of 42 clusters were identified across all 10 datasets.
These clusters also differ in terms of their structure, some are
denser, that is more interconnected, than others (M = 0.0076,
SD = 0.0048). Figure 1 illustrates the nature of network
clusters. This exemplary network illustrates the top largest
clusters of Twitter users talking about the HPV vaccination,
during the week of September 20th, 2016 (N = 4,006 users;
top clusters’ size, n = 853, 414, 307, 141). Each node, a point
on the graph, is a user who talked about the topic during that
data collection period. Each arrow illustrates a retweet, men-
tion, or reply between two users. User-interaction in this
network formed four major clusters. Within each cluster,
users are more interconnected than across clusters, indicating
more reliance on one another for information flow, than
across clusters. In this example, densities of the top four
clusters were .003, .004, .005, and .007, which are much higher
that than the overall graph density of .0003. Furthermore,
while a total of 5,024 connects existed within the four clusters
(3,507, 723, 606, 188), a much weaker interconnectivity—247
links—was found between clusters, making a 7% of the total
within and across clusters links.
In order to test our two hypotheses, we regressed both
positive emotion and negative emotion on graph density
6 I. HIMELBOIM ET AL.
(see Table 1). Positive emotion is positively correlated with
graph density (B = 0.001, SE = 0.001, p < 0.05) while negative
emotion is negatively correlated with graph density
(B = −0.001, SE = 0.1, p = 0.162). A few examples of tweets
with positive emotions are as follows: “Sometimes people ask
me why I am working full time. Well, health insurance is
pretty sweet. Getting the HPV vaccine for $0. YAY”; “The
HPV vaccine has been so successful that current cervical
cancer screening guidelines may need to be revised.”
However, negative emotion failed to reach significance. We
further broke down negative emotion into specific emotions
(i.e. anger, anxiety and sadness). We found that only anger is
a significant predictor for graph density. In our final model,
anger is negatively related with graph density (B = −0.007,
SE = 0.003, p < 0.05) while positive emotion is positively
related with graph density (B = 0.002, SE = 0.0.001,
p < 0.05). In other words, H1a was supported and H1b was
partially supported. The more positive a cluster of tweets are,
members of this cluster are more closely interconnected. On
the other hand, for negative emotion the hypothesis was not
fully supported. But on further examination of discrete nega-
tive emotion, findings revealed an anger effect. The angrier
a cluster of tweets are, members of this cluster are less closely
interconnected with each other. Examples of tweets with
negative emotions are “Got the HPV vaccine, fainted. Fell
down the stairs and hurt my ankle, fainted”; “12-year old
girl dies hours after she is injected with HPV vaccine, ugly”;
and “The vaccine I had in 2009 3 times has completely ruined
my ability to sleep like a normal person. Since then I’m always
exhausted.”
In order to address our research question, we tested corre-
lation between certainty and tentativeness both at cluster as
well as at tweet level (see Table 2). If individuals with strong
attitudes toward HPV and people who are not certain about
HPV connect with each other, we should see certainty and
tentativeness positively correlated at cluster level. While if
individuals with strong attitudes toward HPV vaccine and
people who are not certain about HPV vaccine do not talk
to each other, we should not see a positive relationship
between these two variables. It should either be non-
significant or negative. Our results show that certainty and
tentativeness are positively correlated with each other at clus-
ter level (r = 0.433, p < 0.01). In order to rule out that this
correlation is significant only because individuals use words of
certainty and tentativeness at the same time, we also tested the
correlation between these two variables at tweet level. Because
we have large sample size, they are still significantly correlated
(r = 0.061, p < 0.001). However, this correlation is very weak
in the tweet level, which means among the 27,023 tweets we
analyzed, an extremely small number of tweets used both
certainty and tentative ness words together.
Our manual content analysis shows that clusters with pop-
ular pro-vaccine tweets contain more positive emotion than
negative emotion. Clusters with popular anti-vaccine tweets
contain more negative emotion and less positive emotion.
Additionally, clusters with popular pro-vaccine tweets con-
tained institutional sources such as health organizations and
news media. While clusters with popular anti-vaccine tweets
mostly contained sources with questionable credibility such as
tweets which were being generated by users with no affiliation
to known institutions or known expertise. Examples of
sources which were coded as institutional were academic
journals, NIH, Huffington Post, and interviews with doctors
and Harvard Public Health. On the other hand, sources were
considered questionable when the link to the tweets did not
work, there was no expert sources were used, or site names
such as “Passive Aggressive Raven” were used.
Discussion
As individuals increasingly rely on social media for health-
related information, understanding content characteristics
and patterns of content flow are of great importance for
academics and practitioners. The growing volume of shared
content also poses challenges. This study applied social net-
work analysis to identify smaller, self-formed, subgroups of
users who interact primarily with one another, confiding
information flow. These sub-communities were found to
have common characteristics regarding their approach to the
HPV vaccine and the sources they used. Computer-aided
content and sentiment analysis were also applied to explore
the high volume of content, within these unique subgroups of
users. Lastly, acknowledging the limitation of automated con-
tent analysis, we applied conventional human coding to iden-
tify in more depth the nature of the sentiment and key sources
shared. Our approach and findings have important conceptual
and practical implications.
Network clusters proved to be a meaningful unit of analy-
sis for this study. The literature of homophily (McPherson
et al., 2001) led us to expect similarity of user and content
characteristics within clusters, and dissimilarities across clus-
ters. The findings addressed a gap in the literature in terms of
the context—health communication as well as the type of the
shared content such as sentiment, certainty, and shared linked
content (posted URLs). Furthermore, we found that difference
in network structure (i.e., the density of connections) was
associated with exhibited content characteristics. Denser clus-
ters of users were found to express more positive emotions in
their tweets, while the sparser clusters were angrier in content.
Together with findings that users in denser clusters also
expressed both certain and uncertain content, we can draw
some meaningful insights.
The density of interconnectedness has long been associated
with the speed of information flow (Zubcsek et al., 2014) and
the strength of ties (Granovetter, 1973). Our findings show
that users forming higher interconnected clusters expressed
a more positive tone, while more angry content was associated
with more loosely interconnected users. In other words, posi-
tive HPV vaccine-related content flows faster and among
interconnected user. Angry content flows from a single source
to loosely interconnected groups of users. A further ad-hoc
investigation of key tweets by cluster demonstrated that clus-
ters with favorite pro-vaccine tweets contained more positive
emotion and tweets included links primarily to institutional
and verified sources, such as the CDC, media, and physicians.
In contrast, clusters associated with more negative content
were associated with more anti-vaccine content posting
hyperlinks to unverified sources on the web.

Furthermore, dense clusters brought together both certain
and uncertain users—a finding that was tested and validated
on the cluster and tweet levels. This is an encouraging finding
suggesting that those who search for information, the uncer-
tain users (using words like maybe, fairly, perhaps) and those
who provide the information, the certain users (using of
words like always, absolutely, clearly) interact with one
another for information.
These findings can potentially have meaningful implica-
tions for health communicators. First and foremost, this study
proposes a method for finding a unique set of users who
naturally form clusters of conversations. Positive messages
flow better within subgroups of users who interact more and
exchange content with one another and are also associated
with pro-vaccine and reliable sources. The content denser
interactions were also found to host the certain and uncertain
users. Identifying the denser subgroups of users, practitioners
can reach individuals who are eager to exchange information,
in a productive manner, relying on credible sources.
Communicators may evaluate content and concerns, among
users in such clusters, in determining message strategy, know-
ing that they are more likely to reach an audience that is ready
to listen and possibly consider their content. A denser com-
munity, then, will benefit from a community-based approach,
where a relationship is developed, and trust is built over time
with a group of people. In contrast, the angry content may be
shared a lot, but it is not discussed. Practitioners may need to
evaluate further the nature of the agreed content to determine
the best intervention strategy. They may approach and correct
an inaccurate source, mentioning them in a post. They may
choose to ignore, not to give credibility to conspiracy theories,
for instance. Such low-density clusters tend to be less stable,
either because they rely on one source of information or
because of overall sparse exchange of opinions. The interven-
tion approach for angrier and low-density clusters, if deemed
appropriate, will be hub-focused, as the conversation does not
exist. In other words, identifying a focal source of content—if
it exists—and responding to it with counter-message. Our
approach helps communicators find the right groups of
users to engage with.
As many public opinion scholars indicated, people may
report their attitudes on issues very differently when asked
by researcher or pollster compared to what they would say in
a natural environment where they talk to others about the
same issue (e.g., Cramér, 2016; Zaller, 1992). Knowing that
individuals will engage in more opinion/information
exchanges when there are more positive sentiments may
help health practitioners better communicate risk and benefits
related to the HPV vaccine. Providers should use positively
framed language to engage patients in the conversation about
the HPV vaccine. Secondly, comparing cluster level analysis
and tweet level analysis, we can see that cluster brought
people who are certain about the HPV vaccine with people
who are on the fence together. Even though people with more
certain linguistic style may not necessarily know more than
others, previous research shows that they are indeed more
influential during group deliberation. While discussing health
issues, addressing uncertainty is critical for effective commu-
nication. People who have answers, not necessarily the correct
HEALTH COMMUNICATION 7
ones, and people who are on the fence naturally form HPV
vaccine discussion groups on Twitter. These findings may
help scholars to judge whether Twitter is a viable platform
for group deliberation.
Like all research, our paper comes with some limitations.
We checked these patterns with one topic, HPV vaccine.
Examining other health issues will help in a broader under-
standing of health communication in forums such as Twitter.
Twitter is also a dynamic platform. The findings from this
study could be different if other time frames were chosen. The
sentiment analysis also refers to those particular time frames.
So, the generalization of the results to different times and
topics should be made with caution. Furthermore, the break-
down of negative emotions yielded meaningful insights, how-
ever, the positive sentiment is rarely examined as a set of
individual emotions. Further analysis may explore the role
of personal positive emotions. Despite some of these limita-
tions, our study helps in the understanding of the commu-
nication dynamics within the groups in terms of group
members’ affective and cognitive characteristics in the HPV
vaccine. These analyses are important to understand the con-
versations related to the topic. Considering how social media
has become one of the most common ways to get health
information, understanding these platforms is fundamental.
References
Almack, J. C. (1922). What to teach in history? Historical Outlook, 13(5),
170–179.
Bae, Y., & Lee, H. (2012). Sentiment analysis of twitter audiences:
Measuring the positive or negative influence of popular twitterers.
Journal of the American Society for Information Science and
Technology, 63(12), 2521–2535. doi:10.1002/asi.22768
Barabas, J. (2004). How deliberation affects policy opinions. American
Political Science Review, 98(4), 687–701. doi:10.1017/s0003055404041425
Bott, E. A. (1928). Teaching of psychology in the medical course. Bulletin
of the Association of American Medical Colleges, 3(4), 289–304.
doi:10.1097/00001888-192810000-00001
Briones, R., Nan, X., Madden, K., & Waks, L. (2012). When vaccines go
viral: An analysis of HPV vaccine coverage on YouTube. Health
Communication, 27(5), 478–485. doi:10.1080/10410236.2011.610258
Calloway, C., Jorgensen, C. M., Saraiya, M., & Tsui, J. (2006). A content
analysis of news coverage of the HPV vaccine by U.S. newspapers,
January 2002–June 2005. Journal of Women’s Health, 15(7), 803–809.
doi:10.1089/jwh.2006.15.803
Carley, K., & Wendt, K. (1991). Electronic mail and scientific commu-
nication: A study of the SOAR extended research group. Knowledge,
12(4), 406–440. doi:10.1177/107554709101200405
Carrington, P. J., Scott, J., & Wasserman, S. (Eds.). (2005). Models and
methods in social network analysis. New York, NY: Cambridge
University Press.
Centers for Disease Control and Prevention. (2018). Cancers associated
with human papillomavirus (HPV). Retrieved from https://www.cdc.
gov/cancer/hpv/basic_info/cancers.htm
Christy, S. M., Winger, J. G., Raffanello, E. W., Halpern, L. F., Danoff-
Burg, S., & Mosher, C. E. (2016). The role of anticipated regret and
health beliefs in HPV vaccination intentions among young adults.
Journal of Behavioral Medicine, 39(3), 429–440. doi:10.1007/s10865-
016-9716-z
Clauset, A., Newman, M. E., & Moore, C. (2004). Finding community
structure in very large networks. Physical Review E, 70(6). doi:10.1103/
physreve.70.066111
Conover, M. D., Goncalves, B., Ratkiewicz, J., Flammini, A., &
Menczer, F. (2011, October 9–11). Predicting the political alignment
of Twitter users. In 2011 IEEE Third International Conference on
8 I. HIMELBOIM ET AL.
Privacy, Security, Risk and Trust and 2011 IEEE Third International
Conference on Social Computing: Proceedings, PASSAT/SocialCom
2011, Boston, MA. doi:10.1109/PASSAT/SocialCom.2011.34
Coviello, L., Sohn, Y., Kramer, A. D., Marlow, C., Franceschetti, M.,
Christakis, N. A., & Fowler, J. H. (2014). Detecting emotional con-
tagion in massive social networks. PLoS ONE, 9(3), e90315.
doi:10.1371/journal.pone.0090315
Cramér, H. (2016). Mathematical methods of statistics (PMS-9).
Princeton University Press.
Ekman, P. (1992). An argument for basic emotions. Cognition &
Emotion, 6(3–4), 169–200. doi:10.1080/02699939208411068
Fox, S., & Duggan, M. (2012). Mobile health has found its market:
Smartphone owners. Retrieved from http://www.pewinternet.org/
2012/11/08/mobile-health-2012/
Gollust, S. E., LoRusso, S. M., Nagler, R. H., & Fowler, E. F. (2016).
Understanding the role of the news media in HPV vaccine uptake in
the United States: Synthesis and commentary. Human Vaccines &
Immunotherapeutics, 12(6), 1430–1434. doi:10.1080/21645515.2015.
1109169
Gonçalves, P., Araújo, M., Benevenuto, F., & Cha, M. (2013). Comparing
and combining sentiment analysis methods. In Proceedings of the first
ACM conference on online social networks (pp. 27–38). doi:10.1145/
2512938.2512951
Granovetter, M. (1973). The strength of weak ties. American Journal of
Sociology, 78(6), 1360–1380. doi:10.1086/225469
Hilton, S., Hunt, K., Langan, M., Bedford, H., & Petticrew, M. (2010).
Newsprint media representations of the introduction of the HPV
vaccination programme for cervical cancer prevention in the UK
(2005–2008). Social Science & Medicine, 70(6), 942–950. doi:10.1016/
j.socscimed.2009.11.027
Himelboim, I. (2017). Social network analysis (Social media). In
J. Matthes, C. S. Davis, & R. F. Potter (Eds.), The international encyclo-
pedia of communication research methods (pp. 1–15). Hoboken, NJ:
John Wiley & Sons. doi:10.1002/9781118901731.iecrm0236
Himelboim, I., & Han, J. Y. (2014). Cancer talk on Twitter: Community
structure and information sources in breast and prostate cancer social
networks. Journal of Health Communication, 19(2), 210–225.
doi:10.1080/10810730.2013.811321
Himelboim, I., McCreery, S., & Smith, M. (2013). Birds of a feather tweet
together: Integrating network and content analyses to examine
cross-ideology exposure on Twitter. Journal of Computer-Mediated
Communication, 18(2), 154–174. doi:10.1111/jcc4.12001
Kahan, D. M., Braman, D., Cohen, G. L., Gastil, J., & Slovic, P. (2010).
Who fears the HPV vaccine, who doesn’t, and why? An experimental
study of the mechanisms of cultural cognition. Law and Human
Behavior, 34(6), 501–516. doi:10.1007/s10979-009-9201-0
Keelan, J., Pavri, V., Balakrishnan, R., & Wilson, K. (2010). An analysis of
the human papilloma virus vaccine debate on myspace blogs. Vaccine,
28(6), 1535–1540. doi:10.1016/j.vaccine.2009.11.060
Krakow, M., & Rogers, B. (2016). Collateral damage and critical turning
points: Public health implications of HPV vaccine news coverage for
boys and men in 2011. Health Communication, 31(9), 1081–1088.
doi:10.1080/10410236.2015.1038773
Loomis, C. P. (1946). Political and occupational cleavages in
a Hanoverian village, Germany: A sociometric study. Sociometry, 9
(4), 316–333. doi:10.2307/2785496
Markowitz, L. E., Dunne, E. F., Saraiya, M., Chesson, H. W.,
Curtis, C. R., Gee, J., … Unger, E. R. (2014). Morbidity and mortality
weekly report: Human papillomavirus vaccination: Recommendations
of the advisory committee on immunization practices (ACIP) (Report
No. 63(RR05)). Retrieved from Center for Disease Control and
Prevention website: https://www.cdc.gov/mmwr/preview/mmwrhtml/
rr6305a1.htm
Massey, P. M., Leader, A., Yom-Tov, E., Budenz, A., Fisher, K., &
Klassen, A. C. (2016). Applying multiple data collection tools to quantify
human papillomavirus vaccine communication on Twitter. Journal of
Medical Internet Research, 18(12), e318. doi:10.2196/jmir.6670
McPherson, M., Smith-Lovin, L., & Cook, J. M. (2001). Birds of a feather:
Homophily in social networks. Annual Review of Sociology, 27,
415–444. doi:10.1146/annurev.soc.27.1.415
Mo, P. K., & Coulson, N. S. (2008). Exploring the communication of social
support within virtual communities: A content analysis of messages
posted to an online HIV/AIDS support group. Cyberpsychology &
Behavior, 11(3), 371–374. doi:10.1089/cpb.2007.0118
Myrick, J., Holton, A., Himelboim, I., & Love, B. (2016). #Stupidcancer:
Exploring a typology of social support and the role of emotional
expression in a social media community. Health Communication, 31
(5), 596–605. doi:10.1080/10410236.2014.981664
Newman, M. E. (2003). The structure and function of complex networks.
SIAM Review, 45(2), 167–256. doi:10.1137/S003614450342480
Oh, H. J., Lauckner, C., Boehmer, J., Fewins-Bliss, R., & Li, K. (2013).
Facebooking for health: An examination into the solicitation and
effects of health-related social support on social networking sites.
Computers in Human Behavior, 29(5), 2072–2080. doi:10.1016/j.
chb.2013.04.017
Owens, R. J., & Wedeking, J. P. (2011). Justices and legal clarity: Analyzing
the complexity of U.S. supreme court opinions. Law & Society Review, 45
(4), 1027–1061. doi:10.1111/j.1540-5893.2011.00464.x
Pereira, D. B., Antoni, M. H., Danielson, A., Simon, T., Efantis-Potter, J., &
O’Sullivan, M. J. (2004). Inhibited interpersonal coping style predicts
poorer adherence to scheduled clinic visits in human immunodeficiency
virus infected women at risk for cervical cancer. Annals of Behavioral
Medicine, 28(3), 195–202. doi:10.1207/s15324796abm2803_8
Richardson, H. M. (1940). Community of values as a factor in friendships
of college and adult women. The Journal of Social Psychology, 11(2),
303–312. doi:10.1080/00224545.1940.9918751
Robinson, S., & Bryan, S. (2013). Does the process of deliberation change
individuals’ health state valuations? An exploratory study using the
person trade-off technique. Value in Health : The Journal of the
International Society for Pharmacoeconomics and Outcomes Research,
16(5), 806–813. doi:10.1016/j.jval.2013.03.1633
Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L.,
Ramones, S. M., Agrawal, M., & Ungar, L. H. (2013). Personality,
gender, and age in the language of social media: The open-vocabulary
approach. PLoS ONE, 8(9), e73791. doi:10.1371/journal.pone.0073791
Shepherd, M. A., & Gerend, M. A. (2014). The blame game: Cervical cancer,
knowledge of its link to human papillomavirus and stigma. Psychology &
Health, 29(1), 94–109. doi:10.1080/08870446.2013.834057
Sherris, J., Friedman, A., Wittet, S., Davies, P., Steben, M., & Saraiya, M.
(2006). Chapter 25: Education, training, and communication for HPV
vaccines. Vaccine, 24(S3), 210–218. doi:10.1016/j.vaccine.2006.05.124
Stieglitz, S., & Dang-Xuan, L. (2012). Political communication and influ-
ence through microblogging–An empirical analysis of sentiment in
Twitter messages and retweet behavior. In 45th Hawaii International
Conference on System Sciences (pp. 3500–3509). doi:10.1109/
hicss.2012.476
Sunstein, C. R. (2005). Group judgments: Deliberation, statistical means,
and information markets. N.Y.U. Law Review, 80(3), 962–1049.
Tausczik, Y. R., & Pennebaker, J. W. (2010). The psychological meaning
of words: LIWC and computerized text analysis methods. Journal of
Language and Social Psychology, 29(1), 24–54. doi:10.1177/
0261927x09351676
Tumasjan, A., Sprenger, T. O., Sandner, P. G., & Welpe, I. M. (2011).
Election forecasts with Twitter: How 140 characters reflect the poli-
tical landscape. Social Science Computer Review, 29(4), 402–418.
doi:10.1177/0894439310386557
Wartella, E., Rideout, V., Montague, H., Beaudoin-Ryan, L., & Lauricella, A.
(2015). Teens, health and technology: A national survey. Media and
Communication, 4(3), 13. doi:10.17645/mac.v4i3.515
Wasserman, S., & Faust, K. (1994). Social network analysis: Methods and
applications. New York, NY: Cambridge University Press.
Weaver, J. (2013). More people search for health online. Retrieved from
http://www.nbcnews.com/id/3077086/t/more-people-search-health-
online/#.XDLyjlxKg2x
Wright, K. B., Bell, S. B., Wright, K. B., & Bell, S. B. (2003). Health-related
support groups on the Internet: Linking empirical findings to social
support and computer-mediated communication theory. Journal of
Health Psychology, 8(1), 39–54. doi:10.1177/1359105303008001429
Yoo, W., Namkoong, K., Choi, M., Shah, D. V., Tsang, S.,
Hong, Y., … Gustafson, D. H. (2014). Giving and receiving

emotional support online: Communication competence as
a moderator of psychosocial benefits for women with breast cancer.
Computers in Human Behavior, 30, 13–22. doi:10.1016/j.chb.2013.
07.024
Zaller, J. R. (1992). The nature and origins of mass opinion. New York,
NY: Cambridge University Press.
HEALTH COMMUNICATION 9
Zhang, Y., He, D., & Sang, Y. (2013). Facebook as a platform for health
information and communication: A case study of a diabetes group.
Journal of Medical Systems, 37(3), 9942. doi:10.1007/s10916-013-9942-7
Zubcsek, P. P., Chowdhury, I., & Katona, Z. (2014). Information com-
munities: The network structure of communication. Social Networks,
38, 50–62. doi:10.1016/j.socnet.2014.02.001