See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/234037764

Climate Change Discourse in Mass Media: Application of Computer-Assisted

Content Analysis

Article in Journal of Environmental Studies and Sciences · June 2012

DOI: 10.1007/s13412-012-0074-z

CITATIONS READS

49 1,840

2 authors:

Andrei P Kirilenko Svetlana Stepchenkova

University of Florida University of Florida
129 PUBLICATIONS 4,154 CITATIONS 91 PUBLICATIONS 5,714 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Andrei P Kirilenko on 03 June 2014.

The user has requested enhancement of the downloaded file.

J Environ Stud Sci
DOI 10.1007/s13412-012-0074-z
Climate change discourse in mass media: application
of computer-assisted content analysis
Andrei P. Kirilenko & Svetlana O. Stepchenkova
# AESS 2012
Abstract Content analysis of mass media publications has
become a major scientific method used to analyze public
discourse on climate change. We propose a computer-
assisted content analysis method to extract prevalent themes
and analyze discourse changes over an extended period in an
objective and quantifiable manner. The method includes the
following: (1) sample selection; (2) preparation of the text
segments for computer processing; (3) identifying themes in
the texts using exploratory factor analysis; (4) combining
identified themes into higher order themes using confirmatory
factor analysis; (5) using factor scores to interpret themes
obtained from public discourse; and (6) tracking the main
themes of public discourse through time. We apply the pro-
posed methodology to the analysis of the articles published in
the New York Times on climate change during the period from
1995 to 2010. We found a gradual decline in the volume of
material within the “Science” topic and an expansion of
themes classified under the “Politics” topic.
Keywords Climate change . Global warming . Mass media .
CATA . Content analysis
Electronic supplementary material The online version of this article
(doi:10.1007/s13412-012-0074-z) contains supplementary material,
which is available to authorized users.
A. P. Kirilenko (*)
Department of Earth Systems Science and Policy,
University of North Dakota,
Stop 9011,
Grand Forks, ND 58202-9011, USA
e-mail: andrei.kirilenko@und.edu
S. O. Stepchenkova
The Department of Tourism, Recreation and Sport Management,
University of Florida,
P.O. Box 118208, Gainesville, FL 32611-8208, USA
Introduction
Public perceptions of climate change are significantly influ-
enced by mass media (Wilson 1995). Content analysis of
newspaper publications, TV news, and, recently, Internet
blogs has allowed for the identification of major discussion
themes within the climate change domain (e.g., newspaper
trends, comparison of climate change discourse in different
countries, and contrasting liberal vs. conservative press).
The majority of these studies, however, have manually
processed the texts, limiting textual population size, restrict-
ing the analysis to a relatively small number of themes, and
using time-expensive coding procedures. The use of
computer-assisted text analysis (CATA) software is impor-
tant because the difficulties with manual processing become
more severe with an increased volume of data (Macnamara
2006; Wickham and Woods 2005). To overcome these dif-
ficulties, a CATA approach that allows a large body of text
materials to be surveyed in a quantifiable, objective, trans-
parent, and time-efficient manner is proposed. While staying
within the quantitative tradition of content analysis, the
approach allows for an interpretation of the public discourse
closer to one of more qualitatively oriented methods. This
approach is a further development of the method initially
introduced in (Kirilenko et al. 2012).
Study background
To date, multiple authors have examined media representations
of and public attitudes towards climate change, the impacts of
climate change on the environment, society and economics,
mitigation options and mitigation measures, as well as how
these representations and attitudes differ from scientific knowl-
edge on the issue of climate change (Boykoff and Boykoff

2004; Bord et al. 1998; Henry 2000; Leiserowitz 2006;
Kellstedt et al. 2008; Krosnick et al. 2006; Weber 2006).
Public surveys have provided a direct measurement of the
development of peoples’ attitudes towards the problem of
climate change since the early 1980s (Bord et al. 1998).
Starting with J. Hansen’s congressional testimony in 1988,
when the topic of global warming first appeared in the US
mass media as a “scientifically proven” phenomenon, national
surveys have demonstrated that the public tends to place global
warming at the bottom of the list of the country’s problems
(Krosnick et al. 2000). Meanwhile, scientific compendiums,
such as Intergovernmental Panel on Climate Change (IPCC)
publications, present a scientific consensus on the high impor-
tance of the problems associated with climate change. In
studies exploring the differences between public discourse
and scientific knowledge, scientific understanding of climate
change is mainly derived from expert surveys and an analysis
of the scientific literature. Oreskes (2004) analyzed the
abstracts of 928 peer-reviewed articles and found that 75 %
of the articles confirmed that the changes in climate we are
currently experiencing are anthropogenically driven, 25 % of
the articles dealt with neutral subjects (e.g., paleoclimate), and
none of the publications rejected the hypothesis of man-made
climate change. In an even larger attempt to test the scientific
consensus on climate change, Doran and Zimmerman (2009)
distributed a survey questionnaire to 10,257 Earth System
scientists; 82 % of 3,146 respondents answered “yes” to the
question: “Do you think human activity is a significant con-
tributing factor in changing mean global temperatures?”
Public opinion is influenced by a number of factors. Brulle
et al. (2012) in a recent study of 74 surveys of public concerns
about climate change found that among the analyzed factors
presumably influencing public concerns, (1) mass media cov-
erage, (2) elite cues and (3) structural economic factors were
the most important. The agenda-setting theory (McCombs and
Shaw 1972) defines the role of mass media in affecting public
attitudes as follows: “The news media do not just passively
transmit information, repeating verbatim the words of a public
official or conveying exactly the incidents at an event. Nor do
they select and reject the day’s news in proportion to reality.
Through their day-to-day selection and display of the news,
editors and news directors focus attention and influence the
public perceptions of what are the most important issues of the
day” (McCombs and Bell 1996, pg. 93). (For a review of the
agenda-setting theory studies, refer to McCombs (2004)).
Brulle et al. (2012) maintain that even though 83 % of the
variation in the variable responsible for mass media coverage
of climate change is explained by the factors from the other
two groups, both the media coverage and elite cues are ex-
tremely influential in driving public perceptions of climate
change impacts.
Multiple studies have been conducted on mass media
coverage of climate change. Boykoff and Boykoff (2004),
J Environ Stud Sci
in a content analysis of 636 news articles from New York
Times, the Washington Post, the Los Angeles Times, and the
Wall Street Journal, demonstrated a considerable divergence
between the coverage of global warming in the mass media
and scientific publications. These same authors (2007) ex-
amined a sample of news abstracts from three major US TV
news channels in addition to the news publications listed
above, and found that journalistic norms lead to the deficit
in mass media information on the issue of climate change.
Antilla (2005) performed a cross-sectional study of the
scientific topics addressed in 544 articles containing the
words “global warming” and “climate change” that were
published in 251 US newspapers during 1 year (March 1,
2003 to February 29, 2004) and found 32 unique scientific
themes, which were further condensed into six categories.
These and other studies on the newspaper coverage of
climate change (Boykoff 2007; Bell 1994; Liu et al. 2008;
Billett 2010; Trumbo 1996) are generally based on manual
processing of large textual blocks, following the methodology
of content analysis (Neuendorf 2002). These chunks of text
are commonly classified in terms of belonging to a pre-
determined set of categories (e.g., rejection or acceptance of
climate change), even though a limited number of studies
attempt to extract the discussion themes from relatively small
text populations (Antilla 2005). We offer a more formal pro-
cedure that would replace several of the steps of content
analysis with automated text processing and provide the re-
searcher with quantitative support at each step of the analysis.
Methodology
The methodology used in this study contains several steps: (1)
sample selection; (2) data preparation for computer processing
and obtaining a matrix of keyword frequencies; (3) identifi-
cation of themes in the texts using exploratory factor analysis
(EFA); (4) combining identified themes into higher order
themes using confirmatory factor analysis (CFA); (5) interpre-
tation of obtained public discourse themes using factor scores;
and (6) tracking the development of the main themes of the
climate change discourse through time.
Within the paradigm of the content analysis methodolo-
gy, this study is in line with the correlational tradition of
content analysis (as opposed to the substitution tradition).
The correlational tradition discerns categories in which the
text is coded from the text itself, rather than assigning it to
pre-established categories. In this tradition, categories are
“groups of words with different meaning or connotations
that taken together refer to some theme or issue” (Weber
1983, pg. 140). These themes, or categories, are extracted by
factor analysis (Iker 1974) or other multidimensional scaling
techniques (Iker and Harway 1965) from the matrix of word
covariations (Weber 1986). One can follow the debate on the
J Environ Stud Sci
theoretical soundness and comparative advantages of correla-
tional and substitution tradition in Weber (1983) and Muskens
(1985). The proposed method is time-effective and suitable
for surveying large bodies of textual data; however, as
Neuendorf (2002) rightfully noted, no content analysis is
completely automated, and human involvement is still a cru-
cial part of any research project. The approach relies more
heavily on quantitative indicators than have previously used
approaches. These indicators help researchers retain their
objectivity, provide transparency, and can be used to judge
the “quality” of the obtained results.
Sample selection
The consistent coverage of climate change in the US mass
media started with James Hansen’s congressional testimony in
1988 (e.g., a LexisNexis search for “climate change” or
“global warming” in The New York Times (further NYT)
returned, on average, 43 articles per year in 1983 to 1987
and 258 articles per year for the period of 1988 to 1992).
However, the consensus of the scientific community on the
anthropogenic drivers of current climate change was only
established in the 1995 Second Assessment Report (SAR) of
the IPCC.1 To study the progress in the mass media discussion
of climate change since the establishment of the scientific
consensus on the topic, we selected publications from the
NYT from January 1, 1995 to December 31, 2010. The
NYT was selected based on the concept of “prestige press”
(Stovall 1961), which advocates that the newspaper is consid-
ered to be the most “influential” and “substantially signifi-
cant” of information sources currently available (Lacy et al.
1991; Stempel and Windhauser 1984). As McCombs (2004,
pg. 113) noted, “The elite news media frequently exerts a
substantial influence on the agendas of other news media. In
the United States this role of inter-media agenda-setter is
frequently played by the NYT, a role now so institutionalized
that the Associated Press alerts its members each day to the
agenda of stories scheduled for the next morning’s front page
of the Times.” During the period stated above, articles that
contained the words “climate change” or “global warming”
were downloaded from the LexisNexis Academic database,
and duplicates were removed.2 The search terms that were
chosen are frequently used to extract news text discussing
1
Compare the wording “The balance of evidence suggests that there is
a discernible human influence on the global climate” in SAR with
“Thus the observed [temperature] increase could be largely due to this
natural [climate] variability; alternatively this variability and other
human factors could have offset a still larger human-induced green-
house warming” in the 1990 IPCC First Assessment Report (FAR).
2
LexisNexis is the most frequently used database in the social sciences
(Deacon 2007). However, it does not contain all items that appeared in
the “physical” version of some of the newspapers (Deacon 2007), and
some of the entries were duplicates.
climate change (Antilla 2005; Boykoff and Boykoff 2004,
2007). Using this popular approach, we extracted 10,467
articles, many of which only briefly mentioned climate change
and mostly concentrated on other topics.3 To decrease the
number of articles in which climate change was only occa-
sionally mentioned, we removed articles in which the words
“climate change” or “global warming” occurred only once.
The number of “false positives” (i.e., articles that were in the
sample but focused on topics other than climate change) was
estimated as 2 % (Appendix A). The final text sample
contained 4,043 articles.
Data preparation
The data preparation stage of the analysis included reducing
the articles into textual units centered around the keywords
“climate” and “warming” and constructing a dictionary of
the pertinent words needed to identify the themes in the
climate change discourse. We started processing the selected
articles by replacing the synonyms, followed by lemmatiza-
tion, and removal of stop words. For automation purposes,
we developed Python scripts using the Natural Language
Toolkit (NLTK, http://www.nltk.org) libraries. The text was
standardized by replacing the synonyms (e.g., United States,
U.S., U.S.A., USA, America→US). Then, we processed the
text using the NLTK Wordnet lemmatizer. Lemmatization
reduces inflectional word forms to a word’s lexical roots; as
a result, different inflected word forms can be converted to a
single variable. Then, we removed common words, or stop
words (e.g., “me”, “about”) using a list of 571 stop words
that were developed for the SMART project (Lewis et al.
2004). We extended the list of stop words to include fre-
quently used words that are not specific to the climate
change topic (e.g., member, show, or researcher). Then,
the text was processed with a Porter stemmer (Porter
1980), reducing the already modified text to contain only
word stems. Finally, the text was reduced further by remov-
ing all content not adjacent to the topic of our interest by
setting the windows centered at the words “warming” and
“climate” and removing the words outside of these win-
dows. On average, the processed text segments contained
98 words (10th percentile 41, 90th percentile 181). The
following provides an example of the result of the data
preparation.
Original text (NYT, Cloud data cast doubt about atmo-
sphere. January 30, 1995):
3
E.g., an article on non-for-profit organizations was selected based on
a sentence: “Starting with core campaigns in the 1970s to make the air
and water cleaner and protect natural resources, those groups in the
1980s established large and elaborate programs to address a wide array
of more scientifically complex and politically difficult issues like
global warming and saving obscure animals.” (Schneider, K., Big
Environment Hits a Recession. NYT, 1.1.1995)

The research raises questions about the way complex
computer models mimic the atmosphere. Those mod-
els are a mainstay in the concern about the theory that
gases released into the air could cause the planet to
overheat. The scientists are not sure if the new find-
ings mean they are overestimating or underestimating
the threat of warming. The researchers found that
clouds appear to absorb energy equivalent to burning
a 100-W light bulb for every three square meters of the
Earth’s surface a square roughly 5.6 ft on each side.
The discovery that so much energy is being absorbed
by clouds means that scientists to keep things in bal-
ance must reduce their estimate of the energy added to
the air by the water that evaporates from the oceans
lakes and rivers and then condenses into clouds.
After processing:
model mimic atmospher model mainstay theori ga air
overheat overestim underestim threat warm cloud ab-
sorb energi burn watt light bulb meter surfac discoveri
energi absorb cloud balance
It is important to note that the themes discussed in rela-
tion to climate change and, hence, the words used, vary over
time. Figure 1 demonstrates temporal variation in the num-
ber of articles on climate change published in the NYT. The
spikes on the graphs correspond to major news events: the
Kyoto conference, the beginning of the Bush presidency and
his first visit to Europe, the Genoa G8 Summit, the interna-
tional reaction to the American position on climate change,
the publication of IPCC Fourth Assessment Report (AR-4),
the Nobel Peace Prize being awarded to the IPCC and Al
Gore, and Climategate. We used the dates of these events to
split the article database into six parts: January 1995 to
September 1997 (141 articles), October 1997 to May 2001
(474 articles), June 2001 to December 2006 (1,116 articles),
Fig. 1 The number of articles (per month) on climate change that were
published in the New York Times from January 1995 until December
2010. The major news events (marked with arrows) were used to break
the dataset into six periods to extract the most frequent words for each
period
J Environ Stud Sci
January 2007 to December 2007 (678 articles), January
2008 to October 2009 (863 articles), and November 2009
to December 2010 (492 articles), so that each period would
start within a month of a major “global warming” news
event.
To construct a dictionary of the words pertinent to the
identification of themes in the climate change discourse, we
formed lists of the most frequently used words in the text
corpus for each period separately and then merged the lists
into one single dictionary. After the list was refined, the final
master dictionary included the 623 most frequent keywords
(variables) used in the NYT 1995 to 2010 publications on
climate change.
Exploratory factor analysis
After we obtained a list of the most frequent keywords
(variables) from the 1995 to 2010 climate change discourse,
we calculated the frequency of each variable in each text
segment. The matrix that was generated was further pro-
cessed with EFA (Principal component analysis, Varimax
rotation) to cluster the variables into interpretable factors
that were regarded as themes in the climate change dis-
course (Kirilenko et al. 2012). Initially, we focused on
obtaining an interpretable solution without limiting the
number of factors, which was done in a series of iterative
steps. Each step produced clusters of variables of the fol-
lowing four types: (1) full meaningful factors4; (2) stable
word combinations5; (3) factors with only one variable6; and
(4) variables that were not strongly loaded on any factor.7
We aimed to keep only the high face validity factors, which
clearly define the topics and help avoid ambiguity, by using
the following guidelines: (a) each factor should contain at
least three variables (Kline 1994); (b) all cross-loadings
should be meaningful; and (c) the solution should be stable
(i.e., the solutions with a smaller or a larger number of
factors made few changes). Variables from group (4) were
removed. Variables with low factor loadings, low commu-
nalities, or variables that were spread across multiple factors
were also removed (Kline 1994). Some of the stable word
combinations from group (2) were removed because they
4
An example of a full factor would be the Kyoto Protocol (Cronbachs’
α0.74), which combines the following variables: Kyoto, protocol,
treaty, ratification, ratify, reject, Russia. For clarity, we provide words
from here on, but in reality the variables contained word stems only.
5
Examples of stable word combinations would be NAVI and ship or
Gulf and coast. Though closely associated, these words did not define a
topic and could not be meaningfully interpreted.
6
Some of the factors identified by a single variable were lacking other
variables that were too infrequent in the original texts. The words that
might otherwise “fill in” were not included in the variable list due to an
arbitrary level of word frequency cutoff.
7
Some variables (e.g., environment) were too general to associate with
any particular factor alone.
J Environ Stud Sci
did not aid in topic identification. Multiple iterations of EFA
were run until the solution contained no one-variable factors
(119 factors, 76 factors explained 50 % of the variance).
However, there were multiple factors from group (2) (e.g.,
biological and diversity) that we wanted to break and force the
variables from these factors into interpretable themes.
Therefore, we began to gradually reduce the number of factors
with the goal of coming to a solution that would follow the a–
b–c criteria defined above. The final 76-factor EFA solution
had the following characteristics: the KMO statistic value for
the matrix was .74, which exceeds the minimal recommended
value of .6 (Brace et al. 2003); Bartlett’s test of sphericity:
significance .000; the number of factors was 76, the variance
explained 50.2 %, the range of factor loading was .35 to .91.
The factors/themes obtained according to the procedure above
were interpreted (see section “Interpretation of climate change
discourse themes”) and used in the CFA procedure to organize
these themes into topics of a higher order.
Confirmatory factor analysis
The CFA procedure was conducted using AMOS software.
Our goal at this stage of the analysis was to organize correlated
factors/themes obtained by the EFA procedure into higher
order climate change discourse topics. For example, we noted
that several factors (e.g., the Copenhagen UNFCCC
Conference of Parties (COP) and the Kyoto Protocol) were
correlated and could be sorted into a Politics topic. By apply-
ing the CFA procedure, we aimed to reduce the data into an
even smaller set of interpretable topics. We removed variables
and factors that produced errors (e.g., negative variance) or
had low regression weights (below .2). The final CFA struc-
ture is presented as Table 1 (the variables loading to factors are
not shown). The solution had the following characteristics:
& The case to variable ratio is 14:1, which is slightly below
the ideal ratio of 20:1 but well above the minimum
recommended ratio of 5:1 (Byrne 2009),
& Chi-square 156,033; degrees of freedom 40,361; with a
ratio of 3.9 (recommended minimum range from 2.0 to
5.0 (Hooper et al. 2008)),
& RMSEA .028 (recommended <.05 (Byrne 2009)),
& Standardized root mean square residual (SRMR) .021
(recommended <.05 (Meyers et al. 2006)),
& Hoelter index (.05 significance level) 985 (recommended
>200 (Byrne 2009)),
& PGFI .74 (recommended >.5 (Byrne 2009)),
& PCFI .63 (recommended >.5 (Meyers et al. 2006)).
Index values and good model interpretability (discussed
in section “Results”) indicated an acceptable fit. In the CFA
model, we retained 56 factors that were regarded as major
topics in climate change discourse from 1995 to 2010 (first
column of Table 1).
Results
Interpretation of climate change discourse themes
To interpret the factors, we used text segments factor scores
obtained during EFA. Factor scores indicate how influential
each case is with respect to each particular factor and can be
used to determine the case’s relative standing on the latent
dimension (Brown 2006). The text segments were ranked
according to their factor scores, and then the top five text
segments for each factor were used to interpret that factor.
For example, the variables vostok (.91), mayewski (.90),
intergaci[al] (.78), glaciat[ion] (.75), paleoclimatologist
(.65), oxygen (.59), sampl[e] (.49), and iceag[e] (.41) (the
numbers in parentheses are factor loadings) load on one
factor that, after considering the top five text segments,
was named “Paleoclimatology”. Appendix B provides the
five segments with the highest factor scores to illustrate the
process (note that only a fragment of each segment prior to
lemmatization is given).
We interpret the topics of the discourse (central column
of Table 1) based on the factors that load to the topics.
Altogether, we extracted the following 11 major topics:
1. Polar regions. A discussion of climate change in the
Arctic and Antarctica, the possible abrupt disintegration
of the WAIS and Greenland ice sheet, and the loss of
habitat for polar bears.
2. Physics of climate. A discussion of climatological re-
search, computer simulation of climate, the connection
between atmospheric aerosols, cloudiness, and albedo,
and the effect of solar activity change on surface tem-
perature (with multiple references to the public state-
ments of Dr. J. Hansen (NASA)).
3. Climate change impacts on coastal erosion, permafrost,
droughts and floods.
4. Scientific debates. Including the non-consensual views of
prominent scientists (e.g., Dr. J. Christy’s emphasis of the
problems with satellite data, Dr. R. Lindzen’s work on
atmospheric water vapor sensitivity) and the usage of
snowfall and ENSO to support/reject the hypothesis of a
warmer climate.
These four topics—polar regions, physics of climate,
climate change impacts, and scientific debates—are highly
correlated and, together with the factor of Arctic melt, were
combined into a mega-topic entitled “Science”.
5. Politics. Including articles on international negotiations,
USA domestic politics, carbon emission regulations and
their impact on economics, strategic development in the
energy sector, China GHG emissions and similar topics.
6. Food production and poverty. Including the impact of
climate change on agriculture, vulnerable population
 J Environ Stud Sci

Table 1 CFA model of climate change discourse, standardized regression weights. The variables are not shown

groups, adaptation, the results of the 2005 G-8 summit
and the Bush-Blair negotiations on aid to African
countries.
7. Global warming hoax. Including the recent IPCC AR-4
errors and Climategate and the articles challenging the
science behind climate change from non-scientific
positions.
8. Business responsibility and risks. Including the moral
obligations of businesses and their shareholders and the
risks presented by climate change to investors.
J Environ Stud Sci
9. Smog, car pollution. Including urban smog, nitrogen
oxide car and track emissions and similar articles.
10. Biodiversity, pests. Including the impact of climate
change on biodiversity, changes in species habitats,
and the impact of warmer climate on forest dieback
due to pine beetle infestation.
11. Religion, faith. Including the connection between the
legal challenges to teaching evolution at school and
CO2 regulations, gaps between the scientific knowl-
edge and public perception, climate skepticism (e.g.,
CO2 emissions are beneficial because evolution has
occurred in a CO2-rich atmosphere), climate variability
as a driver of evolution, and the moral obligation of
Christians to guard the planet and climate.
Some factors do not contribute to any higher topic. Most
of these topics pertain to a short period of few years within
the specified 16-year time frame. These factors (further–
terminal factors) are found at the bottom of Table 1 and
include:
1. Paleoclimatology, including past climates, Antarctica
and Greenland ice core drilling and ocean sediments.
The majority of articles8 were published in 1999–
2001; there were very few publications after 2008.
2. Infectious diseases, health impacts and malaria outbreaks.
Many of the contributing articles were published in 1996
to 1999 and discuss pre-Kyoto negotiations, a WHO 1997
report on infectious diseases, warmer temperatures at
higher altitudes, etc., in connection with spread of infec-
tious diseases, especially malaria.
3. Ozone layer, including the “ozone hole” and a com-
parison of the political issues surrounding the accep-
tance of the Montreal Protocol on Substances That
Deplete the Ozone Layer and climate change agree-
ments. Since 1995, the Protocol was revised three
times (1995, 1997, and 1999), and the success of the
Protocol has been frequently referred to in carbon
regulation discussions. Accordingly, the articles dis-
cussing this theme were published throughout the en-
tire period.
4. Alpine ski resorts: discussions of the impact of moun-
tain snow melt on ski resorts. The factor scores were
higher in 2005 to 2006 when the articles describing the
current effects of warmer than normal winters on al-
pine ski resorts were being published.
5. Dr. B. Santer. In 1996, an editorial in the Wall Street
Journal accused Dr. Santer, a Convening Lead Author
(CLA) of the 1995 IPCC SAR with unauthorized alter-
ations of his chapter. The articles were mainly pub-
lished in 1995–1997.
8
As measured by WFI (discussed in “Dynamics of climate change
discourse”).
6. Offshore oil drilling, the Gulf. The annual number
of articles is highly variable: the absolute majority
of the articles were published in 2010 (in relation to
the Gulf of Mexico BP oil spill), with a smaller
contribution in 2009 (related to the climate change
legislation), and 2004 (related to the climate change
and energy independence topic in the US Presidential
campaign).
7. Oil companies (BP, Exxon-Mobile). The articles on
this theme were published every year and cover a large
variety of aspects.
8. Communities. Kivalina (an Alaskan village impacted
by coastal erosion) lawsuit, and other articles on cli-
mate change-related lawsuits; the number of articles
has increased since 2002.
9. Nobel Peace prize to Al Gore, IPCC. The majority of
the articles appeared in 2007; since then, the number of
articles has been declining.
10. Hurricanes, storms. The relevant articles mainly discuss
the connection between warmer climate and hurricanes.
The number of articles starts growing in 2004 in con-
nection with hurricanes Charlie and Francis hitting
Florida and then peeks sharply in 2005 when hurricane
Katrina hit New Orleans. The factor has slowly de-
creased in importance since 2005.
11. Amazon deforestation. The articles with the highest
factor scores were published in 2008 to 2009 and
focused on Brazil’s efforts to reduce deforestation rates
and the economic incentives to logging.
12. Thermohaline circulation, including rapid climate
shifts due to alterations in Atlantic currents. The articles
mainly discuss the 1997–1999 papers and books by Dr.
Wallace Broecker and other authors on the abrupt
stop of thermohaline circulation. Accordingly, the
majority of the articles were published in 1997–
1999.
13. Biofuels, bioethanol. The majority of articles were
published after 2005 in connection with the discussion
of the energy bill the Senate, followed by the discus-
sion of ethanol extraction from corn and sugar cane,
and the food vs. fuel debate. Later, the discussion of
biofuels continued at a slower pace.
14. “Inconvenient Truth” documentary and other movies
on climate change. The articles published in 2006 and
later mainly discuss the “Inconvenient Truth” docu-
mentary. However, a considerable number of articles
contributing to this factor were published in 2004;
these articles discuss the climate change disaster movie
“The Day After Tomorrow”.
15. Acid rain. The articles discuss a combined impact of
acid rain and climate change on ecosystems.
16. Coral reefs. This factor combines two connected
themes: the impact of water pollutions, warmer and

more acid ocean on the coral reefs, and reconstruction
of past climates using the coral reefs data.
17. Property insurance loss, including losses due to ad-
verse weather events such as hurricanes and flooding,
impact of climate change on economy. The articles are
diverse in geography of discussed effects and appear
throughout the entire time period.
18. Indonesian deforestation. The factor becomes promi-
nent in 2007. The articles discuss Indonesian efforts
to secure incentives from industrialized nations to
reduce CO2 emissions, the role of the Indonesian
rainforest in storing carbon, the moratorium on forest
clearing, and similar topics. Additionally, some
articles discuss the international negotiations on en-
vironmental topics, such the COP meeting to replace
the Kyoto Protocol.
An interesting feature of the method, which is based
on word co-occurrences, is that semantically close factors
may have low correlations in the CFA model. For exam-
ple, even though the terminal factor “Paleoclimatology”
is semantically close to the group of factors within the
mega-topic “Science”, it has a correlation below .05 with
any factor in the Science mega-topic. Indeed, we found
that the articles on paleoclimate reconstructions tend to
isolate the discussion to the single theme. Another reason
for low correlation between the factors is the tendency of
some factors to be formed by publications that appear
within relatively short periods. For example, the articles
on Antarctic ice core drilling were mainly published
between 1996 and 1999, and the articles on the abrupt
stop in thermohaline circulation were published between
1997 and 1999. In Kirilenko et al. (2012), the factors
were combined into topics based on expert judgment
alone. In this study, however, the correlational and expert
approaches were combined by grouping the themes that
contained closely correlated factors with terminal factors
based on their semantic proximity. For example, in Fig. 2
(discussed in the next section), the closely correlated
factors from the mega-topic “Science” are followed by
the semantically close factors “Paleoclimatology” and
“Thermohaline circulation”.
Dynamics of climate change discourse
In section “Interpretation of climate change discourse
themes” we demonstrated that factor scores are instru-
mental in factor interpretation. Additionally, factor scores
can be used to track the development of major discussion
themes over time, as we have shown in the discussion of
the terminal factors in section “Interpretation of climate
change discourse themes”. We combined the factor
scores that were extracted during the EFA analysis to
J Environ Stud Sci
obtain the weighed factor index (WFI) for a factor F
over a time period t:

P siF ; siF > m
i 0
WFItF ¼  i ð1Þ
P sf ; sif > m
i;f 0
Here, F—the factor of interest, selected from a set of all
factors {f}: F ∈ {f}, t—time period over which the index is
being computed; Sfi —factor score of article i published
within the time period t on factor f; m—the cutoff value
for the factor scores (we used the value m01; we also tested
m05 and found that the analysis returned similar results).
In a similar way, we computed the weighed topic index
(WTI) as follows:
 i
P sf ; sif > m ; f 2 ffT g
i;f 0
WTITt ¼  i ð2Þ
P sf ; sif > m
i;f 0
Here, T—the topic and {fT}—the set of factors loading to
the topic T; {fT}⊂{f}.
In effect, we rated the weight of a factor (or a topic) related
to climate change in the mass media during a particular time
period by summarizing the factor scores of the articles (cases)
contributing to that factor (or topic) and normalizing the
obtained value. Over time, some climate change discourse
topics received more attention, thus, the relative weights of
various themes change within the discussion. Figure 2
presents two views of these changes. View 2A visualizes the
shift in discussion between the major news events, effectively
merging the publications on the same major topics and termi-
nal factors. View 2B combines the articles published within
the same year. Note that Fig. 2 presents the WFI and WTI, the
indices that measure both the change in the relative number of
the articles and the factor scores of these articles.
The following major trends in the climate change dis-
course over the last 16 years were observed:
& Scientific debates. The topic deals with the views of prom-
inent scientists J. Christy and R. Lindzen (frequently cited
for their “controversial” opinions) and the debate on the
connection between climate change and stronger snowfalls
or ENSO patterns. A major decline in WTI was found.
& Physics of climate. There has been a large decrease in
the discussion of this topic and its related factors (pa-
leoclimatology and thermohaline circulation) since the
1990s. This decline stopped temporarily after the IPCC
AR-4 but restarted after Climategate.
& Climate change impacts. This is the only “Climate”
topic that experienced some growth until the end of
2007 when the IPCC AR-4 was published.
J Environ Stud Sci

Fig. 2 Stacked area charts of

the WFI and WTI index
dynamics. Horizontal axis: a
periods between major news
events (see Fig. 1): (1) January
1995 to December 1997; (2)
January 1998 to August 2001;
(3) September 2001 to January
2007; (4) February 2007 to
December 2007; (5) January
2008 to October 2009; (6)
November 2009 to December
2010; b annual periods. Vertical
axis: WFI for factors, WTI for
topics for m01 (Eqs. 1, 2). The
order (bottom to top) of the
themes and terminal factors
reflects different themes of
discussion: Science (blue);
Politics and GW hoax (brown);
Religion (olive); Climate
change impacts (green, red-
yellow); Business (olive);
Pollution (purple); other factors
(violet)

& Politics. Large overall increases in the discussion of the
political issues surrounding climate change (Fig. 2a)
with high year-to-year variability (Fig. 2b). The large
growth in this topic is mainly due to new issues, which
developed from 2008 to 2010 and include the lack of
commitment from developing countries for emission
reduction, cap-and-trade and CO2 emission auctions,
and the differences between the positions of democrats
and republicans. The discussion of the Kyoto protocol
subsided dramatically after the Protocol was brought
into force in November 2004. The discussion of volun-
tary vs. mandatory emission caps subsided after the
2008 elections. It is worth mentioning that all five
WTI maximums were observed in the US in the years
immediately following presidential elections.
& Global warming hoax. A discussion of the controversy
surrounding climatology has been increasing since
2008. One of the factors from this group, “GW hoax”
(i.e. data manipulation by scientists to make a case for
global warming), seems to have strengthened over time
(Fig. 3c). A new factor, Climategate, appeared in 2010.
A related factor, IPCC errors, stays strong through the
entire period. In 2009 to 2010, the discussion focuses on
Dr. Pachauri’s alleged financial conflicts and the IPCC
AR-4 claim of Himalayan glacier melt by 2035.
& Biodiversity, pests. This included a discussion of biodi-
versity reduction and the integrative impact of warmer
climate and forest insects on forests. The discussion of
the related factors “Amazon deforestation” and
“Indonesia deforestation” appeared around 2007.
& Business responsibility, risks. The maximum WTI val-
ues were found in 2003 and 2007. In addition, there was
a recent increase in WFI for the related factor of
“Offshore drilling”.

Fig. 3 Stacked area charts of the WFI index dynamics for the follow-
ing themes: a Science; b Climate change impacts; c Climate change
denial, global warming hoax; d Politics; e Business; f Pollution; g
Other factors: Biofuels, “Inconvenient Truth” documentary, Nobel
& Smog, car pollution. The topic reached its peak during
the G.W. Bush administration years and then subsided.
Multiple factors are contributing to these trends; thus, to
better understand them, the dynamics of each single factor
have to be analyzed (Fig. 3). Notice that the factors demon-
strate a much higher variability than the topics, which con-
sist of multiple factors. For example, despite the general
reduction of the topics contributing to the mega-topic
“Science”, the contribution of single factors (e.g., threat to
polar bear population) may increase. Though the discussion
J Environ Stud Sci
Peace Award to Al Gore and the IPCC, Religion and faith. The periods
for horizontal axis are the same as in Fig. 2a. Note the differences in
vertical scaling among the charts
of the political themes is increasing, the most important
factors (in terms of WFI values) in this discussion alternate
between the voluntary or mandatory emissions cap, the
Kyoto protocol, the Copenhagen COP, the carbon market,
China emissions, and the views of political parties.
Overall, the discussion of politics and the global warming
controversy demonstrates a spectacular increase since
January 2008. Conversely, we found that the relative share
of articles discussing the science of climate, including cli-
matology and scientific debates, has dropped dramatically
over the years, from being the dominant topic to a marginal
J Environ Stud Sci
one. To test WFI and WTI robustness, we also computed the
normalized number of published articles (the number of
articles that load to a factor, divided by the total number of
articles published within the time period); these indices
demonstrated similar dynamics (Appendix C).
Discussion
The idea of a continuous, scientifically grounded monitoring
of a large body of news media can be traced back to
Tenney’s (1912) proposal of “social weather” reporting,
which is “comparable in accuracy to the statistics of the
US Weather Bureau” (Tenney 1912, pg. 896). Nowadays,
with the existence of electronic newspaper databases,
Internet parsing software capable of collecting data accord-
ing to pre-programmed protocol, and developments in
CATA programs for subsequent data analysis, this ambitious
goal seems to be less intimidating and even within reach.
Traditionally, the analysis of newspaper discourse on cli-
mate change involves manual coding of a representative
sample of newspaper publications, followed by descriptive
statistical analysis. In this paper, we propose a different way
to approach content analysis of large textual bodies on
climate change by introducing techniques that provide quan-
titative support for text interpretation. The approach permits
to efficiently and reliably survey large volumes of data
published over extended time periods, and is reproducible
and transparent. The procedure, though not completely
eliminating the role of the researcher, provides guidance
and quantitative backing for the decisions at each stage of
the content analysis project.
The main idea is to break the text into themes using factor
analysis and then re-organize the themes into topics of a
higher order. In the method, each article or text segment can
contribute to multiple topics. This contribution can be quan-
titatively assessed using factor scores. The factor interpreta-
tion is done by experts but is supported by the factor scores,
which estimate how relevant the text segment is to the
factor. Finally, factor scores are used to construct the WFI
and WTI indices, which measure the input of each identified
topic of the climate change discourse in a particular time
period, allowing for the detection of topic shifts within
discussions, the tracking of developments through time,
and the visualization of discourse evolution. The following
discussion of the proposed approach is conducted from two
perspectives: (1) how findings of this study relate to findings
by other researchers studying the climate change discourse,
and (2) methodological issues of the approach and its
limitations.
With respect to identified themes of climate change dis-
course, the proposed approach allows finer distinctions in
classification of texts into categories. For example, Antilla
(2005) analyzed US newspaper coverage of climate science,
classifying the sample of newspaper articles into 32 catego-
ries. In contrast, we were able to come up with 76 factors at the
EFA stage of our analysis, which were further reduced to 56
and aggregated into 11 higher order topics and 17 first-order
factors. Moreover, using weighted factor and topic indices, we
were able to track the developments of these themes over time,
noting dynamics of particular topics over the whole time
frame of 16 years (e.g., droughts and floods) and emergence
and disappearance of other short-time issues (e.g., Amazon
deforestation). In addition, the quantitatively based procedure
allows for various angles in data analysis; consider, for in-
stance, two graphical representations of climate change dis-
course: Figure 2 illustrates the dynamics of topics identified as
result of CFA procedure, while Fig. 3 shows a more detailed
analysis of the newspaper publications.
One major change in the coverage of global climate
change that we discerned from 1995 to 2010 NYT publica-
tions is the sharp decline in the coverage of science of
climate change (Fig. 3a). This decrease in coverage of
climate change science seems to be in line with the general
decline in the science coverage in the US press; e.g., Russell
(2006) found that the number of the US newspapers pub-
lishing a weekly science section dropped from its maximum
of 95 newspapers in 1989, when the spike in science cov-
erage was driven by computer advertising, to 44 in 1992, to
34 in 2005, and to 24 in 2007. Additionally, the majority of
remaining science sections is now focused on health cover-
age and has declined in size (Russell 2006). Partially, this
decline can be explained by the movement of science con-
tent to newspaper blogs, such as the Dot Earth blog by
Andrew Revkin. These blogs have very limited representa-
tion in standard research text databases (e.g., LexisNexis
returns 228 publications in 2011 January–July NYT blogs,
and none in 2010 NYT blogs). However, other, non-
scientific content is also being shifted online: at the moment,
there are 62 NYT blogs in total.
The decline in science coverage is “compensated” by a
sharp increase in the coverage of political issues related to
climate change discussion (Fig. 3d). This shift can be
explained by the general politicization of the topic (Dunlap
and McCright 2008; McCright and Dunlap 2011). Other
authors noticed that the discussion of climate change in
general is frequently driven not by natural events, such as
the extreme weather, but by political events. For example,
Ungar (1999), in an analysis of abstracts of 1968–1996
news broadcast by three US TV networks, counted the
number of reports containing preset keywords such as heat
waves, droughts, floods, etc. The author found no correla-
tion between weather- and climate change-related reports for
all years, with the exception of the extreme drought of 1988.
On the contrary, the spikes in climate change reporting were
frequently traced to the political cycle. As Boykoff and

Boykoff (2007) note, ecological, political, and scientific
factors interact to draw attention to the issue of climate
change.
When considering the results of this study, the reader
should keep in mind that only one newspaper, the NYT,
was used. Mediating this obvious limitation is the common
recognition of the NYT as a major “prestige press” (Stovall
1961) newspaper, which frequently plays an agenda-setting
role for other news media (McCombs 2004). Especially
important is the influence that the NYT has over the dis-
cussion of climate change science and politics. It has the
largest amount of climate change coverage: e.g., (Boykoff
and Boykoff 2007) found that between four major US news-
papers, 39 % of all publications on climate change appeared
in the NYT. In comparison, in the USA Today, which is the
largest US newspaper in terms of the number of printed
copies, our automated search for the period of January
1995 to July 2011 found only 756 “climate change” articles
compared to 4,000 articles in the NYT (not including the
NYT blog content).
The latter point introduces another limitation of the study,
connected to a general shift of newspaper content to online-
only editions and blogging. In the NYT, a large share of
environmental science content is published exclusively on
its blog, Dot Earth. One could argue that the evident reduc-
tion in the number of articles within the topic classified as
“science” (Fig. 2) could be explained by this shift. However,
other topics, including politics, are being shifted online as
well. It is yet to be investigated if the representation of
discussion themes differs between online and offline news-
paper materials.
One can notice that the studies reporting the total number
of publications on climate change tend to employ an oper-
ational approach to article selection. For example, the
University of Colorado monthly index of the world press
coverage of climate change (http://sciencepolicy.colorado.edu/
media_coverage) takes into account the number of the articles
found in LexisNexis, Factiva, and ABI/Inform databases; the
algorithm performs a Boolean search using ‘climate change
OR global warming’ keywords. In our study, we found that this
approach results in a considerable number (approximately
30 %) of “false positives”: some articles relate to the climate
change topic only tangentially, if at all. We also found that for
some samples (e.g., blogs) the percentage of false positives
tends to be higher towards the beginning of the sampling time
frame. Due to this fact, it is probable that multiple studies that
employ this operational approach to report the multi-year
dynamics of press coverage of climate change tend to overes-
timate the number of articles if they do not correct for false
positives (e.g., (Boykoff and Boykoff 2007)); however, we do
not have data to conclude if the percentage of false positives is
changing over time. While the manual analysis deals with the
problem of false positives in a straightforward manner via a
J Environ Stud Sci
screening process (e.g., (Antilla 2005)), the automated process-
ing offered here requires samples with a low percentage of
false positives. The approach to sample selection that we used
has reduced the number of false positives to less than 2 %. As a
result, the number of false negatives (the articles on climate
change that were rejected due to the sample selection criteria)
may have significantly increased. Were this approach applied
to estimate the overall coverage of climate change, it may
result in significant under-reporting. The “fine-tuned” selection
criteria which would balance the numbers of false positives
and false negatives should be developed for subsequent appli-
cation of the approach to study climate change discourse.
From the methodology perspective, the theoretical
grounding of the approach dates back to Iker and colleagues
(Iker 1974; Iker and Harway 1969) and Weber (1983). The
essence of content analysis, whether manual or computer
assisted, is making inferences from the text about its mes-
sage, source, or audience (Krippendorff 2004). In content
analysis, the text is broken into units, which are classified
into a number of categories (Berelson 1952; Holsti 1969).
There are multiple methodological issues in the computer-
assisted content analysis based on word counts which were
summarized by Weber (1983). Two of these issues are the
use of general vs. specialized dictionaries and using the
assumed vs. inferred categories.
General purpose dictionaries allow mapping texts into a
set of assumed categories. However, how well these cate-
gories are able to represent the texts on scientific topics
remains unclear. For example, frequently used General
Inquirer (http://www.wjh.harvard.edu/∼inquirer), originally
developed by Kelly and Stone (1975), maps texts into 182
preset categories. Processing an article on Antarctic ice
coring (Gases at Level Unmatched In Antiquity Study
Shows, NYT, November 25, 2005) with General Inquirer,
mapped the article into 83 categories, with the most frequent
ones being “Strength”, “Activity”, “Time”, and “Space”,
and none relating to climate. Thus, for the texts on scientific
topics, specialized dictionaries can be instrumental (Weber
1983; Muskens 1985). The issue of the inferred vs. assumed
categories can be reformulated as using EFA to extract the
categories from the text population vs. using CFA to con-
firm the connections within the assumed categories (Weber
1983). In this study, we utilize both approaches, first deriv-
ing the categories from the data, and then exploring the
connections between these categories.
The debate on whether the automated content analysis is
too simplistic for sophisticated interpretation of texts has not
yet settled (Neuendorf 2002; Neuman 1997; Newbold et al.
2002). The discussion primarily centers on the “manifest
versus latent content” issue (Duriau and Reger 2004;
Woodroom 1984), highlighting the inability of automated
analysis to capture the latent meaning of texts. However, as
Duriau and Reger (2004) noted, human coders also exhibit
J Environ Stud Sci
low reliability for latent content and the significance of
capturing the latent content might be overestimated in cer-
tain contexts. Moreover, Riffe and Freitag (1997) found that
only half of content analysis studies of press publish inter-
coder reliability data, and those who do very rarely report
the reliability for specific variables. In our view, that may
indicate problems with reliability of human coding.
The proposed approach, however, is not fully automated.
For example, dictionary development is a stage in the anal-
ysis that requires high expert human involvement, in decid-
ing which of the most frequent words should be included
into the dictionary. The most frequent words are too generic,
and inclusion of these words into the dictionary results in
“bloated factors” (Kirilenko et al. 2012). The variables with
medium frequency, however, tend to be specific enough to
meaningfully contribute to the factor (Kirilenko et al. 2012).
The standard stop word lists (Lewis et al. 2004) provide a
good starting point to exclude generic frequently used
words. Additionally, since public discourse shifts over time,
the dictionary should reflect these changes (in this study, the
most frequent words for six periods within the 16-year time
frame were extracted, and then the lists were merged). An
important aspect in dictionary-building is that in order for
the analysis not to be trivial, the dictionary should be suffi-
ciently large. Overall, the proposed CATA approach suc-
cessfully assists the researchers at each stage of the content
analysis; nevertheless, the very nature of the data—a very
large number of variables, multiple factors in non-forced
and forced EFA solutions, higher order constructs and con-
nections between different parts of the CFA model—all
require human involvement and expert knowledge at each
stage of the analysis.
In conclusion, we described how computer-assisted text
analysis can be applied to the extraction of major themes
from a large body of textual materials and demonstrated the
approach on articles addressing climate change. Although
the proposed CATA approach certainly has its limitations,
we are convinced that it has a number of advantages over
manual processing: it is able to analyze large textual bodies,
is more time efficient, has a higher level of detail, enhances
the richness of interpretation, and is able to reliably track
discourse development through time.
References
Antilla L (2005) Climate of scepticism: US newspaper coverage of the
science of climate change. Glob Environ Change A 15(4):338–
352
Bell A (1994) Media (mis)communication on the science of climate
change. Public Underst Sci 3(3):259–275
Berelson B (1952) Content analysis in communication research. Free
Press, Glencoe
Billett S (2010) Dividing climate change: global warming in the Indian
mass media. Clim Chang 99(1–2):1–16
Bord RJ, Fisher A, O’Connor RE (1998) Public perceptions of global
warming: United States and international perspectives. Clim Res
11:75–84
Boykoff MT (2007) Flogging a dead norm? Newspaper coverage of
anthropogenic climate change in the United States and the United
Kingdom from 2003 to 2006. Area 39:470–481
Boykoff MT, Boykoff JM (2004) Balance as bias: global warming
andthe US prestige press. Glob Environ Chang 14:125–136
Boykoff MT, Boykoff JM (2007) Climate change and journalistic
norms: a case-study of US mass-media coverage. Geoforum 38
(6):1190–1204
Brace N, Kemp R, Snelgar R (2003) SPSS for psychologists: a guide to
data analysis using SPSS for windows. Palgrave Macmillan, New
York
Brown TA (2006) Confirmatory factor analysis for applied research.
The Guilford Press, New York
Brulle RJ, Carmichael J, Jenkins JC (2012) Shifting public opinion on
climate change: an empirical assessment of factors influencing
concern over climate change in the U.S., 2002–2010. Clim
Chang. doi:10.1007/s10584-012-0403-y
Byrne B (2009) Structural equation modeling with AMOS: basic con-
cepts, applications, and programming. Taylor & Francis, New York
Deacon D (2007) Yesterday’s papers and today’s technology digital
newspaper archives and ‘push button’ content analysis. Eur J
Commun 22:5–25
Doran PT, Zimmerman MK (2009) Examining the scientific consensus
on climate change. Eos 90(3):22–23
Dunlap RE, McCright AM (2008) A widening gap: Republican and
Democratic views on climate change. Environ Sci Policy Sustain
Dev 50(5):2008
Duriau VJ, Reger RK (2004) Choice of text analysis software in
organization research: insight from a multi-dimensional scaling
(MDS) analysis. In: 7th International Conference on the Textual
Data Statistical Analysis, March 10–12. Louvain-la-Neuve, Belgium,
pp 382–389
Henry AD (2000) Public perceptions of global warming. Hum Ecol
Rev 7(1):25–30
Holsti OR (1969) Content analysis for the social sciences and humanities.
Addison-Wesley, Reading
Hooper D, Coughlan J, Mullen MR (2008) Structural equation model-
ling: guidelines for determining model fit. Electron J Bus Res
Methods 6(1):53–60
Iker HP (1974) An historical note on the use of word-frequency
contiguities in content analysis. Comput Humanit 8:93–98
Iker HP, Harway NI (1965) A computer approach towards the analysis
of content. Behav Sci 10:173–183
Iker HP, Harway NI (1969) A computer systems approach toward the
recognition and analysis of content. In: Gerbner G (ed) The
analysis of electronic content. John Wiley, New York, pp 381–405
Kellstedt PM, Zahran S, Vedlitz A (2008) Personal efficacy, the infor-
mation environment, and attitudes toward global warming and
climate change in the United States. Risk Anal 28(1):113–126
Kelly EF, Stone PJ (1975) Computer recognition of English word
senses. North-Holland, Amsterdam
Kirilenko A, Stepchenkova S, Romsdahl R, Mattis K (2012)
Computer-assisted analysis of public discourse: a case study of
the precautionary principle in the US and UK Press. Qual Quant
46(2):501–522
Kline P (1994) An easy guide to factor analysis. Routledge, New York
Krippendorff K (2004) Content analysis an introduction to its method-
ology. Sage Publications, Thousand Oaks
Krosnick JA, Holbrook AL, Visser PS (2000) The impact of the fall
1997 debate about global warming on American public opinion.
Public Underst Sci 9:239–260

Krosnick JA, Holbrook AL, Lowe L, Visser PS (2006) The origins and
consequences of democratic citizens’ policy agendas: a study of
popular concern about global warming. Clim Chang 77(1–2):7–43
Lacy S, Fico F, Simon TF (1991) Fairness and balance in the prestige
press. Journal Q 68(3):363–370
Leiserowitz A (2006) Climate change risk perception and policy
preferences: the role of affect, imagery, and values. Clim
Chang 77:45–72
Lewis DD, Yang Y, Rose T, Li F (2004) RCV1: a new benchmark
collection for text categorization research. J Mach Learn Res
5:361–397
Liu X, Vedlitz A, Alston L (2008) Regional news portrayals of global
warming and climate change. Environ Sci Policy 11(5):379–393
Macnamara J (2006) Media content analysis—uses, benefits, and best
practice methodology. Asia Pacific Public Relations Journal 6
(1):1–34, Media Monitors
McCombs M (2004) Setting the agenda. The mass media and public
opinion. Blackwell Publishing, Malden
McCombs M, Bell T (1996) The agenda-setting role of mass commu-
nications. In: Salwen MB, Stacks DW (eds) An Integrated
approach to communication theory and research. Greenwood
Publishers, Westport, pp 93–110
McCombs MF, Shaw DL (1972) The agenda-setting functions of mass
media. Public Opin Q 36(2):176–187
McCright AM, Dunlap RE (2011) The politicization of climate change
and polarization in the American public’s views of global warming.
Sociol Q 52(2):155–194
Meyers LS, Gamst G, Guarino AJ (2006) Applied multivariate research:
design and interpretation. SAGE publications, Thousand Oaks
Muskens G (1985) Mathematical analysis of content. Qual Quant 19
(1):99–103
Neuendorf KA (2002) The content analysis guidebook. Sage, Thousand
Oaks
Neuman WL (1997) Social research methods qualitative and quantitative
approaches. Allyn & Bacon, Needham Heights
Newbold C, Boyd-Barrett O, Van Den Bulck H (2002) The media
book. Arnold Hodder Headline, London
View publication stats
J Environ Stud Sci
Oreskes N (2004) Beyond the ivory tower: the scientific consensus on
climate change. Science 306:1686
Porter MF (1980) An algorithm for suffix stripping. Program 14
(3):130–137
Riffe D, Freitag AA (1997) A content analysis of content analyses:
twenty-five years of Journalism Quarterly. Journal Mass Commun
Q 74(4):873–882
Russell C (2006) Covering controversial science: improving reporting
on science and public policy. Joan Shorenstein Center, Harvard
University, Cambridge, MA
Stempel GH, Windhauser JW (1984) The prestige press revisited:
coverage of the 1980 presidential campaign. Journal Q 61:49–55
Stovall JG (1961) The prestige press covers the 1960 presidential
campaign. Journal Q 38:157–163
Tenney AA (1912) The scientific analysis of the press. Independent
73:895–898
Trumbo C (1996) Constructing climate change: claims and frames in
US news coverage of an environmental issue. Public Underst Sci
5(3):269–283
Ungar S (1999) Is strange weather in the air? A study of U.S. national
network news coverage of extreme weather events. Clim Chang
41:133–150
Weber RP (1983) Measurement models of content analysis. Qual
Quant 17:127–149
Weber RP (1986) Correlational models of content: reply to Muskens.
Qual Quant 20(2–3):273–275
Weber EU (2006) Experience-based and description-based perceptions
of long-term risk: why global warming does not scare us (yet).
Clim Chang 77:103–120
Wickham M, Woods M (2005) Reflecting on the strategic use of
CAQDAS to manage and report on the qualitative research
process. Qual Rep 10(4):687–702
Wilson KM (1995) Mass media as sources of global warming
knowledge. Mass Commun Rev 22(1–2):75–89
Woodroom E (1984) ‘Mainstreaming’ content analysis in the social
science: methodological advantages, obstacles, and solutions. Soc
Sci Res 13(1):1–19