Lectura Kluver

European Union Politics
http://eup.sagepub.com/
Measuring Interest Group Influence Using Quantitative Text Analysis

Heike Klüver
European Union Politics 2009 10: 535
DOI: 10.1177/1465116509346782
The online version of this article can be found at:

http://eup.sagepub.com/content/10/4/535
Published by:
http://www.sagepublications.com
Additional services and information for European Union Politics can be found at:
Email Alerts: http://eup.sagepub.com/cgi/alerts
Subscriptions: http://eup.sagepub.com/subscriptions
Reprints: http://www.sagepub.com/journalsReprints.nav
Permissions: http://www.sagepub.com/journalsPermissions.nav
Citations: http://eup.sagepub.com/content/10/4/535.refs.html
>> Version of Record - Nov 18, 2009
What is This?
Downloaded from eup.sagepub.com at Biblioteca de la Universitat Pompeu Fabra on July 25, 2014
European Union Politics
DOI: 10.1177/1465116509346782
Measuring Interest Group
Volume 10 (4): 535–549
© The Author(s), 2009.
Influence Using Quantitative
Reprints and Permissions:
http://www.sagepub.co.uk/
Text Analysis
journalsPermissions.nav
Heike Klüver
University of Mannheim, Germany
ABSTRACT
The analysis of interest group influence is crucial in order to

explain policy outcomes and to assess the democratic
legitimacy of the European Union. However, owing to
methodological difficulties in operationalizing influence,
only few have studied it. This article therefore proposes a
new approach to the measurement of influence, drawing on
quantitative text analysis. By comparing interest groups’
policy positions with the final policy output, one can draw
conclusions about the winners and losers of the decision-
making process. In order to examine the applicability of text
analysis, a case study is presented comparing hand-coding,
WORDSCORES and Wordfish. The results correlate highly
and text analysis proves to be a powerful tool to measure
interest groups’ policy positions, paving the way for the
large-scale analysis of interest group influence.
KEY WORDS
! influence
! interest groups
! quantitative text analysis
! Wordfish
! WORDSCORES
535
536 European Union Politics 10(4)
Introduction
Analysing interest group influence should be of central concern to scholars

of European Union (EU) policy-making. However, only few researchers have
in fact studied interest group influence, mainly owing to methodological diffi-
culties in operationalizing influence. So far, three different approaches to the
measurement of interest group influence can be identified: process-tracing,
assessing attributed influence and gauging the degree of preference attain-
ment (Dür, 2008). Although process tracing provides high internal validity
and is well suited for checking rival theories and for generating new
hypotheses, it is limited to the analysis of one or just a few policy issues
(e.g. Michalowitz, 2007; Woll, 2007). The attributed influence method draws
either on the self-evaluation of interest groups or on the assessment of experts
and can be applied to a large number of cases (e.g. Dür and de Bièvre, 2007;
Pappi and Henning, 1999). However, it measures perceived rather than
actual influence. The preference attainment approach compares the policy
preferences of interest groups with the policy output in order to draw
conclusions about the winners and losers of the decision-making process
(e.g. Baumgartner et al., 2009; Mahoney, 2007, 2008; Schneider et al., 2007).
This approach is promising since it provides an objective measurement,
covers all channels of influence and can be applied to a large number of cases.
However, one of the major problems associated with the preference attain-
ment technique is how to measure policy preferences. This article, therefore,
proposes a new methodological approach to measure policy positions, thus
paving the way for the large-scale measurement of interest group influence.
The article proceeds as follows. I first present three text analysis
approaches that have been used to measure the policy positions of parties.
This is followed by the research design section. Then I present a case study
comparing the three text analysis approaches. The last part summarizes the
results and draws conclusions for the applicability of content analysis in
interest group influence research.
Overview of content analysis approaches
Textual data are arguably the most widely available source of evidence on
political processes. Content analysis was developed to make systematic use
of this rich data source. Political documents have a great potential to reveal
information about the policy positions of their authors: texts can be analysed
as many times as one wishes and they provide information about policy
positions at a specific point in time. Research on political parties has long
dealt with the measurement of policy positions and has developed three
Klüver Measuring Interest Group Influence Using Quantitative Text Analysis 537
major content analysis techniques for extracting policy positions from party
manifestos: hand-coding, WORDSCORES and Wordfish. Whereas hand-
coding is usually associated with a high degree of validity and low reliability,
the great advantage of WORDSCORES and Wordfish is a high degree of
reliability, but these are often criticized for a lack of validity. Hence, the
validity of WORDSCORES and Wordfish will be tested by comparing them
with hand-coding.
Hand-coding: The Comparative Manifesto Project
The Comparative Manifesto Project (CMP) (Budge et al., 2001; Klingemann

et al., 2006) has probably produced the most well-known and widely used
data set for party positions by manually coding party manifestos. The CMP
developed a classification scheme with 56 categories grouped into seven
policy domains. Where possible, directly opposing pro and contra categories
where specified. In total, 13 categories were defined as left and 13 as right.
Human coders divided the party manifestos into quasi-sentences and allo-
cated them to one of the specified categories. The construction of the left–right
scale is based on saliency theory, which assumes that parties compete with
each other by emphasizing different policy priorities rather than by directly
opposing each other on the same issues (Budge and Bara, 2001: 6–7). The
left–right scale was constructed in the following way. First, the percentages
of left and right categories of the total number of coded quasi-sentences were
calculated. Then, the percentage of left sentences was subtracted from the
percentage of right sentences. Negative scores represent left positions and
positive scores represent right positions.
WORDSCORES
A major step forward was undertaken by Laver et al. (2003): they developed
a fully automated text analysis programme for measuring policy positions.
By comparing the relative frequencies of words in ‘reference texts’ (docu-
ments for which policy positions on predefined policy dimensions are known)
with relative frequencies in ‘virgin texts’ (unknown policy positions), one can
calculate the probability, Pwr, that one is reading a particular reference text r
given a specific word w. So it is assumed that each word provides a little piece
of information about which of the reference texts the virgin text most closely
resembles. Since the policy positions of the reference texts, Ard, are known,
one can use the probabilities, Pwr, together with the reference values, Ard, to
produce a score, Swd, for each word w on dimension d. Then the relative
frequency of each virgin text word as a proportion of the total number of
words in the text, Fwv, is computed. The policy position raw score, Svd, of any
virgin text is then the mean dimension score, Swd, of all the scored words that
it contains, weighted by the frequency of the scored words, Fwv. In order to
compare the scores of the virgin texts directly with those of the reference texts,
these raw scores are finally transformed into S*vd (see Laver et al., 2003).1
Confidence intervals are obtained by estimating the variance, Vvd, of the indi-
vidual word scores around the text’s mean score.
Wordfish
The most recent innovation in quantitative content analysis is Wordfish

(Proksch and Slapin, 2008; Slapin and Proksch, 2008). It is a statistical scaling
model that allows policy positions of texts to be estimated on a predefined
policy dimension simply by drawing on word frequencies in texts without
relying on reference documents.2 The documents used for the analysis need
to be encyclopaedic statements of the actors’ policy positions on a single
dimension. If one wants to calculate the policy position on a specific issue
that is discussed only in a particular section of a document, only this par-
ticular section should be used for the analysis. The model is based on the
assumption that words are distributed according to a Poisson distribution.
This distribution was selected because of its simplicity: it has only one
parameter, λ, which is at the same time the mean and the variance. The model
is the following:
yij ~ Poisson (λij)
λij = exp(αi + ψj + βj * ωi)
yij is the count of word j in text i. α is a set of text effects that control for the
length of the documents. ψ is a set of word fixed effects that control for the fact
that some words, such as articles or prepositions, are generally used more
frequently than other words. β is an estimate of a word-specific weight captur-
ing the importance of word j in discriminating between policy positions and ω
is the estimate of actor i’s policy position. The entire right-hand side of the
equation is estimated by an expectation maximization algorithm (see Slapin
and Proksch, 2008). In order to identify the model, α1 and the mean of all policy
positions of actors are set to 0 and the standard deviation of all policy positions
is set to 1. Confidence intervals are obtained using a parametric bootstrap.
Research design
In this section, I explain in detail which texts I used and what issue I selected
for the case study. In order to analyse interest group influence, I concentrated
on the policy formulation phase since a Commission proposal is the basis for
further debate between the Council and the European Parliament in the first
pillar. The policy positions of interest groups were extracted from their
submissions in an online consultation. Interest groups are consulted on a draft
before the final policy proposal is decided upon. Being aware that the
submissions may reflect ‘strategic’ rather than ‘true’ policy positions, this
should not constitute a problem for two reasons. First, only transmitted policy
positions – even if they over- or understate the ‘true’ ideal policy positions –
are taken into account by the Commission and, therefore, constitute the
basis for the influence measurement. Second, it is unlikely that there is a
systematic variation of strategically over- or understating preferences across
group types so that the revealed policy position can be taken as a proxy for
the true policy position.3
For the analysis of the Commission’s policy position, press releases
accompanying the communication and the adoption of the proposal are used.
In theory, one could also use the communication and the proposal directly.
This is, however, associated with a problem of comparability: whereas the
communication is written as a continuous political text, the proposal consists
of the explanatory memorandum, the preamble and the actual regulation.
Thus, these texts employ a very different lexicon and cannot be compared
directly using computer-based content analysis (Laver et al., 2003: 315).4
In order to test different text analysis approaches, I selected the online
consultation concerning the reduction in CO2 emissions from cars. On 7
February 2007, the European Commission proposed a legislative framework
to reduce CO2 emissions from cars to 120g/km in 2012. The Commission
called for improvements in vehicle technology, tyres and air-conditioning
systems as well as for a greater use of biofuels. Furthermore, fiscal measures,
improved consumer information and a code of good practice were suggested.
The Commission then launched a public online consultation, which ran from
7 February until 15 July 2007 and was open to anyone interested in this issue.
The Commission adopted its final proposal in December 2007. The policy
positions of the Commission and the interest groups are measured on a single
‘pro environmental control’ and ‘anti environmental control’ policy dimen-
sion. Being located at the ‘pro environmental control’ end of the policy scale
implies that interest groups support the framework suggested by the
Commission and might even go beyond the proposed measures. Interest
groups located at the ‘anti environmental control’ end of the policy scale are
against the measures proposed by the Commission.
This issue was selected for various reasons. First, a wide variety of
interest groups took part in this consultation and one can, therefore, assume
a broad range of policy positions. I classified the groups into four classes:
Traditional Automobile Industry groups (n = 6); Alternative Industry groups

(n = 6), which promote the use of biofuels or electric vehicles; Environmental
groups (n = 6); and Other groups (n = 7).5 Second, the number of
submissions is not too high, so that a hand-coded content analysis could be
conducted. In total, 32 comments were submitted by interest groups.6 Five
submissions were not written in English and were excluded since WORD-
SCORES and Wordfish can be applied only to texts in the same language. Two
further submissions were excluded since one consists only of a PowerPoint
presentation and the other is protected by a password. Hence, 25 interest
group submissions remain for the analysis.7
Analysis
Hand-coding
First, a hand-coded analysis largely based on the design of the CMP was
performed. Drawing on in-depth reading of the Commission and interest
group texts, a classification scheme with 41 categories was developed (see
Table 1): 20 categories were classified as ‘pro environmental control’ and 20
as ‘anti environmental control’. All statements that could not be allocated to
one of these categories were grouped into an ‘others’ category. The units of
analysis are natural sentences. Each sentence was allocated to at least one of
the specified categories. The pro/anti environmental control scale was
produced according to the CMP procedure. First, the percentages of pro and
anti environmental control categories in the total number of coded statements
per text were calculated. Then, the pro percentage was subtracted from
the anti percentage. Negative scores represent pro environmental control
positions and positive scores represent anti environmental control positions.
Figure 1 plots the policy estimates obtained using this classification
scheme. In order to guarantee comparability with the other content analysis
approaches, the estimates were transformed. All Traditional Industry Groups
are located closer to the ‘anti environmental control’ end of the policy scale
than the European Commission. All Alternative Industry Groups are located
closer to the ‘pro environmental control’ side of the policy scale than the
Commission. Four of the Environmental Groups are located closer to the ‘pro
environmental control’ side of the policy scale than the Commission and two
(WWF, RSPB) are located in between the two Commission positions. The
Commission moved from a policy position of –1.17 to a policy position of
–0.55, so it clearly moved towards the Traditional Automobile Industry.
Table 1 Hand-coding classification scheme
Environmental
control
——————————
Overall category Pro Anti
Reduction target positive negative

Appropriateness of measure positive negative
Inclusion of vans positive negative
Code of good practice on car advertising positive negative
Improved labelling to promote the purchase of fuel-efficient cars positive negative
Fiscal measures to promote the purchase of fuel-efficient cars positive negative
Penalties to enforce CO2 reductions positive negative
Efficiency improvements of tyres positive negative
Efficiency improvements of air conditioning positive negative
Greater use of alternative fuels or automotive technology positive negative
Long-term reduction strategy positive negative
Averaging negative positive
Pooling negative positive
Banking negative positive
Individual targets for small-scale manufacturers negative positive
Exceptions for special-purpose vehicles negative positive
Weight as a parameter for calculating reduction targets negative positive
Inclusion of CO2 reduction from cars in general Emissions Trading Regime negative positive
Monitoring positive negative
Crediting negative positive
Other – –
Wordfish
In a second step, I analysed the documents using Wordfish. Since all docu-
ments discuss only the Commission initiative for reducing CO2 emissions
from cars, one can assume uni-dimensionality and, thus, the complete texts
were used for the analysis. As recommended, the documents were edited
before the analysis: bullet points, hyphens, group names, contact details and
enumerations were removed from the documents. Then, the spelling and
grammar check of Microsoft Word was used to identify and correct mistakes.
Using the program jfreq, stop words (extremely common words), numbers
and currencies were removed from the documents and the words were
stemmed and transformed into lowercase. Finally, all stems that were
UKAA
FAEP
ETUC
ETSC
ETRMA
BVRLA
WWF
T&E
BEUC
RSPB
GREENPEACE
FOE
FANC
COMMISSION 2
COMMISSION 1
ENGVA ACEA
EBB VDA
AVERE SMMT
AVELE RAI
AEGPL KAMA
ADTS JAMA
–2 –1 0 1 2
Pro Policy position Anti
Alternative Industry Traditional Industry Commission Environmental Groups Others
Figure 1 Hand-coding policy position estimates.

Note: Only the x-axis has substantial meaning. The y-axis is used only for illustration purposes in
order to arrange the different interest groups so that they do not overlap.
mentioned only in one single text were removed so that 1397 stems remain
for the analysis. Figure 2 shows the results of the analysis.
Most of the groups representing the Traditional Automobile Industry are
located closer to the ‘anti environmental control’ end of the policy scale than
the European Commission. Only RAI is located closer to the ‘pro environ-
mental side’ than both Commission positions, and SMMT is located between
the two Commission positions. By contrast, all Environmental and Alterna-
tive Industry groups are located closer to the ‘pro environmental side’ of the
policy scale than the Commission. The Commission moved from 0.50 to a
policy position of 0.93 towards the Traditional Automobile Industry at the
‘anti environmental control’ end of the policy scale. This shift is statistically
significant since there is no overlap of confidence intervals.
I then compared the Wordfish estimates with the results of the hand-
coding. Figure 3 plots the estimates together with a fitted regression line.8
The estimates correlate highly (r = .70, p < .001) and therefore largely cross-
validate each other. However, whereas both methods predict a clear move
towards the ‘anti environmental control’ end of the policy scale, hand-coding
sees the Commission closer to the ‘pro’ end of the policy scale. This differ-
ence could be due to the dichotomous categorization: a sentence is allocated
either to a ‘pro’ or to an ‘anti’ environmental control category. This leads to
UKAA
FAEP
ETUC
ETSC
ETRMA
BVRLA
WWF
T&E BEUC
RSPB
GREENPEACE
FOE
FANC
COMMISSION 2
COMMISSION 1
ENGVA ACEA
EBB VDA
AVERE SMMT
AVELE RAI
AEGPL KAMA
ADTS JAMA
–2 –1 0 1 2
Pro Policy position Anti
90% Conf. interval Alternative Industry Traditional Industry
Commission Environmental Groups Others
Figure 2 Wordfish policy position estimates.

Note: Only the x-axis has substantial meaning. The y-axis is used only for illustration purposes in
order to arrange the different interest groups so that they do not overlap.
VDA
Anti
2
ACEA
ETRMA JAMA
ETUC KAMA
1
Wordfish estimates
COMMISSION 2
SMMT
COMMISSION 1
BEUC
FOE
UKAA
0
GREENPEACE
T&E RSPB
FAEP
FANC RAI
ETSC BVRLA
WWF
EBB
–1
AEGPL
ENGVA
AVERE
AVELE
ADTS
Pro
–2
–2 –1 0 1 2
Pro Hand-coding estimates Anti
Alternative Industry Traditional Industry Commission
Environmental Groups Others Regression line
Figure 3 Comparison of Wordfish and hand-coding policy position estimates.
a certain loss of information since ordinal differences cannot be captured. In

theory, one could also use a more fine-grained categorization scheme. This
would, however, increase the complexity and, thus, lead to lower reliability
and higher coding costs.
WORDSCORES
As illustrated earlier, WORDSCORES requires the choice of reference texts

with known policy positions on a specified policy dimension. Although this
is a rather easy task for party researchers since the CMP and numerous expert
surveys provide independent measures that could be used, interest group
research so far lacks data sets of policy positions. One can, thus, already
conclude that currently WORDSCORES does not constitute a methodological
tool for measuring interest group influence.
However, what one can do using WORDSCORES is to test the robustness
of the policy positions estimated by other approaches. In order to check the
Wordfish estimates presented above, I will use the documents with the most
extreme policy positions at the ‘pro’ and ‘anti’ environmental control ends of
the policy scale as reference texts and their policy estimates produced by
Wordfish as reference values. However, the most ‘pro’ environmental control
document, from ADTS (Associacio per la Divulgacio de les Tecnologies
Sostenibles), comprises only 403 words. Since reference texts should contain
as many words as possible (Laver et al., 2003: 315), I collapsed the four most
‘pro’ documents – from ADTS, AVELE (Asociación para la promoción de
vehículos eléctricos y no contaminantes de España), AVERE (Association
européenne des véhicules électriques) and ENGVA (European Natural Gas
Vehicles Association) – into a single document (6208 words) and assigned the
mean of their Wordfish estimates weighted by the number of words of each
text as its reference value. The most ‘anti’ environmental control text, by VDA
(Verband der Automobilindustrie), comprises 7227 words.
The WORDSCORES estimates correlate highly with the Wordfish esti-
mates (r = .89, p < .001). To illustrate the comparison, Figure 4 plots the
policy position estimates derived from both methods together with a fitted
regression line;9 79% of the variance of the Wordfish estimates is explained by
the WORDSCORES estimates. According to WORDSCORES, the Commission
moved from a policy position of 1.74 to 3.06; thus, we can again clearly
observe a move towards the ‘anti’ environmental control end of the policy
scale. Five Traditional Automobile groups are located closer to the ‘anti’
environmental control end of the policy scale than the first Commission
position. All Environmental and Alternative Industry groups are located on
Anti
2
ACEA
ETRMA
KAMA JAMA
ETUC
1
Wordfish estimates
COMMISSION 2
SMMT
COMMISSION 1
BEUC
FOE
UKAA
0
GREENPEACE
RSPB T&E
FANC FAEP
BVRLA RAI
ETSC
WWF
EBB
–1
AEGPL
Pro
–2
–6 –4 –2 0 2 4
Pro WORDSCORES estimates Anti
Environmental Groups Other Regression line
Figure 4 Comparison of Wordfish and WORDSCORES policy position estimates.
the ‘pro’ environmental control side of the policy scale. Thus, it can be
concluded that the Wordfish results are largely validated by the WORD-
SCORES estimates.
WORDSCORES was then used to test the hand-coding policy positions
estimates. Reference values were obtained from the hand-coding estimates of
the most extreme documents. The VDA text serves as a reference text for the
‘anti’ environmental control end of the policy spectrum and the documents
by ADTS, AVERE, AEGPL (European Liquefied Petroleum Gas Association),
EBB (European Biodiesel Board) and AVELE are used for the ‘pro’ environ-
mental control positions. This time, five documents were collapsed because
AVELE and AVERE submitted word-for-word identical comments that differ
only in two sentences. Thus, if one of these texts were treated as a reference
and the other one as a virgin text, this would lead to an extreme score for the
virgin text. Figure 5 plots the policy position estimates derived from both
methods, together with a fitted regression line.10 The estimates correlate quite
highly (r = .53, p < .05). However, there is a lot of random noise: only 28%
of the variance of the hand-coding estimates can be explained by the WORD-
SCORES estimates. In conclusion, WORDSCORES strongly confirms the
Wordfish results whereas hand-coding is validated to only a medium degree.
Anti
2 ACEA
FAEP
Hand-coding estimates
KAMA
JAMA
1
UKAA
SMMT
RAI
BVRLA
0
ETRMA
ETUC
COMMISSION 2
–1
RSPB WWF
COMMISSION 1
GREENPEACE FOE
ENGVA FANC T&E
ETSC BEUC
Pro
–2
–6 –4 –2 0 2 4
Pro WORDSCORES estimates Anti
Environmental Groups Other Regression line
Figure 5 Comparison of hand-coding and WORDSCORES policy position estimates.
Conclusion
The aim of this article was to illustrate the usefulness of text analysis for the
measurement of interest group influence. Interest group influence can be
measured by comparing the policy preferences of interest groups with the
final policy output. The measurement of preferences, however, still consti-
tutes a big problem. This article therefore examined the applicability of
content analysis for the measurement of policy positions of interest groups.
A case study was carried out in order to compare three content analysis
approaches: hand-coding, WORDSCORES and Wordfish. The policy position
estimates correlate highly and, therefore, largely cross-validate each other.
Hence, in theory, all three approaches are applicable to the study of interest
group influence. However, one has to keep in mind that each approach has
advantages but also disadvantages.
The big advantage of hand-coding is the in-depth knowledge of the content
of the submissions and the high validity of the measurement. However, the
reliability of the results is relatively low compared with computerized content
analysis (Mikhaylov et al., 2008). Furthermore, hand-coding is very labour
intensive and time consuming. Finally, political issues may sometimes be
highly technical so that it might be difficult for researchers to understand the
content, develop a classification scheme and allocate the text units to categories.
WORDSCORES has the advantage of being 100% replicable and, thus,

reliability is not an issue. Furthermore, WORDSCORES allows the analysis of
large amounts of text in short periods of time. However, the usefulness of
WORDSCORES for interest group research is limited: independent policy
position estimates are needed as reference values and, unfortunately, no large
empirical data sets on the policy positions of interest groups are available yet.
Wordfish is also 100% replicable. So, again, reliability is not an issue.
Moreover, Wordfish also allows large amounts of text to be analysed quickly
without requiring reference values. This constitutes an enormous advantage
for interest group research owing to the lack of independent policy position
estimates. What is often criticized is the lack of validity of computerized
content analysis. But, as this article shows, the results obtained by Wordfish
are largely confirmed by hand-coding, which is usually associated with high
validity. So, the validity of the Wordfish estimates has also been demonstrated.
However, a problem for interest group research results from identifying the
model by setting the mean of all policy positions to 0 and the standard devi-
ation to 1. The total variance of policy positions is therefore fixed, so that
absolute distances cannot be compared across different issues. However, there
are two solutions to this problem. First, one could simply code success
dichotomously: the Commission either did or did not move towards an
interest group’s ideal point. Second, one could calculate the relative change
in the distance between the Commission and an interest group from t0 to t1
as a percentage of the original distance at t0.
Hence, all three methods offer certain advantages but also suffer from
disadvantages. In order to assess interest group influence, hand-coding is
useful if only one or a few issues are studied since it is very time and cost
intensive. WORDSCORES provides no opportunity to measure interest group
influence, at least currently, because reference values are not available.
Wordfish, by contrast, can be easily applied to the measurement of interest
group influence since it allows large quantities of text to be analysed without
relying on reference values. Hence, in order to measure interest group influ-
ence on a large empirical scale, Wordfish is clearly the most useful content
analysis technique.
Notes
I thank Berthold Rittberger, Thomas Gschwend, Sven-Oliver Proksch, Beate

Kohler-Koch, Arndt Wonka, Cornelia Züll, Bernhard Miller, Thomas Meyer and
the three anonymous reviewers for their valuable comments and suggestions.
Furthermore, I am grateful to Marcelo Jenny and his students for recoding the
documents. The data set for the empirical analysis in this article, as well as the
Online Appendix, can be found at http://eup.sagepub.com/supplemental.
1 Throughout this article, the standardization procedure suggested by Laver

et al. (2003) is used. Lowe (2008) provides a thorough discussion of WORD-
SCORES and its problems. See Benoit et al. (2005) for an application of
WORDSCORES in European integration research.
2 Whereas the CMP uses quasi-sentences as the unit of analysis, WORD-
SCORES and Wordfish draw on single atomic words. Even though single word
usage ignores the contextual nature of language, Benoit and Laver (2003) have
demonstrated that the results do not differ much, whether one uses single
atomic words or word pairs and triplets as the units of analysis.
3 I thank an anonymous reviewer for mentioning this point.
4 A second solution is to use the communication as a whole and only the
preamble of the proposal. The policy position estimates derived in this way
correlate highly with the estimates derived by using press releases (Wordfish:
r = .99, hand-coding: r = 1.00; p < .001).
5 The exact names of the interest groups as well as the number of words per
text are provided in the Online Appendix.
6 In this article, only membership organizations are considered to be interest
groups.
7 The submissions can be downloaded from the following URL (consulted July
2007): http://ec.europa.eu/reducing_co2_emissions_from_cars/consultation
_en.htm.
8 The results of the OLS regression are: N = 27, R2 = .49, Coeff. = 0.83,
p = .000, SE = 0.17.
9 The OLS regression produced the following results: N = 22, R2 = .79,
Coeff. = 0.27, p = .000, SE = 0.03.
10 The OLS regression produced the following results: N = 21, R2 = .28,
Coeff. = 0. 26, p = .014, SE = 0.10.
References
Baumgartner, Frank R., Jeffrey M. Berry, Marie Hojnacki, David C. Kimball and
Beth Leech (2009) Lobbying and Policy Change: Who Wins, Who Loses, and Why.
Chicago: University of Chicago Press.
Benoit, Kenneth and Michael Laver (2003) ‘Extracting Policy Positions from
Political Texts Using Phrases As Data: A Research Note’, paper presented at
the annual meeting of the Midwest Political Science Association, 3–6 April,
Chicago.
Benoit, Kenneth, Michael Laver, Christine Arnold, Paul Pennings and Madeleine
O. Hosli (2005) ‘Measuring National Delegate Positions at the Convention on
the Future of Europe Using Computerized Word Scoring’, European Union
Politics 6(3): 291–313.
Budge, Ian and Judith Bara (2001) ‘Introduction: Content Analysis and Political
Texts’, in Ian Budge, Hans-Dieter Klingemann, Andrea Volkens, Judith Bara
and Eric Tanenbaum (eds) Mapping Policy Preferences: Estimates for Parties,
Electors, and Governments 1945–1998, pp. 1–16. Oxford: Oxford University Press.
Budge, Ian, Hans-Dieter Klingemann, Andrea Volkens, Judith Bara and Eric
Tanenbaum (2001) Mapping Policy Preferences: Estimates for Parties, Electors, and
Governments 1945–1998. Oxford: Oxford University Press.
Dür, Andreas (2008) ‘Measuring Interest Group Influence in the EU: A Note on
Methodology’, European Union Politics 9(4): 559–76.
Dür, Andreas and Dirk de Bièvre (2007) ‘Inclusion without Influence? NGOs in
European Trade Policy’, Journal of Public Policy 27(1): 79–101.
Klingemann, Hans-Dieter, Andrea Volkens, Judith Bara, Ian Budge and Michael
D. McDonald (2006) Mapping Policy Preferences II: Estimates for Parties, Electors,
and Governments in Eastern Europe, European Union, and OECD 1990–2003.
Oxford: Oxford University Press.
Laver, Michael, Kenneth Benoit and John Garry (2003) ‘Extracting Policy Positions
from Political Texts Using Words as Data’, American Political Science Review
97(2): 311–31.
Lowe, Will (2008) ‘Understanding Wordscores’, Political Analysis 16(4): 356–71.
Mahoney, Christine (2007) ‘Lobbying Success in the United States and the
European Union’, Journal of Public Policy 27(1): 35–56.
Mahoney, Christine (2008) Brussels versus the Beltway: Advocacy in the United States
and the European Union. Washington, DC: Georgetown University Press.
Michalowitz, Irina (2007) ‘What Determines Influence? Assessing Conditions for
Decision-Making Influence of Interest Groups in the EU’, Journal of European
Public Policy 14(1): 132–51.
Mikhaylov, Slava, Michael Laver and Kenneth Benoit (2008) ‘Coder Reliability and
Misclassification in Comparative Manifesto Project Codings’, paper presented
at the annual meeting of the Midwest Political Science Association, 3–6 April,
Chicago.
Pappi, Franz U. and Christian H. C. A. Henning (1999) ‘The Organization of
Influence on the EC’s Common Agricultural Policy: A Network Approach’,
European Journal of Political Research 36(2): 257–81.
Proksch, Sven-Oliver and Jonathan B. Slapin (2008) ‘WORDFISH: Scaling Software
for Estimating Political Positions from Texts, Version 1.2’, URL (consulted
Sept. 2008): http://www.wordfish.org.
Schneider, Gerald, Daniel Finke and Konstantin Baltz (2007) ‘With a Little Help
from Your State – Interest Intermediation in the Pre-Negotiations of EU Legis-
lation’, Journal of European Public Policy 14(3): 444–59.
Slapin, Jonathan B. and Sven-Oliver Proksch (2008) ‘A Scaling Model for Estimat-
ing Time-Series Party Positions from Texts’, American Journal of Political Science
52(3): 705–22.
Woll, Cornelia (2007) ‘Leading the Dance? Power and Political Resources of
Business Lobbyists’, Journal of Public Policy 27(1): 57–78.
About the author

Heike Klüver is a PhD candidate in the Graduate School of Economic
and Social Sciences, Center for Doctoral Studies in Social and
Behavioral Sciences, University of Mannheim, D7, 27, D-68131
Mannheim, Germany.
Fax: +49 621 181 3699
E-mail: hkluever@mail.uni-mannheim.de

Lectura Kluver

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lectura Kluver

Uploaded by

Copyright:

Available Formats

European Union Politics

Measuring Interest Group Influence Using Quantitative Text Analysis

The online version of this article can be found at:

Email Alerts: http://eup.sagepub.com/cgi/alerts

>> Version of Record - Nov 18, 2009

The analysis of interest group influence is crucial in order to

Analysing interest group influence should be of central concern to scholars

Overview of content analysis approaches

Hand-coding: The Comparative Manifesto Project

The Comparative Manifesto Project (CMP) (Budge et al., 2001; Klingemann

The most recent innovation in quantitative content analysis is Wordfish

Traditional Automobile Industry groups (n = 6); Alternative Industry groups

Table 1 Hand-coding classification scheme

Reduction target positive negative

Figure 1 Hand-coding policy position estimates.

Commission Environmental Groups Others

Figure 2 Wordfish policy position estimates.

Environmental Groups Others Regression line

Figure 3 Comparison of Wordfish and hand-coding policy position estimates.

a certain loss of information since ordinal differences cannot be captured. In

As illustrated earlier, WORDSCORES requires the choice of reference texts

Environmental Groups Other Regression line

Figure 4 Comparison of Wordfish and WORDSCORES policy position estimates.

Environmental Groups Other Regression line

Figure 5 Comparison of hand-coding and WORDSCORES policy position estimates.

WORDSCORES has the advantage of being 100% replicable and, thus,

I thank Berthold Rittberger, Thomas Gschwend, Sven-Oliver Proksch, Beate

1 Throughout this article, the standardization procedure suggested by Laver

About the author

You might also like