Journal of Human Rights

ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/cjhr20

Words count: Discourse and the quantitative

analysis of international norms

Carla Winston

To cite this article: Carla Winston (2020) Words count: Discourse and the quantitative
analysis of international norms, Journal of Human Rights, 19:1, 138-151, DOI:
10.1080/14754835.2019.1671180

To link to this article: https://doi.org/10.1080/14754835.2019.1671180

Published online: 17 Jan 2020.

Submit your article to this journal

Article views: 655

View related articles

View Crossmark data

Citing articles: 1 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=cjhr20
JOURNAL OF HUMAN RIGHTS
2020, VOL. 19, NO. 1, 138–151
https://doi.org/10.1080/14754835.2019.1671180

Words count: Discourse and the quantitative analysis of

international norms
Carla Winston
University of Melbourne

ABSTRACT
Scholarship on human rights norms is fraught with inconsistencies
between different interpretations of norm diffusion and effect. This prob-
lem can be remedied in part through a closer examination of the structure
of norms themselves and how that structure informs the process of data
collection, particularly for quantitative analysis. Contemporary international
norms have a tripartite structure, and research on contemporary norms
must gather data on all three components of this structure in order to
accurately identify the object of study. The addition of value statements to
existing datasets offers a method of data validation to ensure that the
behaviors identified are actually expressions of the norm being studied.
The researcher can correct two distinct forms of overcounting: behaviors
that correspond to other norms and behaviors that are not identified as
normative at all. As a demonstration, I use the set of norms known collect-
ively as transitional justice.

Introduction: The structure of international norms and its methodological

implications
There is no standardized definition of international norms in the contemporary International
Relations (IR) literature (Chayes and Chayes, 1995; Jepperson, Wendt, and Katzenstein, 1996: 54;
Finnemore and Sikkink, 1998). However, the underlying structure of many of these definitions is
tripartite. First, a norm presupposes a problem or issue to be addressed. Second, the norm
includes a stated value, which represents the ideational portion of the norm. It is an expression
of something “good” or an avoidance of something “bad” and therefore gives moral weight to the
problem. Third, a norm enjoins a particular behavior: the action to be taken to further the given
value in relation to the problem (Winston 2018).
Not all scholars explicitly engage with this structure in their research. Whereas qualitative and sur-
vey-based studies of norms engage both the ideational and behavioral aspects of norm diffusion and
adoption (Hooghe 2005; Bailey 2008; Clapp and Swanston 2009; Cloward 2014; Kim and Sharman
2014), many quantitative studies have concentrated only on behaviors (Kelley 2008; Gilligan and
Nesbitt 2009; Sikkink 2011; Weldon and Htun 2013; Cole 2015).1 Although understandable, given
limited resources for data collection, this disconnect suggests that scholars researching norms and
norm diffusion empirically may come to different conclusions about the content of the norms they
study, and the states that adopt them, based on which combinations of components they track.

CONTACT Carla Winston carla.winston@unimelb.edu.au W401 John Medley Building, University of Melbourne, Parkville
VIC 3010 Australia.
Data used in this study can be obtained for purposes of replication at: https://dataverse.harvard.edu/dataverse/jhr
Supplemental data for this article can be accessed on the publisher’s website.
ß 2020 Taylor & Francis Group, LLC
 JOURNAL OF HUMAN RIGHTS 139

In particular, behavioralists may encounter the problem of separating self-interest from norma-
tive behavior (Goertz and Diehl 1992: 642), thus losing the signal in the noise. Meanwhile, schol-
ars who look only at discourse and not at implementation miss the practical impact of those
norms on state behavior, as well as issues of meaning-in-use (Wiener 2009). Collection of all
three components of a norm—namely problem, behavior, and value—are necessary to fully and
consistently identify and evaluate the spread of norms worldwide.

Data collection
This article will use the set of norms known as transitional justice (TJ) to demonstrate its argu-
ment. TJ is defined here as the “field of activity and inquiry focused on how societies address leg-
acies of past human rights abuses, mass atrocity, or other forms of severe social trauma,
including genocide or civil war, in order to build a more democratic, just, or peaceful future”
(Bickford, 2004: 1045). Arising out of the third wave of democratic transitions and the develop-
ment of international humanitarian law (Huntington 1991; Bassiouni 2002; Arthur 2009), existing
scholarship counts a number of distinct behaviors and contexts, as well as a varied set of norma-
tive justifications. It has also begun to be evaluated quantitatively (Olsen, Payne, and Reiter 2010;
Olsen, Payne, Reiter, and Wiebelhaus-Brahm 2010; Sikkink 2011; Kim 2012; Sikkink and
Kim 2013).
Defining the TJ case universe entails answering the question, “What problems, behaviors, and
values, and what combinations of the three, appear both in the theoretical literature on TJ and in
the empirical record?” In doing so it uses a modified version of Goertz and Diehl’s (1992:
645–646) suggested measurement approach, cataloguing behaviors but using contemporaneous
official discourse for the subjective piece of a norm’s content rather than a wider deontological
exploration.2 In cases of individual actors, “frequently, norms prompt justification for action and
leave a trail of communication that can be studied” (Bj€orkdahl 2002: 13).

Defining the problem

The problem that animates the practice of transitional justice is, for the purposes of this dataset,
severe or widespread human rights violations. This is a minimalist approach to defining the field,
as compared to adding the additional necessary context of democratization (Sikkink 2011; Payne,
Sikkink, Lessa, Dancy, and Marchesi 2018). It can generally be assumed that most states have
experienced such abuses in living memory (either pre- or postindependence, by their own govern-
ment or another actor), meaning that any state could be a candidate for TJ measures.

Adding behavior
This dataset captures the five most common state-based TJ behaviors: amnesties, trials, lustra-
tions, reparations, and commissions of inquiry. Trials are the pursuit of retributive justice for
human rights violations through the national judicial system.3 Amnesties are the explicit legal
commitment not to pursue trials or other forms of personal accountability for illegal activities, or
pardons for those unjustly convicted. Lustrations are the practice of removing from public office
or service those persons directly involved or complicit in human rights violations. Reparations
are a government’s attempt to “repair” or “restore” various things lost in the course of a conflict
or oppressive regime. Finally, commissions of inquiry (or truth commissions) are “official bodies
set up to investigate a past period of human rights abuses or violations of international humani-
tarian law” (Hayner 1994). I use existing datasets that capture problem and state-based behavior
(De Grieff 2006; Mallinder 2010; Olsen, Payne, and Reiter 2010; United States Institute of Peace
140 C. WINSTON

Table 1. All potential TJ behaviors, 1970–2014

“Potential TJ” behaviors
N %
Amnesty 360 51.43
Lustration 51 7.29
Reparation 58 8.29
Trial 163 23.29
Commission 68 9.71
Total 700 100

2011; Sikkink 2011; Payne et al. 2018). Data on these “potential TJ” behaviors from 1970 to 2014
are counted in an event-based format, with each behavior counting as one event.4
Based on previous existing behavioral datasets, there were seven hundred instances of the
above events that were potentially TJ behaviors from 1970 to 2014, summarized in Table 1.

Recording new data: ‘True-TJ’ values

Value statements, or public explanations of the reason behind the creation of such processes or
institutions, are recorded as much as possible from the original legislation creating those institu-
tions. Primary documents—such as acts of parliament, presidential decrees, and speeches—are
used to determine the official justification for the potentially TJ-related behaviors undertaken
between 1970 and 2015. These are taken from official government gazettes when available.5
When primary source documents could not be found, I used newspaper articles from the period,
retrieved via LexisNexis and The New York Times, to search for statements from official govern-
ment sources of high and relevant rank: heads of government or their spokespersons, relevant
ministers, and, in some cases, statements from Supreme Courts.6
If more than one value is stated, both are recorded, in order of appearance or numerical
superiority.7 If the primary value is not considered “true-TJ” but the secondary is, the behavior is
included in the “true” dataset.
The following eight concepts form the core of the true-TJ values dataset. All are reflected in
the theoretical literature: accountability, rule of law, human rights, truth, democratic transition or
democracy, peace, reconciliation, and selected “others.”8
Accountability is one of the oldest professed goals of transitional justice (Orentlicher 1991). If
a law is broken, the person who broke it must pay a price. Discussions of avoiding impunity also
fall here. Rule of law generally refers to “regularity, stability, and adherence to settled law” (Teitel
2000: 11), but it may also refer to a set of adjustments to the legal and political system so that
the laws serve key aims such as the protection of human rights. Similar terms include
“constitutionality” (Uruguay) and “good governance” (Uganda).
Some of the specific discussions of human rights as a value or goal in TJ legislation (rather
than merely defining the problem) refer to international human rights treaties (Argentina), reaf-
firming a commitment to human rights that had been ignored (United States), and the preven-
tion of recurrence of abuses (Czechoslovakia, Canada). This preventative goal is one of the
foundations of the transitional justice field (Zalaquett 1989).
Truth is often viewed as a necessary precondition for the pursuit of other goals, but the revela-
tion of truth has been argued to be a powerful goal in and of itself. Truth allows victims to have
their suffering acknowledged, systems of oppression to be discovered, and perpetrators to confess
guilt, if they so choose (Albon 1992; Popkin and Roht-Arriaza 1995; Hayner 2001; Dancy, Kim,
and Wiebelhaus-Brahm 2010).
The transition/democracy category includes democratic transitions, recommitments to democ-
racy if the implementing state is already democratic, and transitions from previous regimes,
 JOURNAL OF HUMAN RIGHTS 141

Table 2. “True” Transitional Justice stated values, 1970–2014.

Primary Secondary value – Secondary value if primary All expressed TJ values
value any primary value value is non-TJ (columns 1&2)
N N N N %
Accountability 27 6 1 33 10.15
Human Rights 27 16 43 13.23
Rule of Law 37 10 1 47 14.46
Truth 25 10 35 10.77
Trans/Democ 37 15 1 52 16
Peace 89 19 4 109 33.54
Reconciliation 59 45 1 104 32
Other - yes 18 0 18 7.38
Total 319 8 441 100
None 289
Other- no 86
Total “True TJ” behaviors: 327

whether the new regime is democratic or not. Empirically, it is difficult to separate out
“democratic transition” and “democracy” from many documents.
Peace and reconciliation are perhaps goals that are most identified with conflict management
and resolution rather than with democratic transition or justice, showing evidence of the dual
roots of the TJ field. Peace refers to the cessation of hostilities, the demobilization and reintegra-
tion of troops and militias (Democratic Republic of Congo), stability (Philippines), and/or the
restoration of normal political relations between opposing forces (Croatia). Reconciliation is a
deeper process, including the restoration of social trust and the ability of different actors to work
together within a single political system.
Finally, the other category includes reasons such as remembrance (Germany) or honoring the
victims (South Korea). Although victims form part of the narrative of many value statements,
they are counted here when the purpose of the behavior is expressly for them, rather than for a
more state-oriented goal such as those above. The numbers of events with each of the stated pri-
mary and secondary values are listed in Table 2.
The resulting event-count dataset of 327 true-TJ events—TJ behaviors with publicly expressed
TJ values—is significantly smaller than the list of 700 “potential TJ” events (TJ behaviors with or
without expressed value).9 Given the overall attrition rate—over half the original cases did not
have relevant value statements—several concerns necessarily arise with respect to the potential for
undercounting the diffusion and popularity of TJ by excluding cases that are missing discursive
data. This article argues the opposite: that existing, behavior-only datasets are actually overcount-
ing the number of TJ-related behaviors. The next section suggests that, in many cases, excluding
behavioral information without a discursive counterpart results in a dataset more finely attuned
to normatively relevant activity than does counting behavior alone.

Data validation: Matching values and behaviors to norms

Three potential overcounting problems exist with respect to behavioral data that can be partially
corrected with ideational data: first, the possibility of previously internalized other norms that use
the same behaviors; second, behavior that is not publicly identified as normative; and, third, value
statements that are not part of the set of norms being studied.

Missing values: Older internalized norms

Behaviors need not be singularly identified with a particular norm; indeed, in a world in which
few behaviors are new, most are likely to be identified with multiple norm sets at once. This
142 C. WINSTON

poses a data validity challenge; those behaviors that do not engage with the norm being studied
can distort analyses and lead to potentially false conclusions about which states adopt these
norms. For example, within the dataset of potential TJ events (behaviors), amnesties were by far
the most frequently observed behavior, comprising just over 50 percent of the total. However,
amnesties are part of at least two additional sets of norms: management of armed conflict and
use of executive power (Joinet 1985; Mallinder 2009; Reiter 2014).
The fact that these additional norms are older and more likely to be internalized than TJ
points to a way in which the addition of ideational data allows for a more valid final dataset.
Recall that the norm life cycle (Finnemore and Sikkink 1998) and the spiral model (Risse, Ropp,
and Sikkink 1999) theorized that once a norm has been internalized, states no longer question
the justification behind their behaviors; it is simply “the thing to do” in the current context.
Therefore, overt reference to a value may be missing. However, for newly adopted norms, a
behavior is more likely to be accompanied by an explicit statement of values in order for that
behavior to be understood as comprising part of a “new” norm.
I argue that separating those behaviors with stated values from those without helps to separate
out behaviors for which “no explanation is needed” because the norm they embody has already
been internalized by the state in question. Because TJ is the newest of the norm sets to which
amnesties might belong, it is the least likely to have been internalized by any particular state and
therefore the most likely to have its values stated rather than implicit.10 In other words, TJ efforts
will need public justification, whereas practices that form part of established norms generally
will not.

Missing values: Publicly nonnormative behavior

In addition to alternative settled norms, I contend that publicly nonnormative behavior may also
lead to an overcount. Not all behavior is driven by publicly justifiable goals: Some actions are
coerced, and others are the result of a political arrangement to forestall an outcome unrelated to
the normative goals of TJ. For example, granting an amnesty to avoid a coup has a goal of
remaining in power or forestalling unconstitutional regime change, but the public is unlikely to
accept these justifications as legitimate goals of a TJ effort. Moreover, both TJ-related and non-
normatively identified activities may occur at the same time, especially during a democratic tran-
sition, when decisions must perhaps be negotiated with other parties. Such strategic behaviors
may be seen as necessary, but they are not thought of as “right.” Whereas a strictly problem-
behavior dataset might miss this distinction, the addition of ideational data allows the researcher
to separate cases of nonnormatively identified behavior from the final dataset from those that
clearly align with the norm being studied.11
Examination of Argentina’s specific experience provides evidence in support of this contention.
Jose Zalaquett’s analysis of the TJ process in Argentina from 1983 to 1987 included justifications
for both high-level trials and lower-level amnesties: Trials are associated with punishment and
with the larger goal of “overcoming the past,” whereas amnesties were born of the realization that
“a policy of unrestricted prosecutions and punishments was neither conducive to national pacifi-
cation nor possible to carry out without military unrest turning into open rebellion” (Zalaquett
1989: 24–26).
Zalaquett’s descriptions carry elements of both TJ-consistent and non-TJ-consistent ideational
considerations: Overcoming the past and national pacification are considered appropriate aims of
transitional justice processes, but punishment (as opposed to accountability) and prevention of a
military rebellion are pragmatic and political aims that are unrelated to TJ norms. Based on an
examination of the official documents in question, trials were explicitly instituted for the goal of
rule of law, whereas the amnesties had no stated goal (Due Obedience Law 1987, Full Stop Law
1986, Law of Natural Pacification 1983). I conclude that, according to the Argentinian
 JOURNAL OF HUMAN RIGHTS 143

government, amnesty was not part of its public pursuit of transitional justice. Therefore, the
amnesties—but not the trials—are excluded from the true-TJ dataset.
Following the logic of this and the previous section, I suggest that a large proportion of the
cases in which there is no stated value are likely to be expressions of settled alternative norms or
nonpublicly justified normative behavior—that is, values are not stated because it is either
deemed unnecessary or no publicly acceptable value is involved.

Alternative values: Nontransitional justice normative behavior

Finally, certain practices that may meet the contextual and behavioral criteria for inclusion in a
transitional justice dataset, and that have stated values, may actually be expressions of other
norms and thus should also be excluded. Again, the practice of granting amnesty helps
to illustrate.
The goal of repatriation—of either refugees, foreign fighters, or exiled political elites—appears
in fourteen amnesties. Amnesty for the purposes of repatriation has barely (and only recently)
been mentioned in the theoretical and empirical literature on transitional justice (Wolfson 2005;
Harris Rimmer 2009; Roniger 2011), which suggests that it has not yet been accepted as part of
the norm set. There are also fifty-four amnesties marked as “other” which provide a value state-
ment generally not considered as TJ norms in the scholarly literature, including but not limited
to repentance (Turkey), family reunion (Bangladesh), rescuing youth (El Salvador), self-determin-
ation (France), forgiveness (Iran), political progress (Jordan), confidence-building (Israel), com-
passion (Philippines), and the inauguration of a new president (Madagascar). Those cases with
only a non-TJ value have been excluded from the true-TJ dataset.
Based on these three sets of reductions, the final true-TJ dataset emerges as one with signifi-
cantly fewer cases. The cases that remain are more likely to be expressions of normative activity
in the field of transitional justice than a purely behavioral dataset, thus making the final dataset a
more valid case universe from which to evaluate TJ adopters and effects. However, the addition
of ideational data does not come without some problems. The section below addresses the largest
of them.

Potential data distortion and resolution: The norm life cycle and tipping points
Although the previous section addressed some of the major concerns regarding data loss using
norms theory itself to justify the exclusion of apparently “missing” data, not every such case can
be accounted for in this manner, even within a single norm set. There are several ways in which
the use of ideational data presents additional challenges for quantitative norms researchers. First,
not every primary source document was available. Where a law, edict, directive, or other primary
source could not be found, alternative sources such as news reports were used to fill in missing
information, introducing a certain amount of “human-generated noise” (Benoit, Laver, and
Mikhaylov 2009). Additionally, most of these statements were not in English, so another layer of
variation occurs with the use of translators and translation software.
Second, some information was simply not available at all. For cases in which the researcher
was able to verify that a contextually applicable law had been passed or trial undertaken but was
unable to determine if a justification existed, either because the text of the legislation was unavail-
able or due to limited news, that case was classified as “no stated value.” It therefore remained in
the “potential” dataset but dropped out of the “true” dataset. Although this may impact analysis
of the characteristics of countries adopting the TJ norms, there is no way to compensate for it,
because its observable implications are the same as if there were truly no publicly stated value.
This leads to a potential undercount of true-TJ entries in the dataset.
144 C. WINSTON

Finally, there is the question of norm internalization. Although I argued above that TJ is the
newest field of practice of the norm sets potentially overlapping with it (and thus the least likely
to be internalized), some states may still internalize their TJ norms within the overall time frame
of the dataset. As states internalize a norm, it ceases to need overt justification, and pursuing a
certain behavior in a certain context is just what is done (Finnemore and Sikkink 1998). For
example, a reparations law that previously might have been justified with reference to state
accountability for harm would, if it were extended to new parties, be passed with no ideation
stated. Such a case would drop out of this dataset.
However, this problem is relevant only for addressing certain types of questions, such as the
characteristics of continued norm adopters over time (Finnemore and Sikkink, 1998), or the
depth of norm diffusion, such as the number of times states use repeated behaviors after internal-
izing a norm, or the overall count of true-TJ behaviors (Sikkink, 2011).12 The internalization issue
would not apply to other avenues of research, including the characteristics of new norm adopters
and the potential avenues of norm diffusion based on those characteristics. When looking for the
breadth of norm diffusion, rather than the depth, value statements from new adopters are
assumed to be more likely across all phases of the norm life cycle, because they must still explain
to their audiences (domestic and international) what they are doing and why.
Limiting the case set to the norm’s emergence phase, or comparing between phases, may also
help to mitigate this potential problem. Only the earliest of adopters would be at risk of internal-
izing these norms within the emergence phase, so the potential data loss is minimized. Once the
norm set has cascaded, however, previous adopters who have now internalized norms could per-
haps be separated out from both new- and never-adopters. This may yield valuable comparative
information, especially as norm life cycle theory predicts that the characteristics of new adopters
will change over time (Finnemore and Sikkink 1998).
When examining a norm set over its life cycle, the difference between behavior-only and
value-behavior data is again apparent and conceptually relevant. Limiting analysis to phases of
the norm life cycle requires finding an approximate tipping point to mark the transition between
emergence and cascade. Theorists are not united in their determination of how many and which
states must adopt a norm before it tips into a cascade, but Finnemore and Sikkink (1998: 901)
suggested that this rarely occurs before a third of the states in the system have adopted it. Each
dataset creates a tipping point, but only one aligns with the theoretical literature from the TJ
field itself.
As shown in Figure 1, if one-third of the states in the system adopting a norm approximates a
tipping point, the various TJ behaviors that constitute the norm set “tip” either in 1983 or 1993,
depending on whether we use the full count of potential TJ behaviors or the restricted count of
true-TJ behaviors. Although 1983 has no particular relevance to the field—indeed, coming before
most of the Latin American cases that might be said to be field-defining—1993 is after both of
these transitions and a large expansion of activities from former Soviet states. It is in the late
1980s and early 1990s that the academic field of inquiry forms as well (Zalaquett 1989; Carver
1990; Huntington 1991; Orentlicher 1991; Brito 1993). An approximate tipping point derived
from a values-added dataset, then, demonstrates resonance with existing understandings of the
field’s evolution and major historical events.13
Using 1993 as the tipping point, the true-TJ dataset can be divided into “emergence” and
“cascade” periods, and the characteristics of the dataset and of those adopting TJ behaviors and
values can be examined in theoretically resonant groupings. They are detailed in Table 3.
These distributions are fundamentally different: The behaviors of new TJ norm adopters, and
the values associated with those practices, have evolved over time in statistically significant
ways.14 Amnesties have become far less prevalent while Commissions have risen in popularity.
Justifications based on democratic transitions have fallen while those related to human rights,
accountability, and reconciliation have risen in comparison. These results echo much of the
 JOURNAL OF HUMAN RIGHTS 145

Figure 1. New TJ behavior adopters, “Potential” and “true-TJ” (with value statement), 1970–2014.

qualitative literature on TJ and the normative statements of world leaders, in which the use of
amnesties for severe human rights violations has been met with increasing approbation (Pensky
2008). Similarly, a value shift away from democratic transition and toward reconciliation matches
the trends in truth commission naming in post-South Africa (Ben-Josef Hirsch 2013) and the
relative lack of democratic transitions after 1992.15
When examining TJ and its evolution over time, then, the value-added true-TJ dataset pro-
vides both an appropriate theoretical point from which to divide the case universe and evolution
dynamics that hew closely to existing qualitative literature on the subject.

Table 3. “True-TJ” behaviors (those with TJ stated values), 1970–2014, per phase of the norm life
cycle. The total number of value statements is higher than the total number of behaviors as both
primary and secondary statements are counted.
Emergence (1970–1993) Cascade (1994–2014)
Behavior N % Behavior N %
Amnesty 89 60.14 Amnesty 71 39.66
Lustration 10 6.76 Lustration 13 7.26
Reparation 13 8.78 Reparation 23 12.85
Trial 20 13.51 Trial 33 18.44
Commission 16 10.81 Commission 39 21.79
Total 148 Total 179
True TJ Value N % True TJ Value N %
Accountability 10 5.52 Accountability 23 8.85
Human Rights 19 10.49 Human Rights 24 9.23
Rule of Law 23 12.71 Rule of Law 24 9.23
Truth 9 4.97 Truth 26 10
Trans/Democ 34 18.78 Trans/Democ 18 6.92
Peace 51 28.18 Peace 58 22.31
Reconciliation 32 17.68 Reconciliation 72 27.69
Other-yes 3 1.66 Other-yes 15 5.77
Total 181 Total 260
146 C. WINSTON

Characteristics of norm adopters

A third test to bolster validity claims related to the addition of value statements to a behavioral
dataset examines the adopters themselves. A quantitative analysis of some key characteristics of
states adopting TJ behaviors, with and without stated TJ values, provides evidence that those
states whose key characteristics (political situations, conflict histories, and legacies of violence)
more closely align with the scholarly literature on TJ are more likely to adopt such norms (behav-
ior and value) than those which do not (adopting only behaviors).
TJ mechanisms are theorized as the most “appropriate” way to deal with the problem of severe
human rights violations, often engendered by repressive government or civil war. Therefore, those
states with more recent or more devastating experiences of violations should find the problem
more salient, and therefore be more likely to create TJ mechanisms (versus other-normative or
nonpublicly normative behaviors) to deal with it.16 In addition, the literature suggests that sali-
ence may be increased due to context: a democratic transition or peace accord. According to TJ
theorists, these behaviors are often undertaken by newly democratic or peaceful regimes because
the political will did not previously exist, “justice” writ large is a more salient concern in demo-
cratic societies, and new governments seek to create legitimacy and stability through new institu-
tions (O’Donnell, Schmitter, and Whitehead 1986; Benomar 1993; Kritz 1995; McAdams 1997;
Arthur 2009). However, the political problem of creating a transition in the first place and the
threat of a democratic or conflict reversal have also been theorized to lead to nonnormatively jus-
tified decision making (Huntington 1991; Vinjamuri and Snyder 2003), complicating the picture
for states acting during the immediate democratization or conflict-ending period but enabling
activities to be undertaken once the new regime has settled a bit.
I therefore hypothesize that states for which either the problem of gross human rights viola-
tions or the opportunity to address it (or both) is more salient, due to recent severe conflict or
democratization, are more likely to adopt the TJ norm set (behaviors with value statements) than
those that adopt only behaviors. These theories apply to new adopters at any time, rather than
simply during the emergence or cascade phases.17
To test this hypothesis, I examine the characteristics of states across a number of variables
operationalizing specific aspects of TJ theory. Using the event-count dataset, I examine the likeli-
hood of true-TJ adoptions (behavior plus value statement) versus only potential-TJ behavior
adoptions for a series of variables that serve as proxies for the severity or salience of the TJ prob-
lem. These variables are created from data that reflect the TJ problem based on either severity
(problem-driven salience) or new circumstances (context-driven salience).18
Problem-driven variables include polity score (Marshall, Gurr, and Jaggers 2016), historical
political terror (Gibney, Cornett, Wood, Haschke, and Arnon 2017), and mid-level or higher cur-
rent conflict (four or more major episodes of political violence [MEPV] in the current year;
Center for Systemic Peace 2013). These act as proxies for the likelihood of gross human rights

Table 4. Logistic regression the likelihood of a “True TJ” value statement accom-
panying potential TJ behavior, 1970–2014, based on key theoretical variables.
“True-TJ” value statements Coeff. (St. Err.) P
Polity score (filled) .0361 (.0152) .017
Worst Political Terror .2432 (.1082) .025
Democratization .1542 (.2455) .530
New Democracy .6614 (.2303) .004
Current Conflict .4261 (.2172) .050
Conflict End .6926 (.3637) .057
New Post-Conflict .1588 (.2564) .536
Constant
1.502 (.445) .001
N 644
Prob > chi2 0
Pseudo R2 .0391
 JOURNAL OF HUMAN RIGHTS 147

violations. Context-driven variables include whether a country is currently democratizing (a

regime change to democracy in the current or previous year, via Polity IV) or if it is a new dem-
ocracy (fewer than ten years as determined by Polity IV scores, but not currently democratizing;
Marshall et al. 2016), if a conflict is ending (the two years between which a state’s MEPV drops
below 4), and if it is recently postconflict (within ten years of conflict end, but excluding the con-
flict end years).19
The logistic regression is shown in Table 4. The tested theories about salience and norm adop-
tion appear to generally be supported by my data: States with higher Polity scores or greater his-
torical political terror, as well as new democracies, are more likely to adopt TJ behaviors with
relevant value statements than those with lower scores on these measures (p < .05). There also
appears to be a weak but significant correlation between states with either an ongoing or ending
conflict and the inclusion of a value statement (p < .1). Coefficients range from relatively small
(.0361 per 1-point increase in polity score) to fairly large (.6926 for conflict end), indicating dif-
ferential impacts on the overall likelihood that a state will adopt TJ-normative language with its
behavior. However, they are uniformly positive: Most of the situations theorized as providing
greater problem-driven or context-driven salience appear to correspond to an increased likelihood
of norm adoption.20
On the other hand, states in the middle of a democratic transition or newly postconflict are
not significantly more likely to choose TJ norms than their counterparts, suggesting the influence
of nonnormative considerations during these periods. This also reflects the TJ literature, especially
in pacted transitions and peace agreements (Huntington 1991; Bassiouni 2002; Vinjamuri and
Snyder 2003). The postconflict variable is harder to explain, but it may also be the result of differ-
ences in state behavior after pacted versus won peace, or the distribution of the data (relatively
few “new postconflict” events). In both finding significant differences and not finding them, these
results generally reflect the scholarly TJ field’s basic understandings of its empirical origins and
the theoretical justifications for such extraordinary activities.

Conclusion
This article has suggested that attention to the underlying structure of norms offers a way to
approach data collection for quantitative studies that corrects for potential overcounting in behav-
ioral datasets and displays improved conceptual validity with respect to the study of international
norms. It has demonstrated in multiple forms that the addition of public justifications for behav-
ior as a necessary condition for inclusion in the case universe creates a dataset that, although
smaller, displays greater coherence with qualitative and theoretical studies of the norms in ques-
tion. The test case used has been the set of norms associated with the field of TJ.
This is not to say that the collection of ideational data comes without methodological and evi-
dentiary challenges. In fields that have not yet been formalized or institutionalized with a central
organizing document and reporting process, collecting ideational data from primary and second-
ary sources is both time intensive and subject to information gaps. Translation and local legal
custom add still more confusion and uncertainty, potentially undercounting even as the technique
attempts to correct for previous overcounting.
In addition to these challenges, however, the creation of datasets of official justifications also
offers new material for theoretical analysis: Why do states say they are undertaking these behav-
iors? What is the analytical or theoretical significance of stating a goal of human rights versus
rule of law? That a commission of inquiry is for truth versus reconciliation? Or an amnesty for
democratic transition versus peace? Analysis of this type of discursive data, both alone and in
relation to which specific behaviors states choose, has the potential to impact our understanding
of the diffusion of separate components of the norms puzzle: which are more salient or transfer-
able, to what states or actors, under what conditions they may be combined, and, potentially,
148 C. WINSTON

their influences on both institutional design and effect. Finally, using ideational data in the quan-
titative analysis of norms allows researchers the ability to bridge the gap between what problems
states address, what institutions they call on to do so, and how they justify their choices: with ref-
erence to norms, or not.

Notes
1. A notable exception is the recent work of Beth Simmons and colleagues on the spread of norms against
human trafficking, via both rhetoric and law. See, for example, Charnysh, Lloyd, and Simmons (2015),
Simmons, Lloyd, and Stewart (2018) and Simmons and Lloyd (2015).
2. Goertz and Diehl’s method divorces the identification of a norm’s value component from the particular
actor undertaking the behavior, thus continuing to ignore meaning-in-use. It is also hoped that mapping
the overlap of these two pieces over time will also illustrate the changing history of these norms, but that
is a matter for future exploration.
3. Trials have been condensed into groups to align the Sikkink dataset more closely with other forms of TJ
data; see the Supplementary Appendix for more details on the methodology for this transformation.
4. 1970 is the start date of the largest previously existing dataset of TJ behaviors, the TJBD, and roughly
approximates the major sets of events linked to the field’s founding: the beginning of the third wave of
democratization (1974), the entry into force of the International ill of Human Rights (1976), and the
creation of the two Additional Protocols to the Geneva Conventions (1976).
5. Translated, when necessary, with Google Translate. When not available online, gazettes were sourced
from the Library of Congress. I thank the foreign government gazette specialists for their invaluable aid
and their patience with my often obscure requests.
6. I do not assume that official statements always mirror underlying motivations; as previous norms
research has demonstrated, these may or may not be hypocritical (Kelley 2008; Hyde 2011; Cloward
2014). However, it is these official statements that publicly engage the norms in question, both furthering
debate about the norm’s conceptual boundaries and potentially influencing practice.
7. See Supplementary Appendix materials for examples of primary documents coded as value statements.
8. “Justice” is not included in this group, although it appears several times in discourse. I view justice as the
larger concept to which each of these smaller values belongs, as it is the overall goal of the field of
transitional justice.
9. Please refer to the Supplementary Appendix for full tables of primary and secondary TJ values and
relative distributions and retention rates for the potential TJ and “true-TJ” versions of the data.
10. As noted, an issue exists with regard to amnesties for peace or reconciliation, as this value/behavior
combination is part of both conflict management and transitional justice. These values are on the
boundary between norm sets, where the goals of each blend together, and it is difficult to draw a clear
definitional line. There are fifty-three amnesties for peace or reconciliation in country-years with at least
four major episodes of political violence, which is one-third of the true-TJ amnesties, potentially creating
a real complication with respect to data validity in cases of amnesty in armed conflict. The implications
for TJ values in cases of armed conflict are less numerically clear.
11. As discussed above, “normatively identified-behavior” does not necessarily mean good-faith or actual
normative behavior.
12. Some of this distortion could be mitigated by attaching the true-TJ label to repeat activities undertaken
by previously norm-adopting states that no longer attach value labels to these behaviors. If trials for
severe human rights violations were previously justified with a relevant value statement, it could be
assumed that any following trials for the same violations were merely internalized expressions of the same
norm. There exists a coding in the provided dataset to explore these possibilities.
13. See the Supplementary Appendix for individual behavior adoptions, with and without value statements. If
we argue that a norm set has the same diffusion dynamics as individual norms, then the tipping point
should come sooner for the group than for its parts. However, it is possible that the set of “relevant
states” that are included in the percentages may change depending on the behavior, meaning that the
tipping points are not directly comparable.
14. Behaviors: Pearson Chi2(4) ¼ 14.795, pr ¼ .005. Values: Pearson Chi2(7) ¼ 25.0612, pr ¼ .001.
15. See the Supplementary Appendix for tables of distributions based on the behavior-only dataset. It should
be noted that some of these differences may be due to regional patterns as well as temporal normative
shifts, although this would need further testing to confirm.
16. See Cortell and Davis (2000) and Rosert (2018) on salience and norm adoption.
17. Although internalization in the cascade phase should correlate with moving further away from a
democratization process or recent civil conflict, it is possible that issues of “justice delayed” may affect
 JOURNAL OF HUMAN RIGHTS 149

new adopters in the cascade phase, which would not fit the hypothetical parameters of this test and
would bifurcate the data into “recent events” and “justice delayed” new adopters. The cascade phase
might also see the use of TJ terminology by hypocritical actors, which might tend to be nondemocracies,
creating additional complications. See the potential solutions to these issues in the third section of
this article.
18. See the Supplementary Appendix for correlation tables. Additionally, not all third-party data are available
for all countries; the differing N (number of events) reflects this issue, as cases for which there is no data
vary depending on which variable is being tested. Finally, I use an event-count rather than a country–year
format because I want to capture the depth of TJ behavior: Some states have multiple behaviors in a
single year, of which some may have value statements and others not. Using an event-count dataset
increases this emphasis, whereas a country–year format obscures them.
19. There initially appears to be a discrepancy between the way democratization (first two years of a
democratic shift in the polity score) and conflict end (the year before and year of a significant drop in
MEPV). This is due to the way each dataset counts: Polity records score at the end of the calendar year; a
late democratic transition would still be counted even if the state was authoritarian for most of the year.
The MEPV simply counts events: If a large number of violent events occur before a peace accord occurs
late in the year, the case would still be counted as “in conflict.” I use the two years surrounding the
change to hedge against this possibility.
20. I suspect that the lower statistical significance of the conflict-related variables may be related to the issue
of the potential overlap between noninternalized conflict management norms and TJ norms, which would
result in terms like “peace” and “reconciliation” being used (even in situations of lower-level conflict).

Notes on contributor
Carla Winston is a lecturer in international relations at The University of Melbourne. She has previously taught at
the University of Victoria (Canada) and the University of British Columbia, where she obtained her PhD. She is
interested in complexity and IR theory, norms and norm diffusion, human rights and transitional justice, peace
and conflict, and the uses of popular culture in politics and international affairs. Recent publications have appeared
in The European Journal of International Relations, Interest Groups & Advocacy, and The Atlantic Journal of
Communications.

ORCID
Carla Winston http://orcid.org/0000-0003-0261-966X

References
ALBON, Mary. (1992) Truth and Justice: The Delicate Balance—Documentation of Prior Regimes and Individual
Rights (New York, NY: The Charter Seventy-Seven Foundation).
ARTHUR, Paige. (2009) How ‘transitions’ reshaped human rights: A conceptual history of transitional justice.
Human Rights Quarterly, 31(2), 321–67.
BAILEY, Jennifer L. (2008) Arrested development: The fight to end commercial whaling as a case of failed norm
change. European Journal of International Relations, 14(2), 289–318.
BARAHONA DE BRITO, Alexandra. (1993) Truth and justice in the consolidation of democracy in Chile and
Uruguay. Parliamentary Affairs, 46(4), 579–593.
BASSIOUNI, M. Cherif. (2002) Post-Conflict Justice (Ardsley, NY: Transnational Publishers).
BEN-JOSEF HIRSCH, Michal. (2013) Ideational change and the emergence of the nnternational norm of truth and
reconciliation commissions. European Journal of International Relations, 20(3), 810–833.
BENOIT, Kenneth, LAVER, Michael, and MIKHAYLOV, Slava. (2009) Treating words as data with error:
Uncertainty in text statements of policy positions. American Journal of Political Science, 53(2), 495–513.
BENOMAR, Jamal. (1993) Justice after transitions. Journal of Democracy, 4(1), 3–14.
BICKFORD, Louise. (2004) Transitional justice. In The Encyclopedia of Genocide and Crimes Against Humanity,
Dinah Shelton (ed.) (New York, NY: Macmillan Reference), Vol. 3.
€
BJORKDAHL, Annika. (2002) Norms in international relations: Some conceptual and methodological reflections.
Cambridge Review of International Affairs, 15(1), 9–23.
CARVER, Richard. (1990) Called to account: How African governments investigate human rights violations.
African Affairs, 89 (356), 391–415.
CENTER FOR SYSTEMIC PEACE. (2013) Major Episodes of Political Violence, 1946–2012 (Vienna, VA: Center for
Systemic Peace).
150 C. WINSTON

CHARNYSH, Volha, LLOYD, Paulette, and SIMMONS, Beth. (2015) Frames and consensus formation in inter-
national relations: The case of trafficking in persons. European Journal of International Relations, 21(2),
323–351.
CHAYES, Abram, and CHAYES, Antonia Handler. (1995) The New Sovereignty: Compliance with International
Regulatory Agreements (Cambridge, MA: Harvard University Press).
CLAPP, Jennifer, and SWANSTON, Linda. (2009) Doing away with plastic shopping bags: International patterns
of norm emergence and policy implementation. Environmental Politics, 18(3), 315–332.
CLOWARD, Karisa. (2014) False commitments: Local misrepresentation and the international norms against
female genital mutilation and early marriage. International Organization, 68(3), 495–526.
COLE, Wade M. (2015) Mind the gap: State capacity and the implementation of human rights treaties.
International Organization, 69(2), 405–441.
CORTELL, Andrew P., and DAVIS, James W. (2000) Understanding the domestic impact of international norms:
A research agenda. International Studies Review, 2(1), 65–87.
DANCY, Geoff, KIM, Hunjoon, and WIEBELHAUS-BRAHM, Eric. (2010) The turn to truth: Trends in truth com-
mission experimentation. Journal of Human Rights, 9(1), 45–64.
DE GRIEFF, Pablo. (2006) The Handbook of Reparations (New York, NY: Oxford University Press).
DUE OBEDIENCE LAW, 23521. (1987) (Argentina).
FINNEMORE, Martha, and SIKKINK, Kathryn. (1998) International norm dynamics and political change.
International Organization, 52(4), 887–917.
FULL STOP LAW, 23492. (1986) (Argentina).
GIBNEY, Mark, CORNETT, Linda, WOOD, Reed, HASCHKE, Peter, ARNON, Daniel, and PISANO,  Attilio.
(2017) The Political Terror Scale 1976–2016. [Online]. Available: http://www.polit-ic-al-ter-rorscale.org [12
August 2018]
GILLIGAN, Michael J., and NESBITT, Nathaniel H. (2009) Do norms reduce torture? Journal of Legal Studies,
38(2), 445–470.
GOERTZ, Gary, and DIEHL, Paul F. (1992) Toward a theory of international norms: Some conceptual and meas-
urement issues. Journal of Conflict Resolution, 36(4), 634–664.
HARRIS RIMMER, Susan. (2009) Reconceiving refugees and internally displaced persons as transitional justice
actors. Research Paper no. 187. New Issues in Refugee Research (Canberra: ANU College of Law).
HAYNER, Priscilla B. (1994) Hayner fifteen truth commissions. Human Rights Quarterly, 16(4), 597–655.
HAYNER, Priscilla B. (2001) Unspeakable Truths: Confronting State Terror and Atrocity (New York, NY:
Routledge).
HOOGHE, Liesbet. (2005) Several roads lead to international norms, but few via international socialization: A case
study of the European Commission. International Organization, 59(4), 861–898.
HUNTINGTON, Samuel P. (1991) The Third Wave: Democratization in the Late Twentieth Century (Norman, OK:
University of Oklahoma Press).
HYDE, Susan D. (2011) Catch us if you can: Election monitoring and international norm diffusion. American
Journal of Political Science, 55(2), 356–369.
JEPPERSON, Ronald L., WENDT, Alexander, and KATZENSTEIN, Peter J. (1996) Norms, identity, and culture in
national security. In The Culture of National Security: Norms and Identity in World Politics, Peter J. Katzenstein
(ed.) (New York, NY: Columbia University Press).
JOINET, Louis. (1985) Study on Amnesty Laws and Their Role in the Safeguard and Promotion of Human Rights
(Geneva: United Nations Economic and Social Council, Commission on Human Rights, Sub-Commission on
Prevention of Discrimination and Protection of Minorities).
KELLEY, Judith. (2008) Assessing the complex evolution of norms: The rise of international election monitoring.
International Organization, 62(2), 221–255.
KIM, Hun Joon, and SHARMON, J. C. (2014) Accounts and accountability: Corruption, human rights, and indi-
vidual accountability norms. International Organization, 68(2), 417–448.
KIM, Hun Joon. (2012) Local, national, and international determinants of truth commission: The South Korean
experience. Human Rights Quarterly, 34(3), 726–750.
KRITZ, Neil J. (1995) Transitional Justice: How Emerging Democracies Reckon with Former Regimes, Neil J. Kritz
(ed.) (Washington, DC: United States Institute of Peace).
LAW OF NATIONAL PACIFICATION, 22.294. (1983) (Argentina).
MALLINDER, Louise. (2010) The Amnesty Database. [Online]. Available: http://www.systemicpeace.org/politypro-
ject.html [10 June 2018]
MALLINDER, Louise. (2009) Exploring the Practice of States in Introducing Amnesties. (Berlin, Germany: Springer
Berlin Heidelberg).
MARSHALL, Monty G., GURR, Ted Robert, and JAGGERS, Keith. (2016) Polity IV Project Political Regime
Characteristics and Transitions, 1800–2015.
 JOURNAL OF HUMAN RIGHTS 151

MCADAMS, A. James. (1997) Transitional Justice and the Rule of Law in New Democracies (Notre Dame, IN:
University of Notre Dame Press).
O’DONNELL, Guillermo A., SCHMITTER, Philippe C., and WHITEHEAD, Laurence. (1986) Transitions from
Authoritarian Rule (Baltimore, MD: The Johns Hopkins University Press).
OLSEN, Tricia D., PAYNE, Leigh A., and REITER, Andrew G. (2010) Transitional Justice in Balance: Comparing
Processes, Weighing Efficacy (Washington, DC: U.S. Institute of Peace Press).
OLSEN, Tricia D., PAYNE, Leigh A., REITER, Andrew G., and WIEBELHAUS-BRAHM, Eric. (2010) When truth
commissions improve human rights. International Journal of Transitional Justice, 4(3), 457–476.
ORENTLICHER, Diane F. (1991) Settling accounts: The duty to prosecute human rights violations of a prior
regime. Yale Law Journal, 100(8), 2537–2615.
PAYNE, Leigh A., SIKKINK, Kathryn, LESSA, Francesca, DANCY, Geoff, and MARCHESI, Bridget. (2018)
Transitional Justice Research Collaborative. [Online]. Available: https://transitionaljusticedata.com/ [23 March
2016]
PENSKY, Max. (2008) Amnesty on trial: Impunity, accountability, and the norms of international law. Ethics and
Global Politics, 1(1–2), 1–40.
POPKIN, Margaret, and ROHT-ARRIAZA, Naomi. (1995) Truth as justice: Investigatory commissions in Latin
America. Law and Social Inquiry, 20(1), 79–116.
REITER, Andrew G. (2014) Examining the use of amnesties and pardons as a response to internal armed conflict.
Israel Law Review, 47(1), 133–147.
RISSE, Thomas, ROPP, Stephen C., SIKKINK, Kathryn. (1999) The Power of Human Rights: International Norms
and Domestic Change (Cambridge, UK: Cambridge University Press).
RONIGER, L. (2011) Transitional justice and protracted accountability in re-democratised Uruguay, 1985–2011.
Journal of Latin American Studies, 43(4), 693–724.
ROSERT, Elvira. (2018) Salience and the emergence of international norms: Napalm and cluster munitions in the
Inhumane Weapons Convention. Review of International Studies, 45(1), 77–99.
SIKKINK, Kathryn, and KIM, Hun Joon. (2013) The justice cascade: The origins and effectiveness of prosecutions
of human rights violations. Annual Review of Law and Social Science, 9(1), 269–285.
SIKKINK, Kathryn. (2011) The Justice Cascade: How Human Rights Prosecutions Are Changing World Politics
(New York, NY: Norton).
SIMMONS, Beth A., LLOYD, Paulette, and STEWART, Brandon M. (2018) The global diffusion of law:
Transnational crime and the case of human trafficking. International Organization, 72(2), 249–81.
SIMMONS, Beth A., and LLOYD, Paulette. (2015) Framing and transnational legal organization: The case of
human trafficking. Transnational Legal Orders (Cambridge, UK: Cambridge University Press).
TEITEL, Ruti G. (2000) Transitional Justice (New York, NY: Oxford University Press).
UNITED STATES INSTITUTE OF PEACE. (2011) Truth Commissions Digital Collection (Washington, DC: US
Institute of Peace).
VINJAMURI, Leslie, and SNYDER, Jack. (2003) Law and politics in transitional justice. Annual Review of Political
Science, 18(1), 303–327.
WELDON, S. Laurel, and HTUN, Mala. (2013) Feminist mobilisation and progressive policy change: Why govern-
ments take action to combat violence against women. Gender and Development, 21(2), 231–247.
WIENER, Antje. (2009) Enacting meaning-in-use: Qualitative research on norms and international relations.
Review of International Studies, 35(1), 175–193.
WINSTON, Carla. (2018) Norm structure, diffusion, and evolution: A conceptual approach. European Journal of
International Relations, 24(3), 638–661.
WOLFSON, Steven. (2005) Refugees and transitional justice. Refugee Survey Quarterly, 24(4), 55–59.
ZALAQUETT, Jose. (1989) Confronting human rights violations committed by former governments: Principles
applicable to political constraints. In State Crimes: Punishment or Pardon: papers and report of the conference,
November 4–6, 1988, Wye Center, Maryland (Queenstown, MD: Aspen Institute Justice and Society Program).