How to Read a (Quantitative) Journal Article

Prepared for Sociology 210

Instructor: Greta Krippner
September 21, 2000

Note: Handout refers to Jeremy Freese, Brian Powell, and Lala Carr Steelman, “Rebel
Without Cause or Effect: Birth Order and Social Attitudes,” American Sociological Review
64 (1999): 207-231.

1. The first thing to realize is that quantitative articles follow a formula.  They all have more
or less the same structure: an introductory section in which the problem is introduced and
the objectives of the paper are previewed; a theoretical section in which the literature that
relates to the problem addressed in the paper is described; a data section where the data
sources for the analysis are described; the analysis or results section, where the various
statistical tests performed are explained and the findings presented; and finally, a
discussion or conclusion section in which the main findings are linked back to the
theoretical literature.

2. The most important thing to realize about reading a quantitative article is that (nearly)
everything that is presented in the tables is discussed in the text.  So read the text along
with the tables.  The text will draw your attention to which numbers in the tables are
important. 

3. Your first task in reading the text is to identify what problem is being addressed by the
research.  Typically, this will be clear in the first one or two pages.  In the Freese paper,
the authors identify their problem (pp. 208-9) as testing the effects of birth order on
various social attitudes, including conservatism.  In addition to identifying what the
problem is, try to determine who or what the author is arguing against—i.e., where does
the author situate him/herself in existing debates?  In the Freese paper, the authors are
arguing against Sulloway, who they recognize has made a major contribution by being
the first to study the relationship between birth order and social attitudes (p. 208), but
who they criticize for suggesting that birth order is more important than standard
sociological variables (gender, race, class, age, number of siblings).

4. Next, you should identify the relevant variables in the study and how they are measured. 
In the Freese (pp. 213-215) study, the main independent variable is birth order, measured
 dichotomously—i.e., the respondent is first born or the respondent is not first born. 
Similarly, the dependent variable, social attitudes, is operationalized using six specific
measures: political self-identification, opposition to liberal social movements,
conservative views of race and gender, support for existing authority, and “tough
mindedness.”  Each of these measures of social attitudes is operationalized in turn.  For
example, Freese et al. (p. 215) ask respondents to indicate how patriotic they are (“How
proud are you to be an American?”) as a measure of the variable “support for existing
authority.”

5. The “Results” section is the core of the article.  It is also the hardest to read, because most
technical.  The text will help you to interpret the tables.  The first thing you must figure
out is how variables are coded—i.e. what does a positive versus a negative coefficient
mean?  For example, the Freese (p. 215) article notes that measures are coded so that
positive coefficients are consistent with the hypothesis that first borns are more
conservative in their social attitudes.  Negative coefficients, then, do not support the
hypothesis.  There are two significant coefficients in the first model (p. 216). 
“Significance” means that the observed effect is strong enough that we can rule out
chance as an explanation of the observation.  Significant effects are indicated with an
asterisk (or several asterisks—meaning we can be even more confident that the
observation is not produced by chance).  In this case, the first significant coefficient is a
positive number.  We can interpret this as saying that first borns are more likely to vote
for Bush, which supports the hypothesis.  On the other hand, the negative coefficient on
the significant “tough on crime” measure tells us that first borns are less likely to be
tough on crime than later born children—this contradicts the hypothesis.  On balance,
then, this first model does not lend much credence to birth order theory—only two of 24
measures are significant, and of these two, only one supports the hypothesis that first
borns are more conservative.  Hmmm….not very convincing, right?

6. The next thing to notice, however, is that there are various “models.”  Specifying
different models allows the researchers to take more than one crack at discerning a
pattern in the data.  In this case, Freese and his co-authors know from other research that
variables such as sex, age, race, parents’ education, and sibship size are related to social
attitudes.  So perhaps there really is a relationship between birth order and conservative
attitudes, but it is being obscured by these other variables.  The way to handle this
possibility is to introduce the various demographic variables as control variables, which
means holding them constant so that the effect of birth order can be isolated.  This is what
Freese et al. are doing in Model 2.  But they still don’t find much of a relationship
between birth order and social conservatism.  Look for the significant coefficients in
Model 2.  What do they indicate? 
7. Not to be dissuaded, the researchers throw more controls into Model 3 and Model 4.  
The additional controls specify other factors known to be correlates with social attitudes
—parents’ occupational prestige, parents’ marital status, the loss of a parent before age
16, childhood religion, region of the country in which the respondent was raised
(MODEL 3); and respondent’s education and occupational prestige (MODEL 4).  But in
Models 3 and 4, just as in Model 2, only 3 of 24 measures of social attitudes are
significant, and they are also in the wrong direction!  Remember, because of the way the
variables are coded, a negative number contradicts the hypothesis that first borns are
more conservative.

8. So, on this evidence, support for birth order theory is weak.  But notice what Freese et al.
(pp. 218-219) do next.  They now examine each of the variables that served as controls in
“Model 2”—sex, age, race, parents’ education, and sibship size—and compare their
effect to the effect of birth order.  Notice that in Table 2 these variables are no longer
functioning as control variables—they are not being held constant, but rather allowed to
vary, so that they can be related to variance in the dependent variable.  Freese et al. are
able to show that these variables are far more powerful predictors of social attitudes than
is birth order—for each variable, at least 12 of the measures are significant.  However, in
looking at the pattern formed by significant measures, Freese et al. (p. 219) note that only
age is consistent—the other independent variables tend to contain contradictions.  For
example, respondents with well educated parents tend to be more liberal on attitudinal
measures than respondents with less well educated parents, yet they are also more likely
to identify themselves as Republican than Democrat.  Freese et al.’s (p. 219) conclusion
from all of this is that labels like “conservative” may not actually capture a unified set of
values, and that perhaps proponents of birth order theory achieved their results by relying
on vague concepts that actually have little purchase in the real world.

9. Typically, following the main analysis, researchers will try several other tests to establish
the robustness of their findings.  They want to be sure that the results they are getting are
not a quirk of the particular way they manipulated the data.  In the Freese paper, the
authors establish the robustness of findings by using a different data set—one that has
intra-familial data—and by testing a wider variety of measures of social attitudes from
the GSS.  Neither of these tests changes their results.  This increases their confidence that
their results are correct.

10.  A final test done by the researchers is for interaction effect.  The idea of an interaction
effect is that the way a certain variable operates is affected by the presence or absence of
another variable.  The interaction effect they are testing is birth order and spacing of
children: theory suggests that the effect of birth order on social attitudes is most
 pronounced when there is moderate spacing (2 to 5 years) between adjacent siblings. 
Again, there is no evidence from their analysis of the data that this is the case.

11. In sum, in interpreting tables like Table 1 and Table 2 in the Freese paper, there are two
things to consider: 1) are any of the variables significant? And 2) if significant, does the
given variable affect the dependent variable in the predicted direction?

 
Vocabulary
Bivariate Relationship: a relationship between two variables considered in isolation from other
factors.  The Freese paper first examines the bivariate relationship between birth order and each
measure of social attitudes (pp. 215-217). 

Control Variable: A variable that is held constant so that the effect of a third variable (which
you are interested in) on the dependent variable can be observed more clearly.  Generally, you
control by selecting only individuals who share in common the variable you are controlling for
(age, gender, etc.) Whenever a variable is held constant, that variable cannot account for
variation in the dependent variable, so you are eliminating its effect from consideration.  For
example, if you want to explain variation in levels of aggression and you control for gender by
studying only males, then the variable “gender” cannot account for any of the observed variation
in aggression.  This does not mean that there is not a relationship between gender and aggression,
but it is not the relationship that you are studying.  Holding variables constant is a means of
simplifying complex social situations by ruling out variables that are not of immediate interest
but that might otherwise explain part of the phenomenon that the investigator wishes to
understand.  In the Freese study, sex, age, race, parents’ education, and sibship size are included
as controls because the authors want to eliminate their affect on social attitudes so that they can
observe the effect of birth order directly (p. 215).

Correlate: a variable bearing a relationship to another variable.

Correlation: a measure of the strength and direction of the relationship between two variables. 
Correlations vary between –1.00 and 1.00.  Thus, they may be positive (a direct association),
indicating that two variables tend to move in the same direction, or negative (an inverse
association), meaning they move in opposite directions. Education is positively correlated to
earnings.  Smoking is negatively correlated with longevity.  Correlations vary in strength: there
is a 0.95 correlation in the height of identical twins, but only a .34 correlation between education
and earnings (for men age 25-34 in 1988).  A correlation of 0 indicates that there is no
relationship between two variables.  Note: correlation does not imply causation because it does
tell us which variable is cause and which is effect. 
Dependent Variable: A variable that the researcher tries to explain or predict; the presumed
effect of one or more independent variables.

Dichotomous Variable: a variable which can take on only two values, yes or no.  In the Freese
(p. 213) paper, birth order is a dichotomous value: either the respondent is first-born, or the
respondent is not first born.

Explanatory Variable: another term for an independent variable—i.e., a variable which is

presumed to be a cause of the dependent (outcome) variable.

Independent Variable: a presumed cause of the dependent variable.

Interaction Effect: An outcome in which the effect of one independent variable on the
dependent variable varies according to the value or level of another independent variable.  That
is, the effects of the variables together differ from the effects of either alone. 

Multivariate Relationship: a relationship between many (more than two) variables considered
simultaneously.

Null Hypothesis: The hypothesis that there is no relationship between variables you are testing. 
When Freese et al. (p. 217) talk about the “null results” they mean the evidence provided by their
data of no effect of birth order on social attitudes.

Operationalization: the detailed description of research operations or procedures necessary to

assign units to the categories of a variable.  More concretely, how are abstract concepts (which
we represent with a variable such as “social attitudes”) measured?  In the Freese (p. 214) study,
opposition to a liberal social movement is an operationalization of the concept social attitudes.

Population: The total membership of a defined class of people, objects, or events; also called
universe.  In the Freese paper (p. 212), which uses GSS data, the population is all non-
institutionalized, English-speaking adults in the United States.

Regression: A statistical technique for studying linear relationships among variables. 

Regression may either consider bivariate relationships (called simple regression) or multivariate
relationships (called multiple regression).

Regression Coefficient: A regression coefficient is a way of expressing the effect of one

variable on another.  Under specific conditions, a regression coefficient has a rather neat
interpretation: it indicates how much the dependent variable changes with a one-unit change in
the independent variable. 

Robustness: Robust results are results that hold up under a wide range of different kinds of
tests.  You do the statistical test one way and get the result; you do it another way and you
produce the same result; you use different data and you still have the same result—then your
finding is robust.
Sample: A subset of cases (typically individuals) selected from a population for study.  The
sample also is used to specify the method of selection—i.e., a random sample or a full
probability sample is a sample in which every individual with the desired attributes has an equal
probability of being selected.  Samples may be further restricted for the purposes of answering
certain questions.  For example, when measuring beliefs about racial equality, Freese et al. (p.
214) restrict their sample to whites in order to ensure comparability of their results with the
research that they are critiquing (which used an all-white sample).

Significance: The effect of a given variable is statistically significant if it is sufficiently strong

that we can rule out chance as an explanation of the observed relationship. 

Standard Deviation: A measure of how much variability there is in a set of observations.  The
standard deviation—also called the variance—indicates the average “spread” of observations
around the mean.  Observations that are tightly clustered around the mean have low variance. 
Observations that are widely dispersed have high variance. 

Note: Many definitions based on Singleton, Straits, and Straits, 1993, Approaches to Social
Research.