Research & I/O Psychology Research & I/O Psychology

I/O psychology research is conducted in

academic institutions and by practitioners in
Research Methods organizations

With the growth in I/O psychology, more applied

Chapter 2 research is being conducted within organizations

PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK

2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Copyright©2021 Taylor & Francis Group, an informa business Organizational Psychology Organizational Psychology Organizational Psychology

Theory, Research, & Theory, Research, and Relationship between

the Research Process the Research Process Theory & Research
Research published in journals typically involves For the most part, good research is based on a good
Research theory
a similar structure:
1. Review past theory and research Theory: description of the relationships among
variables and how these variables influence each
2. Describe study methods and results other in order to explain a particular phenomenon
3. Conclude with implications for practice, theory,
and future research Theory provides a starting point for studying a
phenomenon and guidance for how to study a
Theory Practice Why do journals use this phenomenon
structure?
PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK
2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology

Relationship between Relationship between

Deduction vs. Induction
Theory & Research Theory & Research
Empirical Research: research that is based on
direct or indirect observations to see if
Research Deductive Approach Inductive Approach
theoretical principles hold when tested
start with a strong start with observing a
Researchers should use empirical findings to theory and then set
out to test hypotheses
VS. phenomenon and then
develop a theory to
clarify the theory it was based on and to based on the theory explain it
propose directions for future research
Theory Practice
PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK
2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology
 Basic Research Concepts Basic Research Concepts Basic Research Concepts
Independent Variable (IV): variable that is manipulated In non-experimental studies (where there is not a true
by the research to see how it affects participants manipulation of the IV), other terms are used:

IV DV
Dependent Variable (DV): variable that is affected by
the independent variable IV
• Antecedent
DV
• Consequence

IV DV
• Predictor • Outcome
• Criterion Variable

PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK

2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology

Research Designs True Experiments True Experiments

There is a
Key features of a true experiment: control group
not receiving a
– Uses an experimental group that receives a manipulation
manipulation There is an Participants are
experimental randomly
– Uses a control group that does not receive the group receiving assigned to
manipulation and is used for comparison a manipulation groups
– Involves random assignment of participants to There is a true
experimental and control groups to ensure experiment
only when all
equivalence between groups and to avoid of these are
confound variables met

PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK

2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology

Field Experiments Laboratory Experiments Quasi-Experimental Designs

True experiment conducted in actual organizational True experiments conducted in a laboratory or “Almost” experimental designs
settings called field experiment
online settings called laboratory experiments
“Gold standard” for evaluating the effects of a workplace Approximates a true experiment but may be
policy or intervention Typically involve college students or online samples missing one aspect such as random assignment
Relatively uncommon because: to conditions
– Difficult to carry out Can be cost- and time-effective
– Require organization that is willing/able to assign workers Can be very practical in a field setting
to different conditions Primary limitation: May not be generalizable (or
– Certain manipulations might be unethical or illegal in transferrable)
organizational contexts
PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK
2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology
 Correlational Studies Challenges with Correlational Studies Data Collection Methods
Focus on the relationship among variables, and • Difficult to determine causality (i.e., direction of
relationships between variables)
lack manipulation of the IV and experimental – Temporal ordering can be used as a first step toward
control demonstrating causality

Advantages: • Inflated relationships due to common method (e.g.,

similar measure format used) and/or common source
– Can be done in organizations using samples of (e.g., same participant rating survey items)
actual working people – Separate time points can be used to reduce inflated
– Typically involve practical and inexpensive forms relationships due to mood or other participant factors
of data collection such as surveys – Multiple sources of data can be used to reduce inflated
relationships
PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK
2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology

Data Collection Methods Survey Methods Qualitative Data Sources

• Survey methods Method in which people Include approaches such as:
report their responses on a – Interviews with employees
• Qualitative data sources paper or online questionnaire – Focus groups
– Interviews Advantage: collect a large – Observational methods
amount of data from many
– Focus groups participants relatively easily Advantage: can provide rich data with a lot of detail
– Observational methods about individual and their experience
Disadvantage: data may lack
• Archival sources richness and depth Disadvantage: relatively time-consuming and
difficult to collect data from large samples
PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK
2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology

Archival Sources Statistics Basic Statistical Concepts

Datasets that have already been collected by others and Central tendency statistics:
are made available for analysis
– Mean: average
May come from an organization or through government-
sponsored survey projects – Median: central score in a group or distribution of
scores
Advantage: can be inexpensive and capture more
representative samples from the population – Mode: most frequently occurring number in a
group of scores
Disadvantage: rarely collected with researchers’ research
questions in mind and, thus, do not likely have all
variables needed for a research project
PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK
2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology
 Basic Statistical Concepts Common Statistical Tests Correlation (r)
Spread-of-scores statistics: Common statistical tests: Tests the relationship between variables
– Correlation
– Range: highest score minus lowest scores – Magnitude of relationships
– Linear regression
– Standard Deviation: variability of scores around – t-test – Sign of relationships (+/-)
the mean, represented as average deviation from – Analysis of variance (ANOVA)
the mean; square root of the variance – Meta-analysis Correlation ranges from -1.00 to +1.00, where
0.00 indicates no relationship and values closer
Statistical significance: results of a study appear to to +/-1.00 indicate stronger relationship
be “real” and are not just due to chance

PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK

2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology

Correlation: Scatterplot Linear Regression Linear Regression: Line of Best Fit

5 5
Predicting one score from another
4 Allow one to estimate a person’s score on Y based on the 4
person’s score on X

Y 3 Produces an equation that describes the best fitting line for

expression of the relationship between two variables:
Y 3

2 y = bx + a 2

where y is the predicted score, b is the relative weight of the

1 predictor, and x is the score on the predictor, and a is the y- 1
intercept (constant)
1 2 3 4 5 1 2 3 4 5

X PSYCHOLOGY AND WORK

Perspectives on Industrial and
Organizational Psychology
PSYCHOLOGY AND WORK
2nd Edition
An Introduction to Industrial and
Organizational Psychology
X PSYCHOLOGY AND WORK
2nd Edition
An Introduction to Industrial and
Organizational Psychology

t-test t-test: Example Analysis of Variance (ANOVA)

Trained Group
Control Group
Tests whether there is a significant difference (Classroom Training) Tests whether there is a significant difference
between two means
56.00 70.00
A t-test can determine
between three or more means
57.00 65.00
whether there is a
53.00 68.00 statistically significant Uses the F-test to determine whether differences
Often used to compare the scores for 44.00 70.00 difference between
among three or more means are statistically
67.00 82.00 these two means
participants in the experimental group to those 50.00 59.00 significant
from the control group 70.00 72.00
45.00 68.00
Often used to compare the scores for participants
78.00 80.00 in two or more experimental groups to scores for
60.00 60.00 participants from a control group
Mean=58.00 Mean=69.40
PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK
2nd Edition 2nd Edition PSYCHOLOGY AND WORK
An Introduction to Industrial and An Introduction to Industrial and Perspectives on Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology
 ANOVA: Example Analysis of Variance (ANOVA) Meta-Analysis
Trained Group Trained Group
Control Group (Classroom (Online
An F-test is an omnibus test, and thus only Statistically synthesizes and summarizes the
Training) Training)
56.00 70.00 68.00
An F-test can
determine whether informs us whether there are significant findings of multiple studies for a given
57.00 65.00 65.00 there are phenomenon/relationship (e.g., X à Y)
53.00 68.00 68.00 statistically differences among three or more means
44.00 70.00 70.00 significant
Can take into account a number of statistical issues:
67.00
50.00
82.00
59.00
83.00
59.00
differences among
these three means
Follow-up t-tests are required to test the – Sample size
70.00 72.00 77.00 significance of pairwise comparisons of specific – Unreliability
45.00 68.00 68.00
78.00 80.00 82.00 means
60.00 60.00 56.00 Helps to identify whether there might be
Mean=58.00 Mean=69.40 Mean=69.60 moderators of the focal phenomenon/relationship
PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK
2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology

Meta-Analysis: Example Measurement of Variables Types of Measurement Scales

Correlation Nominal Scale: classifies people into categories
Sample Sample-Weighted
between Age – Example: gender (male vs. female)
Unique Studies Size Correlation
and Happiness
(N) rc = r * (N/NTotal) Ordinal Scale: indicates the place someone falls on a scale relative to others,
(r)
but lacks meaningful interval between scale anchors
Study 1 .35 623 .15 – Example: placing in running race (1st, 2nd, 3rd)
Study 2 .19 399 .05 Interval Scale: has meaningful differences (intervals) between scale anchors,
Study 3 .23 180 .03 such that difference between 1 and 2 is the same as difference between 2
and 3, and so on; lacks an absolute zero
Study 4 -.11 78 -.01 – Example: score on a depression scale
Study 5 .15 178 .02
Ratio Scale: like an interval scale except that it has an absolute zero value
– Example: income
k=5 Sum (NTotal): 1458 rc = .24

PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK

2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology

Reliability Reliability Validity

Refers to the dependability of a measure or its consistency
in measuring people in a group

As reliability goes up, error Error

variance goes down, meaning Reliability Variance
we can be more confident
that differences in scores are
meaningful and can
differentiate among
individuals

PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK

2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology
 Validity Content Validity Criterion-Related Validity
Extent to which a measure is actually measuring what it Process that shows that the measure was developed in Involves the empirical demonstration that the test predicts a criterion
or outcome that you care about. In I-O psychology, this is often job
is intended to measure a way that adequately sampled the domain of interest performance

There are not three different types of validity (validity is Typically uses a correlation or regression-based approach to determine
Example: the degree of association between a test and a criterion
validity!), but there are three of the most common
ways to demonstrate validity: – Develop a test of college algebra Validity Coefficient: correlation between a predictor and a criterion
– Content – Refer to algebra textbooks and consult with Example: Give the algebra test to incoming college students and see if
– Criterion-Related algebra instructors to develop items that cover the it predicts their later grades in Math classes.
– Construct domain of college algebra

PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK

2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology

Construct Validity Construct Validity Construct Validity

Refers to the accumulation of evidence that a measure Example: A test of algebra correlates with other tests of
is really measuring what it is intended to measure, algebra, and even with tests of geometry, but not with
often based on its relationship with other measures. a test of English grammar. (Good convergent and
discriminant validity)
Convergent vs. Discriminant Validity:
Content Validity Criterion-Related Classic Construct
• Convergent Validity: degree to which a measure correlates Construct validity subsumes other forms of validity Validity Validity Evidence
measure adequately
with other measures with which it should have a relationship samples a domain measure has an measure shows
• Discriminant Validity: degree to which a measure does not empirical relationship convergent and
show a relationship with things it should not relate to with an outcome discriminant validity

PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK

2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology

Ethical Issues Ethical Issues Ethical Issues

In research, it is important to consider the ethical Outside of academic settings, there are several codes of
treatment of research participants ethics that I/O psychologists can follow:
– American Psychological Association code of ethics
Informed consent ensures that individuals understand – Academy of Management code of ethics
the study risks/benefits before volunteering to
participate
Institutional Review Board (IRB): entity that governs
the research process at many universities

PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK

2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology
 Technology and I-O Legal Issues Global Implications
• Sophisticated analytics can help employers understand and make • How to go about validating a test may seem to be a purely
decisions about their workforces. academic subject, but nothing could be further from the
truth. Issues when conducting cross-cultural research
• Example: an organization wishing to increase workforce diversity
might multiple issues, such as recruiting sources, hiring procedures, • As we’ll see in Chapter 7, the procedures for validating tests in organizations:
onboarding, training, and the composition of work teams.
used to make hiring decisions are stipulated not only by
professional guidelines (such as SIOP’s Principles for the 1. Establish that the survey measures are
– Might uncover ways to increase diversity that might not have Validation and Use of Personnel Selection Procedures, equivalent across cultures, particularly if
been apparent through traditional research methods. 2018), but also by federal court cases and guidelines
(Uniform Guidelines on Employee Selection Procedures, translated into different languages
• Big data often requires some sort of graphics so that the data can
be better understood and used by organizational decision-makers. 1978). 2. Determine whether a concept that exists in one
– Big data visualization graphically illustrates the relationships • In fact, failing to comply with validation guidelines can culture even exists in another culture
among variables.
result in costly lawsuits, bad hiring decisions, and bad
publicity for the organization.
– Increasing used in large organizations.
PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK
2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology

Translation and Back-Translation Current Workplace Issues Current Research Issues

• There is a growing interest in in intervention research - things
employers can do to improve the attitudes, well-being, health, and
Original English item performance of workers. Increasing use of online samples such as Amazon
• “I am happy with my boss.”
• For example: Mechanical TurkTM (mTurk)
• Crain et al. (2016) describe the implementation of an intervention at
an information technology company that included work-life balance – Convenient and efficient way to test survey scales
training for supervisors and employees, leading to improvements in
Translated into Spanish by one person… sleep. – Criticized as artificial; how much respondents are
• “Estoy contento con mi jefe.” • McCarthy et al. (2017) describe how giving brief explanations to test- paid.
takers prior to an employment test affected their perceptions of how
fair the test was and whether they were treated with respect. • “Open science” - ensuring that research practices
Back-translated into English by another • One question is for whom does the intervention work. Hammer et al.
(2019) found that a worker safety and health intervention consisting
used in a study are transparent and fully reported
person. of supervisor and team training worked best in teams where there and that research findings can be reproduced by
was low team cohesion and poor relationships with the leader to
• “I am happy with my boss.”
begin with. other scientists (Banks et al., 2019).
PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK PSYCHOLOGY AND WORK
2nd Edition 2nd Edition 2nd Edition
An Introduction to Industrial and An Introduction to Industrial and An Introduction to Industrial and
Organizational Psychology Organizational Psychology Organizational Psychology