Machine Translated by Google

Psychological Act 229 (2022) 103696

Contents lists available at ScienceDirect

Psychological Act

journal homepage: www.elsevier.com/locate/actpsy

Linear correlation is insufficient as the sole measure of associations: The

case of technology use and mental health
to,* b
Jean M Twenge , Jessica L. Hamilton
to
Department of Psychology, San Diego State University, United States of America
b
Department of Psychology, Rutgers University, United States of America

ARTICLE INFO ABSTRACT

Keywords: It is common for psychology studies to rely solely on linear correlation (r) or similar statistics and not include other
Mental health
measures of association (such as relative risk, which examines differences in the number of people affected). For
Technology use
example, the association between smoking and lung cancer (r = 0.06) could be dismissed as “small” if only linear r
effect size
relative risk
is examined, even though 30 times more smokers than non-smokers get lung cancer. Many studies concluding that
associations between technology use and well-being as too small to be of practical importance relied solely on linear
r. We show that, across five datasets, “small” correlations between technology use and mental health exist alongside
practically important risk associations. As there are several valid types of association, and characterizing an
2 – can be
association based on a single type of a measure – such as linear r or r
misleading.

1.Introduction to do with their use of social media” (BBC, 2019).

Several recent debates within psychology have focused on the size of
effects and whether they are large enough to be practically important, an
increasingly vital consideration as psychology research moves away from
statistical significance testing and the false dichotomy of the 0.05 p-value
cutoff ( Cumming , 2014; Funder & Ozer, 2019). These discussions
around effect size have occurred across several different research
areas, including the practical importance (or existence) of links between
the Implicit Association Test and behavior (Greenwald et al., 2015; Oswald
et al., 2013), violent video game play and physical aggression (Anderson
et al., 2017; Bushman & Huesmann, 2014; Elson & Ferguson, 2014;
Prescott et al., 2018), and stereotype threat and performance (Shewach
et al., 2019).
Similarly, several researchers have described the association be
tween time spent on technology and mental health as weak or too small
to be practically important (eg, Berryman et al., 2018; Ferguson, 2017;
Ivie et al., 2020; Orben & Przybylski , 2019a, 2019b; Przybylski &
Weinstein, 2017; Stronge et al., 2019). For example, Ferguson (2017)
concluded that “excessive use of screens is only weakly associated with
negative outcomes” on the basis of a linear r of 0.13, which he called “very
small.” Similarly, Orben et al. (2019) summarized the results of their study
by saying, “99.75% of a person's life satisfaction has nothing
Most of these studies relied exclusively on linear correlation (r) or its
derivatives such as r 2 . Linear r measures the linear predictability of one
variable from another. However, linear r is only one aspect of the
association between two variables. For example, linear r does not measure
other types of associations such as relative risk (RR, the likelihood of
occurrence in an exposed group compared to an unexposed group;
Andrade, 2015). Relative risk (and a related statistic, odds ratio) is
commonly used in public health and medicine to understand the number
of people in one group versus another with a certain outcome (for example,
who develop a disease, die, attempt suicide, fit clinical criteria for
depression, or are above or below cutoffs on mood or mental health
scales, eg, Baiden et al., 2020; Petrie et al., 2018; Primack et al., 2021;
Shensa et al., 2020). Linear r and relative risk can be computed from the
same data, but cannot be directly converted from one to the other as r, d,
t, and binary F can; r and RR measure different aspects of the association
(similar to area and circumference in the measurement of a circle).
Because linear r is only one type of association, relying solely on linear r
to gauge practical importance may produce misleading conclusions
(Funder & Ozer, 2019; Pearce, 2010; Rosnow & Rosenthal, 2003). We
give several examples to illustrate.
First, consider the association between smoking and lung cancer. In
2018, 13.7% of US adults smoked cigarettes out of a total of 249.6

* Corresponding author at: Department of Psychology, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182-4611, United States of America.
E-mail address: jtwenge@mail.sdsu.edu (JM Twenge).

https://doi.org/10.1016/j.actpsy.2022.103696
Received 28 March 2022; Received in revised form July 27, 2022; Accepted 1 August 2022
Available online 11 August 2022
0001-6918/© 2022 The Authors. Published by Elsevier BV This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
Machine Translated by Google
J.M. Twenge and J.L. Hamilton
million adults (CDC, 2020). There are approximately 228,820 new cases of lung cancer
each year (Seigel et al., 2020), a rate of 0.092 %. Between 10% and 25% (an average of
17.5%) of lung cancer cases occur among non-smokers, and the rest among smokers
(Bhopal et al., 2019). In a hypothetical sample of 1 million US adults, there would be 920
lung cancer cases each year. If 17.5% of these lung cancer cases occurred among non-
smokers, 161 would be among non-smokers and 759 among smokers, leaving 862,839
non-smokers unaffected and 136,241 smokers unaffected. According to an effect size
calculator for a 2 by 2 contingency table (Wilson, 2020), this produces linear r = 0.06, r
2
= 0.036%.
If we apply the same reasoning used by psychologists describing results on technology use
and well-being of a similar size, we would conclude that smoking has little to no practical
importance for getting lung cancer and that 99.64 % of getting lung cancer has nothing to
do with smoking.
Does this mean people might as well start smoking because it's not related to lung cancer?
No, because linear r is only one type of association; it does not capture the risk of
developing lung cancer based on smoking. These same numbers produce a relative risk of
29.86 – meaning that people who smoke are 30 times more likely to develop lung cancer
than non-smokers. Because most smokers don't develop lung cancer in a given year, the
linear r is low. However, the relative risk of smoking on lung cancer is very high; Smokers
are much more likely to get lung cancer than non-smokers.
Second, consider the association between getting a COVID vaccine and developing
the disease. In the Moderna vaccine trial of 30,000 participants in 2020, 185 people who
received the placebo developed COVID-19, compared to 11 in the vaccine group (Loftus,
2020). If we assume there were 15,000 participants in each group, 14,815 in the placebo
group did not get sick, and 185 did; 14,989 in the vaccine group did not get sick and 11 did.
This produces linear r = 0.07, r
2
= 0.049%.
This r would typically be considered small in psychology and might be labeled as having
little practical value. So was the vaccine ineffective and of little practical value in 2020? No,
because linear r does not capture the risk of developing COVID based on receiving the
vaccine or not. These same numbers produce a relative risk (RR) of 0.06, or a 94%
reduction in disease in the vaccine group (an RR of 1 means no effect; RR's above 1
indicate increases and RR's below 1 indicate reductions).
Thus, the RR indicates substantial effectiveness and practical value, despite the low linear
r.
Third is the association between wearing a seat belt and dying in a car accident. In
Pennsylvania in 2014, there were 121,317 reportable car accidents and 1195 deaths. Of
those who got in a car accident, 78.7% wore a seatbelt; of those who died, 51.9% wore a
seat belt (Edgar Snyder and Associates, 2020). Thus, 620 of those who died in car
accidents were wearing seat belts and 575 were not, leaving 94,856 seat belt wearers
alive and 25,266 non-wearers alive. These numbers yield r =
2
0.07 (r = 0.049 %) but a relative risk of 3.43 – so those not wearing seat belts are more
than three times as likely to die in a car accident as those wearing a seat belt. Thus, an
association that explains less than on half of 1% of the variance (which could be
characterized as “99.5% of dying in a car accident has nothing to do with wearing a seat
belt”) shows a tripling of risk when comparing one behavior vs. another.
Fourth, consider the association between gender and suicide at tempts among
adolescents. In the 1991–2019 Youth Risk Behavior Surveillance System (YRBSS)
administered by the Centers for Disease Control, 5.3% of US high school males attempted
suicide in the past 12 months, as did 10.7% of females. These numbers produce r = 0.10,
but RR = 2.02, meaning that females are twice as likely to have attempted suicide than
males. An association that could be dismissed as “explaining only 1% of the variance”
shows a substantially higher risk of suicide attempts among adolescent girls.
Fifth, we should consider an example with continuous variables, as some might argue
that linear r should not be used for binary or ordinal variables (although it often is in
psychology, including for studies examining technology use and mental health: eg, Orben
& Przybylski , 2019a). Exercise frequency and BMI in YRBSS (controlled for gender,
2
Psychological Act 229 (2022) 103696
race, and grade) correlate r = ÿ 0.07. However, those who exercise 3 or fewer days/week
are 42% more likely to be obese (BMI > 30) than those who exercise 4 or more days/week
(16.7% vs. 11.8%; RR = 1.42). If the highest exercise group (exercising every day, 9.5%
obese) is compared to the lowest exercise group (never exercising, 17.6% obese), as is
often done in studies published in medical journals (eg, Petrie et al., 2018; Primack et al.,
2021; Riehm et al., 2019), RR = 1.85, so never-exercisers are 85% more likely to be obese
than every-day-exercisers. Thus, even when variables are initially continuous, linear r can
be low when relative risk suggests practical importance.
To research psychologists trained to rely on linear r, these examples may seem
shocking – how can data with such low r's have such large relative risks and thus
considerable potential for practical importance?
Does this disconnect frequently occur with data relevant to psychological factors? We
address this question in this paper, analyzing five example datasets assessing time spent
on technology and mental health.
We focused on studies in the area of technology use and well-being with publicly available
data that had been analyzed using only linear r or related statistics and not with relative risk
or odds ratios. We began with three previously published papers on technology use and
mental health (Ferguson, 2017; Orben & Przybylski, 2019b; Przybylski, 2014) and then
added one dataset provided by the authors of a paper that used linear regression only
(Stronge et al. , 2019) and one other publicly available dataset that had not been examined
cross-sectionally in a published paper.
There is considerable precedent in published research for using relative risk or odds
ratios (a related statistic) to examine associations between technology use and mental
health (eg, Kelly et al., 2019; Kim et al., 2020; Kremer et al., 2014; Lin et al., 2016; Mathers
et al., 2009; Messias et al., 2011; Primack et al., 2017, 2021; Riehm et al., 2019; Sampasa-
Kanyinga & Lewis, 2015; Shensa et al. ., 2020). For example, Messias et al. (2011) used
odds ratios to compare non-users to those reporting <1 h, 1 h, 2 h, 3 h, 4 h, and 5+ h of
electronic device use on mental health variables in YRBSS. Riehm et al. (2019) used
relative risk to compare non-users of social media to those reporting <30 min, 30 min to <3
h, 3 to 6 h, and 6 or more hours of daily use on whether they met cutoffs for internalizing
and externalizing mental health issues.
Primack et al. (2021) used odds ratios to compare those in the lowest quartile of social
media use to those in higher quartiles of use on whether they met a cutoff for depression
on a 9-item scale.
However, it is rare for the same datasets to be analyzed using both linear r and relative
risk, perhaps because linear r is more commonly used in psychology and communications
and relative risk is more commonly used in public health and medicine. This might be one
reason why researchers in different areas end up talking past each other, with little
consensus around the practical importance of the associations be tween time spent on
technology and mental health. Generally, re searchers (including meta-analyses) using
linear r have reported weak associations between technology use and mental health (eg,
Berryman et al., 2018; Ferguson, 2017; Ivie et al., 2020; Orben & Przybylski, 2019a, 2019b;
Przybylski & Weinstein, 2017; Stronge et al., 2019), while researchers using relative risk
have reported practically significant associations (eg, Kelly et al., 2019; Kim et al., 2020;
Kremer et al., 2014 ; Lin et al., 2016; Mathers et al., 2009; Messias et al., 2011; Primack et
al., 2017, 2021; Riehm et al., 2019; Shensa et al., 2020). This may be one reason why
some observers have concluded that the literature in this area is inconclusive (eg, Best et
al., 2014; Meier & Reinecke, 2021). We attempt to provide some clarity by examining the
same datasets using both linear r and relative risk. We focus on cross-sectional studies;
Thus, they can establish associations but not causation.
2. Study 1: UK Understanding Society Youth Panel Wave 1
We first examined the association between electronic gaming and well-being in the UK
Understanding Society Youth Panel Wave 1. These data were originally analyzed by
Przybylski (2014), who used linear
Machine Translated by Google

JM Twenge and JL Hamilton Psychological Act 229 (2022) 103696

regression and concluded that links between gaming time and well being were “small,” explaining
<1.6% of the variance. We analyze the same dataset using linear r as well as relative risk
analyzes similar to those in other studies of technology use and mental health, which generally
compare the percentage of participants above or below cutoffs on measures of mental health
across levels of technology use (eg , Kim et al., 2020; Primack et al., 2021; Riehm et al., 2019).
some of the data in each comparison. Thus, the data may also be analyzed by dichotomizing the
variable (for example, by a median split), which includes all of the data in a single comparison
(although it also has the disadvantage of less specificity). We also examined the data this way to
determine if relative risk still produced effect sizes that seemed practically significant when all of
the data were included.

2
Results. Table 1 presents the original results (r from contrast-coded

Method. Participants (n = 4873; 48.5% female; ages 10 to 15, mean age = 12.5 years old)
reported time spent on electronic gaming (game console and computer gaming) on a typical
school day (none, 1 h, 1–3 h , 4–6 h, 7 or more hours). Participants also completed two measures
of well-being to assess externalizing problems, internalizing problems, prosocial behavior, and life
satisfaction. The Strengths and Difficulties Questionnaire (Goodman, 1997) measured externalizing
problems, internalizing problems, and prosocial behavior. Life satisfaction was assessed using a
regressions comparing non-users to heavy users), linear correlations and (expressed as a
2 their associated r percent, given the common use of “percent variance explained”),
outcomes within usage groups, and relative risk comparisons. The contrast-coded regressions
and linear correlations both produced effects usually considered to be small, with < 4 % and often
< 1 %. Yet, compared to non-users, youth who spend 4+ hours on console gaming (heavy users)
2r are three times more likely to be low in prosocial behavior (9.3% vs. 30.6%, RR = 3.09) and

measure in which participants rated happiness on a scale of 1 ('completely happy') to 7 ('not at
all happy') across the 6 domains of school, schoolwork, appearance, family, friends, and life as a
whole. The cutoff for problematically low prosocial behavior on the SDQ is 5 and under (Youth in
Mind, 2016). For life satisfaction, internalizing problems, and externalizing problems scores in the
lowest or highest ~15% of the sample were considered high, as this is a common cutoff for clinical
issues (Schalet et al., 2014). The cutoff scores were 5 and under for life satisfaction, 9+ for
internalizing problems, and 11+ for externalizing problems.
Analytic Approach. We used the specifications of the original article (Przybylski, 2014):
sample weights but no controls. For linear analyzes to calculate r and r
2
, gaming hours were recoded as 0 = 0, less
than an hour = 0.5; 1 to 3 hours = 2; 4–6 hours = 5; 7 or more = 8. For relative risk, we compared
non-users to each of the other categories as is frequently done in studies of technology use and
mental health published in medical journals (eg, Kim et al., 2020; Messias et al., 2011; Riehm et
al., 2019).
Although the technique of comparing each category of use to a reference category is very
common, some may argue that it leaves out
twice as likely to have high levels of externalizing problems (13.4% vs. 27.9%, RR = 2.08); see
Table 1 and Fig. 1. Thus, although r and r
2
appear low,
comparisons using relative risk produce differences that seem practically significant.
Comparing two usage groups (up to an hour vs. an hour or more), and thus utilizing all of the
data, also produces seemingly important effects. For example, twice as many light/non-users of
console games (10.5%) than heavier users (21.4%) were low in prosocial behaviors. In addition,
55% more heavier users of computer games were high in externalizing symptoms (see
Supplemental Table 1). Thus, seemingly meaningful differences also appear when all of the data
are included.
discussion. Playing video games 4+ ha day is associated with a substantially increased risk
of negative outcomes such as externalizing problems and low prosocial behavior. Yet this the
same data, with identical analytical specifications for measures, weighting, and controls, that
yielded r
2
of 1 % of the variance in Przybylski (2014).
Why the discrepancy? Linear r shows how one variable can be pre dictated from another.
However, linear r cannot answer other important research questions, such as identifying the
increased risk of getting lung cancer if one smokes (vs. not) or the increased likelihood of having

Table 1
Electronic gaming and well-being, UK Understanding Society Youth Panel, Wave 1, UK.

Low prosocial behavior Low life satisfaction Internalizing problems Externalizing problems

Console gaming
Linear r
2
Contrast coded regression (0 vs. 4+), r 0.74% 0.46% 0.90% 1.2%
2
r (r ) ÿ 0.19*** (3.6%) 0.08*** (0.64%) 0.05** (0.25%) 0.17*** (2.9%)
relative risk
None 9.3% (1154) 16.0% (1146) 11.9% (1153) 13.4% (1153)
Ref. Ref. Ref. Ref.
0.5h 11.3 % (1614) 12.5% (1603) 11.1 % (1612) 11.0 % (1610)
1.22 (0.97, 1.53) 0.78 (0.65, 0.94) 0.93 (0.76, 1.52) 0.82 (0.67, 1.01)
1–3 hours 19.5 % (1410) 16.0% (1401) 13.4 % (1411) 16.8 % (1412)
2.10 (1.71, 2.59) 1.00 (0.84, 1.20) 1.13 (0.92, 1.38) 1.25 (1.04, 1.51)
4+ hours 30.6 % (297) 23.9% (299) 14.2 % (295) 27.9 % (295)
3.09 (2.39, 3.98) 1.49 (1.17, 1.90) 0.82 (0.87, 1.65) 2.08 (1.64, 2.63)
computer gaming
linear r
2
Contrast coded regression (0 vs. 4+), r 0.29% 0.81% 0.90%^ 1.5%
2
r (r ) all values ÿ 0.09*** (0.81%) ÿ 0.15*** (2.3%) 0.11*** (1.2%) 0.16*** (2.6%)
relative risk
None 13.4% (654) 14.5% (649) 8.8% (654) 11.9% (652)
Ref Ref Ref Ref.
0.5h 12.8% (1640) 10.9% (1625) 10.0% (1636) 10.5% (1636)
0.96 (0.79, 1.16) 0.75 (0.60, 0.95) 1.13 (0.85, 1.50) 0.87 (0.68, 1.12)
1–3 hrs 14.1% (1902) 15.7% (1886) 13.1% (1901) 15.1% (1901)
1.05 (0.88, 1.28) 1.08 (0.87, 1.34) 1.48 (1.13, 1.94) 1.26 (1.00, 1.59)
4+ hrs 19.4% (404) 27.2% (409) 18.9% (404) 25.5% (404)
1.44 (1.13, 1.85) 1.87 (1.46, 2.39) 2.12 (1.54, 2.92) 2.13 (1.63, 2.78)
2
Notes: 1. Contrast-coded regressions are from Przybylski (2014). 2. Linear r's are partial correlations with controls, and r are given as percentages (1 % = 0.01, given
^
the common use of “percent variance explained”). 3. = reported as 9 % in Przybylski (2014); assumed to be 0.90 % given other results. 4. For relative risk cells, numbers are % with low well-being, (n),
relative risk, and (95 % confidence interval). 5. Specifications: Weighted, no controls. 6. RR's with 95 % CI's not including 1 are in bold.

3
Machine Translated by Google
J.M. Twenge and J.L. Hamilton
30
25
twenty
iee%
gnllw ow
bl
fifteen
10
Hours a day console gaming
Fig. 1. Associations between console gaming time and low well-being, UK Understanding Society Youth Panel Wave 1.
externalizing problems if one plays video games 4+ ha day (vs. not at all). Comparing
outcomes between groups using relative risk can do that, and it appears that there
is in fact a practically important association between playing video games more
frequently and negatively out comes such as externalizing problems. This was true
whether relative risk calculations compared each group to non-users or included all
of the data by combining usage groups to make a dichotomous variable. Thus, the
association between time spent on technology and mental health outcomes can
appear small and supposedly insignificant when only linear r is examined, but more
practically significant when statistics more commonly used in medicine and public
health are employed.
3. Study 2: Youth Risk Behavior Surveillance System 2013
In Study 2, we sought to examine differences among effect size metrics in
another large study measuring digital media use and well being originally analyzed
using linear regression (Ferguson, 2017).
This study includes several dichotomous measures of depression, addressing
concerns about dichotomizing continuous variables. With an ordinal (not continuous)
measure of technology use and dichotomous measures of depression and suicidal
ideation, this dataset is an excellent candidate for the use of relative risk.
Method. The 2013 Youth Risk Behavior Surveillance System (YRBSS) collected
data from youth from the US state of Florida in 2013.
Participants included 6089 adolescents in 9th through 12th grade (ages 12–18) who
were 51% female. Participants reported their hours per day of 1) television and 2)
electronic device use. In the original analysis, these two items were added to create
a total “screen time” measure.
To measure depression, Ferguson (2017) added together three dichotomous items
(feeling sad or hopeless for two weeks or more, thinking
4
Psychological Act 229 (2022) 103696
outsourcing
problems
high
internalizing
problems
high
prosocial
behavior low
Life
sasfacon
low
about suicide, and making a suicide plan) over the last 12 months.
Analytic Approach. Ferguson (2017) analyzed the 2013 YRBSS data using four
contrast-coded regressions by dividing total screen time into four groups: abstainers
(no screen time), low users (<2 h/day), mod erate users (3–6 h/ day defined as +/ÿ
1 SD of mean on screen time), and excessive users (>1 SD above the mean of
screen time). We used specifications based on Ferguson (2017): no weighting and
the inclusion of control variables for gender, sleep, and physical activity.
We also analyzed the data using the specifications of a previous analysis using
a similar dataset (Twenge et al., 2018): examining electronic device use separately,
including controls for race and grade level in addition to gender, and following
current guidance to exclude possible mediators (sleep, physical activity) from the
list of control variables (Rohrer, 2018; Schisterman et al., 2009). The YRBSS
measured electronic device time using the following response choices: None, less
than an hour, one hour, two hours, three hours, four hours, and five or more hours.
For linear r analyses, TV and electronic device use in hours was recoded as 0,
0.5, 1, 2, 3, 4, and 6. We examined mental health using both a summary variable
(endorsement of at least one of the three dichotomous depression and suicidal
ideation items) and by examining each dichotomous item separately. Percentage
increases for each hour of use were calculated by averaging relative risks between
consecutive usage levels.
Results. In the original contrast-coded regressions (Ferguson, 2017), 2 the r
for total screen time and the sum of the depression and suicide items was
1.7 %. However, when outcomes are compared between usage groups using
relative risk, 50% more heavy users of screens than non users endorsed at least
one of the depression and suicide items (see Table 2 and Fig. 2). The contrast
between the results using linear r and
Machine Translated by Google

J.M. Twenge and J.L. Hamilton Psychological Act 229 (2022) 103696

Table 2
Screen time or electronic device time and well-being, US Youth Risk Behavior Surveillance System, Florida, 2013.

At least 1 of 3 depression/suicide items Depression (Y/N) Suicidal thoughts (Y/N) Suicide plan (Y/N)

Total screen time

linear r
– – –
Contrast coded regression (0 vs. 7+), r (r)2 r (r 0.13*** (1.7%)
2
) 0.10*** (1.0%) 0.08*** (0.64%) 0.09*** (0.81%) 0.07*** (0.49%)
relative risk
None 25.0% (252) 22.2% (255) 12.1% (255) 10.4% (252)
Ref Ref Ref Ref
0.5–2.5h 27.3% (1633) 23.7% (1646) 11.4% (1646) 8.4% (1641)
1.09 (0.87, 1.37) 1.06 (0.83, 1.35) 0.94 (0.66, 1.34) 0.82 (0.55, 1.21)
3–6.5hrs 31.6% (2248) 26.8% (2277) 14.2% (2272) 10.6% (2257)
1.26 (1.01, 1.58) 1.20 (0.94, 1.52) 1.17 (0.83, 1.65) 1.03 (0.70, 1.51)
7+ hrs 37.5% (1150) 32.2% (1171) 19.0% (1167) 13.0% (1155)
1.50 (1.20, 1.88) 1.44 (1.13, 1.84 ) 1.56 (1.10, 2.22 ) 1.26 (0.85, 1.87)
Electronic device time
linear r
2
r (r) ) all values 0.13*** (1.7%) 0.13*** (1.7%) 0.13*** (1.7%) 0.10*** (1.0%)
relative risk
None 28.2% (911) 24.3% (921) 11.4% (881) 9.2% (876)
Ref. Ref. Ref. Ref.
<1h 26.4% (805) 23.3% (813) 11.6% (757) 7.7 % (753)0
0.94 (0.80, 1.09) 0.95 0(0.80, 1.12) 1.02 (0.78, 1.34) .83 (0.60, 1.15)
1 hour 22.5% (688) 19.8% (697) 11.1% (650) / 8.7 % (647)0
0.80 (0.67, 0.95) 0.81 (0.67, 0.98) 98 (0.73, 1.30) .94 (0.68, 1.30)
2 hours 25.9% (875) 22.7% (889) 10.4% (836)0 8.3 % (831)0
0.92 (0.79, 1.07) 0.93 (0.79, 1.10) .92 (0.70, 1.21) .90 (0.66, 1.22)
3 hours 29.4% (719) 23.9% (725) 14.8% (701) 10.6 % (698)
1.04 (0.90, 1.21) 0.98 (0.83, 1.17) 1.31 (1.01, 1.69) 1.15 (0.85, 1.55)
4 hours 35.6% (4 22 ) 28.0% ( 426) 18.6 % ( 409) 12.1% (407)
1.26 (1.07, 1.49) 1.15 (0.95, 1.39) 1.64 (1.24, 2.15) 1.30 (0.93, 1.82)
5+ hours 40.5% (1135) 34.9% (1154) 20.8% (1119) 14.2% (1106)
1.44 (1.27, 1.63) 1.44 (1.25, 1.65) 1.83 (1.48, 2.28) 1.54 (1.19, 1.98)
1 hour vs. 5+ 1.80 (1.54, 2.10) 1.76 (1.49, 2.09) 1.88 (1.47, 2.40) 1.64 (1.23, 2.19)
h Mean % increase with each hour after 1 h/day 15.9% 15.4% 18.4% 13.7%

Notes: 1. Contrast-coded regressions are from Ferguson (2017). 2. Linear r's are partial correlations with controls, and r 2 are given as percentages (1 % = 0.01, given the
common use of “percent variance explained”). 3. For relative risk cells, numbers are % endorsing at least one depression item, (n), relative risk, and (95 % confidence
interval). 4. Specifications: Total screen time: No weighting; controlled for gender, sleep, physical activity. Electronic device use only: Weighted; controlled for gender,
grade, and race. 5. RR's with 95% CI's not including 1 are in bold.

Relative risk was also apparent when examining each of the dichotomous
items separately. Although the r 2 for total screen time and suicidal
thoughts was 0.81 % (“less than 1% of the variance”), heavy users were
56 % more likely than non-users to have thoughts about suicide. How
ever, relative risks for total screen time and making a suicide plan were
not significant.
Using specifications based on another previous analysis (electronic
devices only, and controlling for gender, race, and grade) also produced
2 linear r and r often considered low, but notable comparisons between
groups. For example, 44% more heavy users (vs. non-users) endorsed
at least one depression or suicide item, and 83% more had thought
about suicide (see Table 3).
The dose-response curve for electronic device use shows evidence
of non-monotonic effects with the lowest depression among light users
(see Fig. 2), suggesting it may be useful to also compare light users (an
hour a day) to heavy users ( 5+ ha day). Heavy users were 80% more
likely to endorse at least one depression or suicide item than light users.
With each additional hour of electronic device use beyond one hour,
16% more adolescents were depressed, and 18% more thought about suicide.problems. Depression symptoms over the past 2 weeks were assessed
discussion. As in Study 1, smaller linear r's co-occur with
substantially higher risks between groups. This was true even when
examining items that were already dichotomous, suggesting that the
discrepancy between linear r and relative risk is similar whether continuous vari
ables are dichotomized or the measure is already dichotomous.
4. Study 3: Growing Up in Ireland
In Study 3, we examined differences between linear r and relative
risk in a study using contemporary time diaries to measure screen
time rather than the retrospective reports used in the other datasets.
Method. Data for Growing Up in Ireland (GUI) was collected August
2011 to March 2012 from 5023 13-year-olds (51% female) in primary
schools (recruited via random sampling; full study details are elsewhere
(Williams et al., 2009 ) ). Participants selected one activity (out of 21
pre-coded activities) for each 15-minute period on the time-use daily.
Following Orben and Przybylski (2019b), we examined the media
activities “using the internet/emailing (including social networking,
browsing etc.),” “playing computer games (eg, PlayStation, PSP, X-Box
or Wii),” “talking on the phone or texting,” or “watching TV, films, videos
or DVDs,” which were used to calculate the total number of hours that
participants engaged in electronic device use. To be consistent with
Orben & Przybylski, we include only time diaries from days during the
school term, and to be consistent with other datasets we included only
time diaries from weekdays (N = 3373). Well-being was measured using
the adjustment domains of the Strengths and Difficulties Questionnaire,
which included the sum of items assessing hyperactivity or inattention,
emotional symptoms, behavior problems, and peer-relationship
using the short form of the Mood and Feelings Questionnaire.
Analytic Approach. Consistent with the specifications of Orben and
Przybylski (2019b), weighting was not used and gender and age were
included as controls. Low well-being was defined as those scoring ~15%
above the mean, which was 8+ for depression and 11+ for the SDQ.
results. As Orben and Przybylski (2019b) did, we found small effect.
sizes when using r and r 2 However, the relative risk of depression was
53 % higher for heavy users (5+ h) than non-users (see Table 3 and Fig.
3). Each additional 2 h of screen time beyond 1 h was associated with
20% more youth high in depressive symptoms or adjustment

5
Machine Translated by Google

JM Twenge and JL Hamilton Psychological Act 229 (2022) 103696

40

35

electronic
device me

30

Total screen
I
sne
/noisla m
dosria eu%
cpei+ dd
1tsi

25

twenty

Hours a day electronic devices/ total screen me

Fig. 2. Associations between electronic device time/total screen time and depression, Youth Risk Surveillance System, Florida, 2013.

Table 3
Total screen time on weekdays and well-being, Growing up in Ireland.
depression SDQ adjustment problems
linear r
2
r (r ) 0.06*** (0.36%) 0.07*** (0.49%)
relative risk
None 12.1% (367) 14.7% (368)
Ref. Ref.
<1h 13.3% (473) 15.8% (476)
1.11 (0.78, 1.59) 1.07 (0.78, 1.48)
1–2.99 hrs 12.9% (1737) 14.4% (1740)
1.08 (0.79, 1.46) 0.98 (0.75, 1.29)
3–4.99 a.m. 13.5% (553) 18.7% (554)
1.13 (0.80, 1.60) 1.28 (0.95, 1.73)
5+ hours 18.2% (234) 20.6% (235)
1.53 (1.04, 2.26) 1.39 (0.98, 1.99)
<1 hour vs. 5+ hours 1.38 (0 .97, 1.97) 1.42 (1 .07, 1.87)
Mean % increase with each 2 h after 1 h/day 19.7% 20.0%
2
Notes: 1. Linear r's are partial correlations with controls, and r are given as percentages (1 % = 0.01, given the common use of “percent variance explained”). 2. For
relative risk cells, numbers are % endorsing at least one depression item, (n), relative risk, and (95 % confidence interval). 3. Specifications: During term, weekdays
only; no weighting; control for gender and age. 4. RR's with 95% CI's not including 1 are in bold.

difficulties.
discussion. Orben and Przybylski (2019b) concluded that the superior and
less confounded measurement of time diaries might explain the effects in the
GUI were smaller than those found in studies using retrospective reports.
However, comparisons using relative risk in this dataset are fairly similar to
those found in studies using retrospective reports, suggesting the difference
might instead lie in the statistical test used.
5. Study 4: New Zealand Attitudes and Values Study
In Study 4, we examined adults from the New Zealand Attitudes and
Values Study (NZAVS). Data were generously supplied by Stronge et al.
(2019). This dataset provides a view of technology use and mental health
among adults using a mental health measure with a well-established cutoff for
serious issues.
Method. The 2016 NZAVS included complete responses from 18,438
participants from a national probability sample of New Zealand adults (mean
age = 49.45, 62 % women). Participants answered an array of demographic
questions, two questions about health indicators (average sleep time and body
mass index) as well as estimating the number of hours a week they spent on
social media, working, doing housework, looking after children, exercising, using
the internet, playing video/computer games, commuting, watching or reading
the news, watching

6
Machine Translated by Google
J.M. Twenge and J.L. Hamilton
twenty
hgct%
ro
eh iw
hs
fifteen
10
Hours a day total screen me
Fig. 3. Associations between screen time and depression/SDQ adjustment, Growing up in Ireland.
TV/Netflix/movies, and volunteering. Participants also completed the
Kessler-6, a widely used screening tool for serious mental distress that
shows substantial concordance with DSM structured clinical interviews
(Kessler et al., 2003, 2010). Serious mental distress is indicated with
scores of 13 or higher (Prochaska et al., 2012).
Analytic Approach. We used the same specifications as Stronge
et al. (2019): No weighting and entering all other variables into the
regression equation. To follow current guidance not to include possible
mediators, we also performed an analysis excluding time use variables
and the two health indicator questions as controls.
results. As Stronge et al. (2019) also found, there was only a small
correlation between frequency of social media use and mental distress.
Yet heavy users were 41% more likely than non-users, and 62% more
likely than light users, to have serious mental distress. When possible
mediators are excluded as controls, heavy users were 81% more likely
than non-users and twice as likely than light users to have serious
mental distress (see Table 4 and Fig. 4).
discussion. Not surprisingly, given that they were relying on linear
regression and found small Betas, Stronge et al. (2019) concluded that
the practical impact of social media use on psychological distress is
“questionable” and that “social media use is typically not a serious risk
factor for psychological distress.” However, when comparing usage
groups, the risk of serious psychological distress is notably elevated
with heavy social media use.
6. Study 5: UK Understanding Society Youth Panel at Wave 8
In Study 5, we examined Wave 8 of the UK Understanding Society
Youth Panel, a dataset that has not previously been examined cross
sectionally in published research. Like the datasets in Studies 1 and 2,
the measure of time use is ordinal and not continuous.
Method. Wave 8 of the UK Understanding Society Young Panel
7
Psychological Act 229 (2022) 103696
SDQ
adjustment
problems
depression
included 1818 10- to 15-year-olds (51% female). To measure social
media use, participants responded to the following question, “How many
hours do you spend chatting or interacting with friends through a social
web-site or app like that on a normal school day?” Response options
included “None,” “Less than an hour,” “1–3 h,” “4–6 h,” and “7 or more
hours.” Similar to Study 1 (Wave 1 of the UK Understanding So ciety
Youth Panel), life satisfaction was assessed using the sum of par
ticipant's happiness across the six domains of school, schoolwork,
appearance, family, friends, and life as a whole. Response options
ranged from 1 ('completely happy') to 7 ('not at all happy') and were
recoded so higher scores reflect higher levels of life satisfaction. The
other measures from Study 1 (prosocial behavior, internalizing
problems, and externalizing problems) were not included in this wave
of the study.
Analytic Approach. Data were weighted by cross-sectional youth
interview weight (to correct for sampling biases and make the sample
nationally representative) and controlled for age, rural/urban location
and (for combined analyses) sex. For linear analyses, social media
hours were recoded to none = 0, <1 h = 0.50, 1–3 h = 2, 4–6 h = 5, and
7+ h = 8. For relative risk analyses, the cutoff for low life satisfaction
was the lowest 15 % (~1 SD below the mean: 4.83 or below). Given a
substantial non-monotonic effect (see Fig. 5), light users (<1 ha day)
served as the reference group for relative risk comparisons.
results. Social media use and in life satisfaction produced relatively.
low linear r and r 2 However, 65% more heavy users of social media (vs.
light users) experienced low life satisfaction (see Table 5 and Fig. 5).
With each additional two hours of social media use after one hour a
day, the chances of a youth being low in life satisfaction increased 28%.
discussion. These results again illustrate that the same data
producing linear r's often considered small also produces practically
significant relative risks. As in several other datasets, the association
was non-monotonic (consistent with some past research: Przybylski & Weinstein,
Machine Translated by Google

J.M. Twenge and J.L. Hamilton Psychological Act 229 (2022) 103696

Table
4 Social media use and serious mental distress, New Zealand adults.
Serious mental distress

Linear r 2
r (r) ) all controls 0.05*** (0.03%)
2
r (r) ) demo controls only 0.08*** (0.64%)
relative risk
All controls
None 6.1% (5385)
Ref
<1hr 5.3% (9755)
0.87 (0.76, 1.01)
1–2 hrs 5.3% (2503)
0.87 (0.72, 1.06)
2–3 hours 5.3% (895)
0.86 (0.64, 1.16)
4+ hrs 8.6% (421)
1.40 (1.01, 1.95) 1.61
<1h vs. 4+ hrs (1.17 , 2.23)
Non-mediator controls only
None 5.8% (5156)
Ref
<1hr 5.2% (9456)
0.90 (0.78, 1.03)
1–2 hrs 5.7% (2415)
0.99 (0.81, 1.20)
2–3 hours 6.2% (863)
1.08 (0.82, 1.43)
4+ hrs 10.5% (394)
1.79 (1.32, 2.45) 2.00
<1h vs. 4+ h (1.4 8 , 2.71) 29.3%
Mean % increase with each hour after 1 h/day

Notes: 1. Linear r's are partial correlations with controls, and r 2 are given as percentages (1 % = 0.01, given
the common use of “percent variance explained”). 2. For relative risk cells, numbers are % endorsing at least
one depression item, (n), relative risk, and (95 % confidence interval).

10

All controls
6
rstei%
rhite
sslsaueton m
iw
d
s

4 demographic
controls

Hours a day social media use

Fig. 4. Associations between social media time and serious mental distress, New Zealand adults.

8
Machine Translated by Google

J.M. Twenge and J.L. Hamilton Psychological Act 229 (2022) 103696

25

Girls
twenty

all
nocafswea%
fsili
o
n

fifteen

Boys

10

Hours a day social media use

Fig. 5. Associations between social media time and low life satisfaction, UK Understanding Society Youth Panel, Wave 8.

Table 5
Social media time and life satisfaction, UK Understanding Society Youth Panel, Wave 8.
all Girls Boys
linear r
2
r (r ) all values ÿ 0.14*** (2.0%) ÿ 0.15*** (2.3%) ÿ 0.10*** (1.0%)
relative risk
None 23.3% (113) 21.1% (32) 23.0% (81)
1.54 (1.06, 2.25) 1.27 (0.64, 2.54) 1.76 (1.11, 2.82)
<1h 15.0% (763) 17.3% (372) 13.2% (391)
Ref. Ref. Ref.
1–3 hrs 19.3% (646) 18.3 % (365) 20.4 % (281)
1.30 (1.03, 1.63) 1.07 (0.78, 1.46) 1.53 (1.08, 2.15)
4+ hrs 24.6% (296) 26.2 % (195) 21.3 % (101)
1.65 (1.27, 2.14) 1.52 (1.10, 2.10) 1.64 (1.05, 2.56)
Mean % increase with each 2 h after 1 h/day 28.1% 24.5 % 29.5 %
2
Notes: 1. Linear r's are partial correlations with controls, and r are given as percentages (1 % = 0.01, after the common use “percent variance explained”). 2.
Specifications: Weighted, controlled for age, urban/rural location, sex.

2017). Thus, the association may not be well-captured by an effect size metric such as and dismissed as producing only small effects would show substantial relative risks if
2
r (and r ) that assumes a linear, monotonic relationship analyzed that way? This is especially important for in investigations of technology use
(although a technique such as LOESS might capture it). and mental health as studies on this topic are published in journals across a variety of
disciplines, including psychology, communication, public health, and medicine. If studies
7. General discussion published in psychology and communication journals rely solely on linear r and conclude
that the association is small, and studies published in public health and medicine
Across five datasets on technology use and mental health, linear r's were at levels journals rely solely on relative risk and conclude that the association is practically
often considered small, while relative risk was seemingly practically important. This has significant, results appear mixed and general. conclusions are difficult to draw. Given
broad implications for psychology research as a whole: How many datasets analyzed this state of
with linear correlation

9
Machine Translated by Google
J.M. Twenge and J.L. Hamilton
the literature, it is tempting to conclude, as several observers have, that
studies in this area are inconclusive (Best et al., 2014; Meier & Reinecke,
2021), perhaps due to biased analysis choices (Orben & Przybylski,
2019a). However, when datasets are analyzed using both linear r and
relative risk, the associations are fairly consistent across studies. Thus,
the root of the disagreement across disciplines appears to primarily rest
not in different datasets, measures, or control variables, but in the
statistical researchers choose for their data analyses. This is not to say
that relative risk is better than linear r, but that both provide important
information.
There are also practical implications for therapists, physicians, policy
makers, and parents who wish to gauge the practical importance of as
associations to make well-informed decisions on issues around
technology use and mental health (such as, for example, whether heavy
technology use warrants more careful screening for mental health
issues, or, assuming some of the causation goes from technology use
to low well being, limiting hours of device use to moderate levels).
These decisions might be very different if decision-makers hear that
technology time explains <1 % of the variance in well-being (based on
linear r), as opposed to hearing that twice as many heavy users of
technology are depressed, dissatisfied with their lives, and/or disinclined
to help others (based on relative risk). These results suggest that linear
r's of ÿ 0.10, often considered “small,” are often not small in terms of
practical importance. As an additional example, the association between
child hood exposure to lead and adult IQ is r = 0.11 (Reuben et al., 2017).
Technology use and mental health are not unique examples of this
discrepancy. Linear r does not capture the substantial association be
tween smoking and lung cancer, receiving a COVID vaccine and
developing the disease, wearing a seat belt and dying in a car accident,
gender and suicide attempts, or BMI and exercise. Although the largest
dis crepancies between linear r and relative risk appear when base rates
are low (such as lung cancer), they also appear when base rates exceed
1 in 10 (such as exercise and obesity and in the datasets analyzed here
on technology use and mental health). Some have argued that medical
research examples like these should not be compared to psychology
examples in terms of effect size (Ferguson, 2009). However, as our
results demonstrate, the discrepancy between linear r and relative risk
appears when both are used to analyze the same data on psychological
questions. Thus, this is not just an issue of comparing medical research
to psychological research, but an issue of differences between statistical
tests.
We are not the first to point out the limitations of linear correlation
and percent variance explained (r-squared) as gauges of practical
importance. Rosnow and Rosenthal (1989, 2003) provided several
examples of low correlations that mask large relative risks. In their article
on effect sizes, Funder and Ozer (2019) concluded that 2r “Allows writers
to disparage certain findings that they find incompatible with their own
theoretical predilections.” There is also the matter of which research
question is actually being answered by the data at hand. Linear
correlation answers the question: How much can Y predict X among all
other factors? Given that most studies do not include all possible factors
that may influence outcomes (including genetic predisposition), that may
not be the research question most studies of behavior and well-being
are designed to answer. The relevant research question may instead
be: What is the difference in mental health factors between those who
behave one way vs. another? Comparing groups using relative risk thus
can provide information that linear correlation cannot.
Given the limitations of linear correlations, research psychology may
want to consider including relative risk comparisons in addition to linear
r. In this we compete with Funder and Ozer (2019), who recommend
that researchers focus on effect sizes capturing the number of people
affected. This could be done in addition to reporting linear r or d; thus, it
would add information rather than taking anything away, especially as
each statistic has its strengths and weaknesses. If this recommendation
is headed, some principles should be kept in mind to minimize the
potential downsides of using relative risk. First, relative risk should not be
10
Psychological Act 229 (2022) 103696
reported without the absolute numbers on which it is based. A relative
risk of 2 could mean an increase in prevalence from 1% to 2%, from
10% to 20%, or from 40% to 80%, all of which may have different
implications for practical importance. Second, decisions for selecting
groups must be explained; the necessity of cutting continuous variables
to use relative risk is one of the primary reasons linear r is more often
used in psychology. In some cases, creating groups to compare is
straightforward: For example, if hours of technology use are measured
within categorical survey response categories (“none,” “less than an
hour,” etc.), or if well-being measures are dichotomous (eg, a yes or no
answer to whether a respondent has self-harmed in the last year). Some
continuous mental health measures have established cutoffs in the
published literature (eg, a score of 13 or above is indicative of serious
mental distress on the Kessler-6), while other cutoffs may need to be
set with by other means, such as the commonly used guideline of a
score one standard deviation above (or below) the mean (Schalet et al., 2014).
However, dichotomization does not appear to be the reason for the
discrepancies between linear r and relative risk; as Study 2 showed,
similar discrepancies between linear r and relative risk appear whether
mental health measures are continuous or already dichotomous. Third,
even if researchers choose not to use relative risk, including a figure
showing the data distribution between the primary variables should be
standard. Figures immediately reveal non-monotonic (curvilinear) pat
terns, as well as illustrating the size of differences across groups.
Of course, correlational studies like these cannot determine
causation (for example, whether technology use causes depression,
depression causes technology use, or third variables cause both). Thus,
statements such as “heavy users are 50% more likely to be depressed”
could instead note that “depressed people are 50% more likely to be heavy users.”
Although several random-assignment experiments have established a
causal link between social media use and lower well-being (eg, Allcott
et al., 2020), studies of association such those analyzed here should
not be assumed to show the direction of the causal arrow . In addition,
more research is needed to establish mediators of the link – if heavy
technology use does cause depression, why, and how?
In conclusion, researchers need to consider more than linear
correlation when trying to establish the association between two
variables. As the examples of smoking and lung cancer, seat belts and
car accident deaths, and exercise and BMI show, linear correlation can
lead one to conclude that associations are not practically important
when in fact they are. Five datasets on technology use and mental
health also show that data with relatively small linear correlations can
produce substantial differences between groups. It may be time for
psychology to regularly add relative risk comparisons in research.
Supplementary data to this article can be found online at https://doi.
org/10.1016/j.actpsy.2022.103696.
Declaration of competing interest
JMT has received speaking honoraria for presenting research, has
consulted for law firms and state attorneys general about social media
and mental health, and receives royalties from several books, most
recently iGen: Why Today's Super- Connected Kids Are Growing Up
Less Rebellious, More Tolerant , Less Happy – and Completely
Unprepared for Adulthood. JLH declares no conflicts of interest.
Acknowledgments
The authors thank David Stein for statistical consulting. JLH was
supported by an Institutional National Research Service Award by the
National Heart, Lung, and Blood Institute (NHLBI: T32).
References
Allcott, H., Braghieri, L., Eichmeyer, S., & Gentzkow, M. (2020). The welfare effects of
social media. American Economic Review, 110, 629–676.
Machine Translated by Google
J.M. Twenge and J.L. Hamilton
Anderson, CA, Bushman, BJ, Bartholow, BD, Cantor, J., Christakis, D., Coyne, SM,
Donnerstein, E., Brockmyer, JF, Gentile, DA, Green, S., Huesmann, R.,
Hummer , T., Krah'e, B., Strasburger, V.C., Warburton, W., Wilson, B.J., & Ybarra, M.
(2017). Screen violence and youth behavior. Pediatrics, 140(5, Supp 2). S142–S147.
Andrade, C. (2015). Understanding relative risk, odds ratio, and related terms: As simple as it
can be obtained. Clinical and Practical Psychopharmacology, 76, e857–e861.
Baiden, P., LaBrenz, CA, Asiedua-Baiden, G., & Muehlenkamp, JJ (2020). Examining the
intersection of race/ethnicity and sexual orientation on suicidal ideation and suicide
attempt among adolescents: Findings from the 2017 youth risk behavior survey.
Journal of Psychiatric Research, 125, 13–20.
BBC. (2019, May 7). Social media effect 'tiny' in adolescents, large study finds.
https://www.bbc.com/news/health-48147378.
Berryman, C., Ferguson, C.J., & Negy, C. (2018). Social media use and mental health
among young adults. Psychiatric Quarterly, 89, 307–314.
Best, P., Manktelow, R., & Taylor, B. (2014). Online communication, social media and
adolescent wellbeing: A systematic narrative review. Children and Youth Services
Review, 41, 27–36.
Bhopal, A., Peake, MD, Gilligan, D., & Cosford, P. (2019). Lung cancer never
smokers: a hidden disease. Journal of the Royal Society of Medicine, 112(7), 269–271.
Bushman, B.J., & Huesmann, L.R. (2014). Twenty-five years of research on violence in digital
games and aggression revisited: A reply to elson and Ferguson (2013).
European Psychologist, 19, 47–55.
Centers for Disease Control. (2020). Cigarette Smoking Among US Adults Hits All-Time Low.
https://www.cdc.gov/media/releases/2019/p1114-smoking-low.html.
Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25(1), 7–
29. https://doi.org/10.1177/0956797613504966 Edgar
Snyder and Associates. (2020). Seat belt statistics. Retrieved from: https://www.
edgarsnyder.com/car-accident/defective-products/seat-belts/seat-belts-statistics.ht ml.
Elson, M., & Ferguson, C.J. (2014). Twenty-five years of research on violence in digital games
and aggression: Empirical evidence, perspectives, and a debate gone astray.
European Psychologist, 19, 33–46.
Ferguson, C. (2009). Is psychological research really as good as medical research? Effect size
comparisons between psychology and medicine. Review of General Psychology, 13,
130–136.
Ferguson, C. (2017). Everything in moderation: Moderate use of screens unassociated with
child behavior problems. Psychiatric Quarterly, 88, 797–805. https://doi.org/ 10.1007/
s11126-016-9486-3
Funder, D.C., & Ozer, D.J. (2019). Evaluating effect size in psychological research: Sense
and nonsense. Advances in Methods and Practices in Psychological Science, 2, 156–
168.
Goodman, R. (1997). The strengths and difficulties questionnaire: A research note.
Journal of Clinical Psychology and Psychiatry, 38, 581–586.
Greenwald, AG, Banaji, MR, & Nosek, BA (2015). Statistically small effects of the implicit
association test can have societally large effects. Journal of Personality and Social
Psychology, 108, 553–561.
Ivie, EJ, Pettit, A., Moses, LJ, & Allen, NB (2020). A meta-analysis of the association between
adolescent social media use and depressive symptoms. Journal of Affective Disorders,
275, 165–174.
Kelly, Y., Zilanawala, A., Booker, C., & Sacker, A. (2019). Social media use and
adolescent mental health: Findings from the UK millennium cohort study.
EClinicalMedicine, 6, 59–68. https://doi.org/10.1016/j.eclinm.2018.12.005
Kessler, RC, Barker, PR, Colpe, LJ, Epstein, JF, Gfroerer, JC, Hiripi, E.,
Howes, MJ, Normand, SL, Manderscheid, RW, Walters, EE, & Zaslavsky, AM
(2003). Screening for serious mental distress in the general population. Archives of
General Psychiatry, 60, 184–189.
Kessler, RC, Green, JG, Gruber, MJ, Sampson, NA, Bromet, E., Cuitan, M., …
Zaslavsky, AM (2010). Screening for serious mental distress in the general
population with the K6 screening scale: Results from the WHO world mental health (WMH)
survey initiative. International Journal of Methods in Psychiatric Research, 19 (Suppl. 1),
4–22.
Kim, S., Favotto, L., Halladay, J., Wang, L., Boyle, M.H., & Georgiades, K. (2020).
Differential associations between passive and active forms of screen time and
adolescent mood and anxiety disorders. Social Psychiatry and Psychiatric
Epidemiology, 55, 1469–1478.
Kremer, P., Elshaug, C., Leslie, E., Toumbourou, J.W., Patton, G.C., & Williams, J.
(2014). Physical activity, leisure-time screen use and depression among children and
young adolescents. Journal of Science and Medicine in Sport, 17, 183–187.
Lin, L.Y., Sidani, J.E., Shensa, A., Radovic, A., Miller, E., Colditz, J.B., & Primack, B.A.
(2016). Association between social media use and depression among US young adults.
Depression and Anxiety, 33, 323–331. https://doi.org/10.1002/da.22466 Loftus, P.
(2020, November 30). Moderna asks health regulators to authorize its COVID 19 vaccine.
Wall Street Journal. https://www.wsj.com/articles/moderna-to-ask-h ealth-regulators-to-
authorize-its-covid-19-vaccine-11606737602.
Mathers, M., Canterford, L., Olds, T., Hesketh, K., Ridley, K., & Wake, M. (2009).
Electronic media use and adolescent health and well-being: Cross-sectional
community study. Academic Pediatrics, 9, 307–314.
Meier, A., & Reinecke, L. (2021). Computer-mediated communication, social media, and mental
health: A conceptual and empirical meta-review. Communication Research, 48 (8), 1182–
1209.
Messias, E., Castro, J., Saini, A., Usman, M., & Peeples, D. (2011). Sadness, suicide, and
their association with video game and internet overuse among adolescents: Results from
the youth risk behavior survey 2007 and 2009. Suicide and Life-threatening Behavior, 41,
307–315.
eleven
Psychological Act 229 (2022) 103696
Orben, A., Dienlin, T., & Przybylski, A.K. (2019). Social media's enduring effect on
adolescent life satisfaction. PNAS Proceedings of the National Academy of Sciences of
the United States of America, 116(21), 10226–10228.
Orben, A., & Przybylski, A.K. (2019a). The association between adolescent well-being and
digital technology use. Nature Human Behavior, 3, 173–182. https://doi.org/ 10.1038/
s41562-018-0506-1
Orben, A., & Przybylski, AK (2019b). Screens, teens, and psychological well-being: Evidence
from three time-use-daily studies. Psychological Science, 30, 682–696. https://doi.org/
10.1177/0956797619830329
Oswald, FL, Mitchell, G., Blanton, H., Jaccard, J., & Tetlock, PE (2013). Predicting ethnic and
racial discrimination: A meta-analysis of IAT criterion studies. Journal of Personality and
Social Psychology, 105(2), 171–192.
Pearce, N. (2010). Epidemiology in a changing world: Variation, causation, and
ubiquitous risk factors. International Journal of Epidemiology, 40, 503–512.
Petrie, KJ, Pressman, SD, Pennebaker, JW, Øverland, S., Tell, GS, & Sivertsen, B.
(2018). Which aspects of positive affect are related to mortality? Results from a general
population longitudinal study. Annals of Behavioral Medicine, 52, 571–581.
Prescott, A.T., Sargent, J.D., & Hull, J.G. (2018). Meta-analysis of the relationship between
violent video game play and physical aggression over time. PNAS, 115, 9882–9888.
Primack, BA, Shensa, A., Sidani, JE, Whaite, EO, Lin, LY, Rosen, D., … Miller, E.
(2017). Social media use and perceived social isolation among young adults in the U.
S. American Journal of Preventive Medicine, 53, 1–8.
Primack, BA, Shensa, A., Sidani, JE, Escobar-Viera, CG, & Fine, MJ (2021).
Temporal associations between social media use and depression. American Journal of
Preventive Medicine, 60, 179–188.
Prochaska, J.J., Sung, H.-Y., Max, W., Shi, Y., & Ong, M. (2012). Validity study of the K6 scale
as a measure of moderate mental distress based on mental health treatment need and
utilization. International Journal of Methods in Psychiatric Research, 21, 88–97.
Przybylski, A.K. (2014). Electronic gaming and psychosocial adjustment. Pediatrics, 134
(3), e716–e722. https://doi.org/10.1542/peds.2013-4021
Przybylski, A.K., & Weinstein, N. (2017). A large-scale test of the goldilocks hypothesis.
Psychological Science, 28, 204–215. https://doi.org/10.1177/0956797616678438
Reuben, A., Caspi, A., Belsky, D.W., Broadbent, J., Harrington, H., Sugden, K.,
Houts, RM, Ramrakha, S., Poulton, R., & Moffitt, TE (2017). Association of childhood
blood lead levels with cognitive function and socioeconomic status at age 38 years and
with IQ change and socioeconomic mobility between childhood and adulthood. JAMA,
317, 1244–1251.
Riehm, KE, Feder, KA, Tormohlen, KN, Crum, RM, Young, AS, Green, KM, & Mojtabai, R.
(2019). Associations between time spent using social media and internalizing and
externalizing problems among US youth. JAMA Psychiatry, 1–9. https://doi.org/10.1001/
jamapsychiatry.2019.2325 Rohrer, JM (2018). Thinking
clearly about correlations and causation: Graphical causal models for observational data.
Advances in Methods and Practices in Psychological Science, 1, 27–42.
Rosnow, R.L., & Rosenthal, R. (1989). Definition and interpretation of interaction
effects. Psychological Bulletin, 105, 143–146.
Rosnow, R.L., & Rosenthal, R. (2003). Effect sizes for experimenting psychologists.
Canadian Journal of Experimental Psychology, 57, 221–237.
Sampasa-Kanyinga, H., & Lewis, R.F. (2015). Frequent use of social networking sites is
associated with poor psychological functioning among children and adolescents.
Cyberpsychology, Behavior, and Social Networking, 18, 380–385.
Schalet, B.D., Cook, K.F., Choi, S.W., & Cella, D. (2014). Establishing a common metric for
self-reported anxiety: Linking the MASQ, PANAS, and GAD-7 to PROMIS anxiety.
Journal of Anxiety Disorders, 28(1), 88–96. https://doi.org/10.1016/j.
janxdis.2013.11.006
Shewach, OR, Sackett, PR, & Quint, S. (2019). Stereotype threat effects in settings with
features likely versus unlikely in operational test settings: A meta-analysis.
Journal of Applied Psychology, 104, 1514–1534.
Schisterman, EF, Cole, SR, & Platt, RW (2009). Overadjustment bias and
unnecessary adjustment in epidemiologic studies. Epidemiology, 20(4), 488–495.
https://doi.org/10.1097/EDE.0b013e3181a819a1
Shensa, A., Sidani, JE, Escobar-Viera, CG, Switzer, GE, Primack, BA, & Choukas Bradley, S.
(2020). Emotional support from social media and face-to-face relationships:
Associations with depression risk among young adults. Journal of Affective Disorders,
260, 38–44.
Siegel, R.L., Miller, K.D., & Jamal, A. (2020). Cancer statistics, 2020. CA: A Cancer Journal,
70, 7–30.
Stronge, S., Mok, T., Ejova, A., Lee, C., Zubielevitch, E., Yogeeswaran, K., Hawi, D.,
Osborne, D., Bulbulia, J., & Sibley, CG (2019). Social media use is (weakly) related to
psychological distress. Cyberpsychology, Behavior, and Social Networking, 22, 604–
609.
Twenge, J., Joiner, TE, Rogers, AE, & Martin, G.N. (2018). Increases in depressive symptoms,
suicide-related outcomes, and suicide rates among US adolescents after 2010 and links
to increased new media screen time. Clinical Psychological Science, 6, 3–17.
Williams, J.S., Greene, S., Doyle, E., Harris, E., Layte, R., McCoy, S., & McCrory, C.
(2009). Growing up in Ireland national longitudinal study of children: The lives of 9-year
olds. Dublin, Ireland.
Wilson, D.B. (2020). Practical meta-analysis effect size calculator: 2 by 2 contingency
(frequency) table. https://campbellcollaboration.org/escalc/html/EffectSizeCa lculator-
R4.ph.
Youth in Mind. (2016). Scoring the strengths & difficulties questionnaire for ages 4–17 or 18+.
Retrieved from https://www.ehcap.co.uk/content/sites/ehcap/uploads/New sDocuments/
236/SDQEnglishUK4-17scoring-1.PDF.