Results

Descriptive Statistics

Amongst all participants, the data showed that liberal students at Northwestern University

tended to skew more towards the liberal extreme, with an average liberal extremism score of

4.95 out of 7. Within this sample, participants seemed to have a difficult time identifying

whether news articles were real or fake; both real and fake news accuracy scores were

approximately 50% across all participants. However, they were slightly better at identifying real

news articles as real than fake news articles as fake. When asked about confidence in identifying

fake news, participants reported significantly higher confidence pre-test (M = 6.16) in

comparison to the post-test self-reported confidence (M = 4.58). Notably, the mean of post-test

confidence was 4.58, which was interestingly high given most participants had only answered

approximately half of the questionnaire questions correctly.

For additional planned data analysis, participants were split by social media news usage.

The data indicated that liberal extremism increased with social media usage, meaning that

regardless of the social media platform, the more participants relied on social media for news,

the more likely they were to be liberally extreme. Descriptive results showed that participants

with High SMU of their news were 11.1% less accurate in identifying fake and real news based

on their mean overall test scores compared to participants with Low SMU of their news

consumption. Likewise, participants with High SMU were on average 17% less accurate in

identifying real news and 5.1% less accurate in detecting fake news than those with Low SMU.

Interestingly, the mean pre-test confidence levels were highest among the 24 participants who

self-reported the highest levels of social media usage for news.

Table 1.
Descriptive statistics for all participants of interest, separated by percentage of news consumed
through social media.
SMU Avg. Time on Pre-test Fake Real Overall Post-test
for liberal news conf. ident. ident. score conf.
news extremism score score

Low 13 12 13 13 13 13 13

Medium 23 23 21 23 23 23 18
N
High 24 22 23 24 24 24 21

Total 60 57 57 60 60 60 52

Low 4.69 (0.56) 6.67 (5.6) 5.15 (2.1) 0.57 (0.17) 0.64 (0.13) 0.61 (0.08) 4.00 (2.1)

Medium 4.64 (0.95) 6.13 (5.3) 6.38 (1.9) 0.46 (0.19) 0.52 (0.25) 0.49 (0.19) 4.89 (1.9)
M (SD)
High 5.38 (0.97) 5.36 (4.6) 6.52 (2.2) 0.52 (0.22) 0.47 (0.19) 0.49 (0.16) 4.67 (1.8)

Total 4.95 (0.95) 5.95 (5.0) 6.16 (2.1) 0.50 (0.20) 0.53 (0.21) 0.52 (0.17) 4.58 (1.9)

Low [4.00, 5.67] [1, 15] [2, 8] [0.38, 1.00] [0.50, 0.88] [0.50, 0.75] [1, 7]

Medium [2.50, 6.00] [1, 20] [1, 10] [0.00, 0.88] [0.00, 0.88] [0.00, 0.88] [1, 9]
Range
High [3.00, 7.00] [1, 22] [2, 10] [0.00, 0.88] [0.00, 0.75] [0.00, 0.69] [2, 9]

Total [2.50, 7.00] [1, 22] [1, 10] [0.00, 1.00] [0.00 0.88] [0.00, 0.88] [1, 9]
Note. SMU for news stands for “social media usage for news” in percentage groups, where Low
ranges 0-25%, Medium is 50%, and High ranges 75-100%. Time spent on news is in hours per
week. Identification scores range from 0 (fully incorrect) or 1 (fully correct).

Planned Analyses

Liberal Extremism vs News Identification Scores

We found a non-significant, weak inverse relationship between liberal extremism and

overall news identification score (r(57) = -0.059, p = 0.655, 95% CI: [-0.308, 0.198]): having

more extreme liberal beliefs did not predict lower overall news identification performance.

Although the correlation value demonstrated there is a weak, inverse relationship between the
two factors, the p value indicated this correlation to be statistically insignificant. Given these

results, we failed to reject the null hypothesis that there is no effect of liberal extremism on news

identification performance.

Another Pearson’s correlation model revealed a non-significant weak direct relationship

between liberal extremism and fake news identification score (r(57) = 0.058, p = 0.658, 95% CI:

[-0.199, 0.308]): having more extreme liberal beliefs did not predict lower fake news

identification performance. Although the correlation value demonstrated there is a weak direct

relationship between the two factors, the p value indicated this correlation to be statistically

insignificant. Given these results, we failed to reject the null hypothesis that there is no effect of

liberal extremism on fake news susceptibility.

The same Pearson’s correlation model lastly revealed a non-significant weak inverse

relationship between liberal extremism and real news identification score (r(57) = -0.147, p =

0.263, 95% CI: [-0.386, 0.111]): having more extreme liberal beliefs did not predict lower real

news identification performance. Although the correlation value demonstrated there is a weak

inverse relationship between the two factors, the p value indicated this correlation to be

statistically insignificant. Given these results, we failed to reject the null hypothesis that there is

no effect of liberal extremism on real news identification performance.

Social Media News Consumption vs News Identification Scores

We conducted one-way ANOVA tests to compare the effect of the amount of news

ingested by social media on news identification scores. There was not a statistically significant

difference in mean fake identification score across social media usage groups (F(2, 33.4) = 1.57,

p = 0.222, ηp2 = 0.045): although 4.5% of variance in fake news identification performance can

be explained by social media usage, the effect was not statistically significant, so we failed to
reject the null hypothesis. However, there was a statistically significant difference in mean real

news identification score across social media usage groups (F(2, 36) = 5.24, p = 0.010, ηp2 =

0.092): 9.2% of variance in real news identification performance could be attributed to social

media usage, with the difference between Low and High usage providing the greatest statistical

and practical significance (t = 0.170, p = 0.051). In addition, there was a statistically significant

difference in mean overall score across social media usage groups (F(2, 37.8) = 5.42, p = 0.009,

ηp2 = 0.082): 8.2% of variance in overall performance could be attributed to social media usage,

with the difference between Low and Medium usage (t = 0.117, p = 0.102) along with Low and

High usage (t = 0.111, p = 0.122) providing the greatest statistical and practical significance.

Thus, social media usage revealed to be a strong predictor of real news identification

performance and overall news identification performance, especially for lower usage groups

where we observed the best performance.

Figure 3.

Mean overall identification score by social media use

Figure 4.

Mean fake identification score by social media use

Figure 5.

Mean real identification score by social media use

Exploratory Analyses

Confidence

We found a moderately strong, positive correlation between pre-test confidence and post-

test confidence (r(52) = 0.701, p < 0.001); while average post-test confidence was lower,

participants who were more confident pre-test were also more confident post-test. Pre-test

confidence was not correlated with fake news identification score at all (r(57) = 0.001, p =

0.996); this result was extremely statistically insignificant, indicating that pre-test confidence is

likely a completely random predictor of fake identification score. Post-test confidence was a

somewhat better predictor of fake news identification score (r(57) = 0.057, p = 0.689) but was

still statistically insignificant and weak (Table 2). These results were insufficient to determine

whether greater average confidence would be associated with lower fake news identification

scores; thus, confidence may not actually be a predictor of ability to detect fake news.

Interestingly, weekly time spent on news had a moderate correlation with post-test

confidence and was statistically significant (r(50) = 0.327, p = 0.020) but pre-test confidence was

not (r(55) = 0.047, p = 0.735). This indicated that students who spent more time ingesting news
were more likely to report higher amounts of confidence after taking the test than students who

spent less time ingesting news; furthermore, students who spent more time ingesting news were

no more likely to report higher levels of confidence prior to taking the test in comparison to

students who spent less time ingesting news.

News Literacy

Additionally, we found that news literacy was not a statistically significant predictor of

fake news susceptibility. Independent samples t-tests revealed non-significant differences with

regards to fake news identification score (t(57) = -1.220, p = 0.228, d = -0.39, 95% CI: [-1.031,

0.247]), real news identification score (t(57) = -0.265, p = 0.792, d = -0.086, 95% CI: [-0.719,

0.549]), and overall news identification score (t(57) = -0.944, p = 0.349, d = -0.31, 95% CI: [-

0.941, 0.333]) between those that have taken a news class before and those that have not. Even

though the Cohen’s d values for fake news scores and overall news scores indicate a medium

sized inverse relationship with news literacy, they are not statistically significant; hence, students

who took classes focused on news literacy were no more/less likely to identify real news as real

or fake news as fake in comparison to students who have not taken classes focused on news

literacy.

Relation between SMU and Liberal Extremism

Lastly, we conducted a one-way ANOVA test to compare the effect of the amount of

news ingested via social media on liberal extremism (Figure 4). We found that social media

usage for news is a statistically significant effector of liberal extremism (F(2, 36.7) = 4.51, p =

0.018, ηp2 = 0.141): 14.1% of variance in average liberal extremism could be attributed to social

media usage for news. Thus, social media usage was revealed to be an extremely important

predictor of liberal extremism, where the difference between Low and Medium usage (t = -0.690,
p = 0.073) along with Low and High usage (t = -0.737, p = 0.017) provided the greatest

statistical and practical significance.

Figure 4.

ANOVA of social media usage vs average liberal extremism.

Table 2.
Correlation matrix of all quantitative continuous variables.
Avg. liberal Overall Fake Real Time Pre-test Post-test
extremism score ident. ident. on confidence confidence
score score news

Avg. liberal r —
extremism

p —
Overall score r -0.059 —

p 0.66 —
Fake ident. r 0.058 0.79*** —
score

p 0.66 < .001 —

Real ident. r -0.15 0.81*** 0.28* —
score

p 0.26 < .001 0.034 —

Time spent on r -0.062 0.035 0.14 -0.078 —
news
p 0.65 0.80 0.30 0.57 —
Pre-test r -0.089 -0.042 0.001 -0.062 0.047 —
confidence
p 0.51 0.76 0.99 0.65 0.74 —
Post-test r -0.13 0.096 0.057 0.077 0.33* 0.70*** —
confidence
p 0.34 0.50 0.69 0.59 0.020 < .001 —
Note. * p < .05, ** p < .01, *** p < .001
Note. SMU for news stands for “social media usage for news.” Calculated r value is Pearson’s r
value.

Discussion

Unsurprisingly, our study findings suggest that fake news is a very nuanced subject. A

part of our hypothesis was not supported: liberal extremism was not a significant predictor of

fake news identification performance. We hypothesized that being more extremely liberal would
decrease one’s ability to detect fake news, as they might be “blinded” by the actual facts an

article does or does not provide; however, this was not supported. No results for overall score,

fake news identification score, or real news identification score were statistically significant. Nor

did we find any correlation between extremism and ability to detect fake news. This is in line

with trends across the general United States population; regardless of political affiliation, people

generally overestimate their ability to detect fake news and are ultimately susceptible to fake

news online. Additionally, Casino and Jenkins found that members of both political parties and

ideologies can be susceptible to conspiracy theories and fake news. Given their findings on

susceptibility to conspiracy theories, it is possible that the headlines utilized in our study were

not quite to the level of extremity as studied in the past, and that may be why political extremism

did not seem to be a predictor for fake news identification. Alternatively, it is possible that being

more liberally extreme was not a large factor in fake news detection simply because there could

be a “baseline” that most people are susceptible to in the first place.

Despite liberal extremism not being a significant predictor of fake news identification, we

found that social media usage is a predictor of fake news identification. Social media usage and

overall identification scores were found to be very statistically and practically significant,

indicating that social media usage appears to have an effect on fake news detection, which

supports our hypothesis. In fact, social media usage accounted for 8% of all variance in possible

reasons for fake news identification, showing there is likely a strong relationship between the

two. This was seen further when examining the real news identification scores: again, there was

a relationship with social media usage, this time accounting for 9% of variance. Lastly,

participants who reported Low social media usage for news scored better across all metrics,

compared to those who had Medium and High social media usage, once again supporting our
hypothesis. This is supported by Allcott and Gentzkow, who found that those who use more

social media are more likely to believe ideology-aligned articles—since we were examining

liberal participants, those who had High social media usage likely could have immediately

believed any headlines from liberal news sources, while those with Low social media usage

could have been more skeptical. Alternatively, this could be explained by confirmation bias,

where liberal users with higher social media usage are more likely to be surrounded by liberal

media. Thus, they are more likely to believe liberal headlines.

Given the statistically insignificant relationship between liberal extremism and fake news

detection capabilities, as well as the significant relationship between social media usage and fake

news detection capabilities, we looked to identify any relationship between liberal extremism

and social media usage. Social media usage is a significant predictor of liberal extremism,

accounting for 14% of variance. Confirmation bias could explain this large effect. The more time

a person spends on social media, the more likely their social network is to be aligned with their

political ideologies.

Limitations and Future Directions

Limitations of our study include having exclusively liberal participants, underpowered

samples, and the lack of consideration for misinformation vs disinformation of fake news.

Northwestern University is an institution known for its liberal student body, and our sample

consisted of only Northwestern students. Thus, our sample was heavily concentrated with liberal

extremes, as reflected in the average liberal extremism score of 4.95 out of 7. If our sample

included more participants across the liberal extremism spectrum, we could have looked at

differences between those with a more moderate liberal ideology and more extreme liberals.

Further, had our study included an equal number of liberal, moderate, and conservative
participants, we would be able to examine fake news susceptibility predictors across all political

affiliations. This could provide insight as to whether the same fake news susceptibility predictors

are applicable to the entirety of the political spectrum.

Additionally, this study was underpowered. We estimated a minimum sample size of 82

participants was required to maintain a medium effect size. While there were initially 87

respondents to the survey, 25 participants were removed from our analysis after leaving the

survey incomplete or having a conservative or moderate political affiliation. This means that

despite significant findings regarding social media usage as a predictor of fake news

identification, there is a chance of a false positive given that our sample size was not large

enough.

As defined by the American Psychological Association, misinformation is false

information, whereas disinformation is information that was falsified with the intention of

misleading individuals. The wording in our survey was intentionally ambiguous around the word

“fake” as we were interested in how various participants would respond. However, because it

was not specified whether participants in our study were identifying fake news based on if they

thought the information was false or false with the intentions of misleading, it is unclear whether

participants identified fake news sources as fake due to misinformation, disinformation, or both.

It is possible participants would have scored differently on the overall score and fake news

identification score had there been more specification between the two.

Due to the limited number of studies available regarding liberal extremism and mode of

news consumption as potential predictors of fake news susceptibility, as well as the limitations of

our own study, more research is necessary to provide evidence for a significant relationship

between fake news susceptibility and liberal extremism or mode of news consumption. Future
researchers need to take into consideration the political affiliation of those sampled in order to

provide an inclusive sample of individuals across the liberal extremism spectrum as well as an

ample amount of individuals across the political spectrum, the power of the sample in order to

avoid false positives, and detailed definitions of misinformation and disinformation along with

specific instructions on which one is to be considered when answering questions about fake

news.

Conclusion

In conclusion, we found that liberal extremism is not a significant predictor of fake news

identification. However, social media usage is a large, significant predictor of liberal extremism.

This suggests that low social media usage predicts better performance, especially at identifying

real news as real. Additionally, people are often far more confident at their abilities to detect fake

news than they actually are. It could be easy for malicious actors to prey on this disjunction,

creating disinformation that is intentionally difficult to detect. Possible warning signs or pre-

viewing programs could be implemented on social media based news sites themselves to combat

these findings, as investigated by Pennycook et al., who found that priming individuals to think

about accuracy resulted in a tripled level of discernment between sharing real and fake news in

relation to the control group who were not primed. With the continuous rise of both social media

and fake news on social media, it is recommended that this study be used for educational

purposes for not only politicians and voters, regardless of age, but the general public as well.