Accepted Manuscript

Impact of a Mental Health Curriculum for High School Students on Knowledge and
Stigma: A Randomized Controlled Trial

Robert Milin, MD, Stanley Kutcher, MD, Stephen P. Lewis, PhD, Selena Walker, MA,
Yifeng Wei, MEd, Natasha Ferrill, BA, Michael A. Armstrong, MA
PII: S0890-8567(16)30044-2
DOI: 10.1016/j.jaac.2016.02.018
Reference: JAAC 1392

To appear in: Journal of the American Academy of Child & Adolescent

Psychiatry

Received Date: 25 October 2015

Revised Date: 25 February 2016
Accepted Date: 25 February 2016

Please cite this article as: Milin R, Kutcher S, Lewis SP, Walker S, Wei Y, Ferrill N, Armstrong
MA, Impact of a Mental Health Curriculum for High School Students on Knowledge and Stigma: A
Randomized Controlled Trial, Journal of the American Academy of Child & Adolescent Psychiatry
(2016), doi: 10.1016/j.jaac.2016.02.018.

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to
our customers we are providing this early version of the manuscript. The manuscript will undergo
copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please
note that during the production process errors may be discovered which could affect the content, and all
legal disclaimers that apply to the journal pertain.
 ACCEPTED MANUSCRIPT
Classroom Mental Health Education

Impact of a Mental Health Curriculum for High School Students on Knowledge and Stigma:

A Randomized Controlled Trial

RH = Classroom Mental Health Education

Robert Milin, MD, Stanley Kutcher, MD, Stephen P. Lewis, PhD, Selena Walker, MA, Yifeng Wei,

PT
MEd, Natasha Ferrill, BA, Michael A. Armstrong, MA

RI
This article is discussed in an editorial by Dr. Bernice Pescosolido on page xx.

Clinical guidance is available at the end of this article.

SC
Supplemental material cited in this article is available online.

Accepted March 2, 2016

U
Dr. Milin, Mss. Walker and Ferrill and Mr. Armstrong are with The Royal Ottawa Mental Health
Centre, Institute of Mental Health Research, University of Ottawa, Ottawa. Dr. Kutcher and Ms.
Wei are with IWK Health Centre, Dalhousie University, Halifax, NS, Canada. Dr. Lewis is with
AN
University of Guelph, Guelph, ON, Canada.

This study was funded by The Royal Ottawa Foundation and supported by the University of
Ottawa Institute of Mental Health Research, with grants from Telus, The Ottawa Senators
M

Foundation, and the Do It For Daron (DIFD) fund. The sponsors played no role in the conduct of
the study; collection, management, analysis, and interpretation of the data; preparation or
approval of the manuscript.
D

The authors would like to thank the Ottawa-Carleton District School Board, the Ottawa Catholic
School Board, and the Renfrew County District School Board for their assistance in this study, as
TE

well as the teachers and students who volunteered to be part of it. The authors would also like to
thank Darcy Santor, PhD, University of Ottawa, for his contribution to the items on attitudes
towards mental illness.

Disclosure: Dr. Milin has served on the Pediatric Advisory Board of US WorldMeds and has
EP

received speakers’ honoraria for CME activity from the American Academy of Addiction
Psychiatry. Dr. Kutcher serves as the Sun Life Financial Chair in Adolescent Mental Health.
Dr. Lewis, Mss. Walker, Wei, and Ferrill, and Mr. Armstrong report no biomedical financial
interests or potential conflicts of interest.
C

Correspondence to Robert Milin, MD, FRCPC, FAPA, Associate Professor, University of Ottawa,
AC

Department of Psychiatry, 1145 Carling Ave., Ottawa, Ontario, Canada K1Z 7K4; email:
Robert.Milin@theroyal.ca.
 ACCEPTED MANUSCRIPT
Classroom Mental Health Education

ABSTRACT
Objective: This study evaluated the effectiveness of a school-based mental health literacy intervention

for adolescents, on knowledge and stigma.

Method: Twenty-four high schools and a total of 534 students in the regional area of Ottawa, Canada

participated in this randomized controlled trial (RCT). Schools were randomly assigned to either the

PT
curriculum or control condition. The curriculum was integrated into grade 11 and 12 Provincial “Healthy

Living” courses, and delivered by teachers. Changes in mental health knowledge and stigma were

RI
measured using pre- and post-test questionnaires. Descriptive analyses were conducted to provide

sample characteristics, and multilevel modelling was used to examine study outcomes.

SC
Results: For the curriculum condition, there was a significant change in stigma scores over time (p =

.001) with positive attitudes towards mental illness increasing from pre to post. There was also a

U
significant change in knowledge scores over time (p < .001), with knowledge scores increasing from pre
AN
to post. No significant changes in knowledge or stigma were found for participants in the control condition.

A meaningful relationship was found whereby increases in knowledge significantly predicted increases in
M

positive attitudes towards mental health (p < .001).

Conclusion: This is the first large RCT to demonstrate the effectiveness in mental health literacy of an
D

integrated, manualized mental health educational resource for high school students on knowledge and
TE

stigma. Findings also support the applicability by teachers and suggest the potential for broad-based

implementation of the educational curriculum in high schools. Replication and further studies are

warranted.
EP

Clinical trial registration information—Impact of a Mental Health Curriculum for High School Students

on Knowledge and Stigma; http://clinicaltrials.gov/; NCT02561780

C

Key words: Mental Health Education, Stigma, Youth, Early Awareness, Classroom Intervention
AC

INTRODUCTION
1,2
Adolescence is a critical period for the onset of mental illness. If unrecognized and untreated,

early-onset mental disorders may lead to substantial negative personal, social, and community
2-6
consequences. The majority of mental disorders arising during adolescence are mild or moderate in
7 8,9
severity and respond well to evidence-based interventions. Early detection of a mental illness along

with early intervention may lead to better health outcomes and more positive attitudes towards mental
 ACCEPTED MANUSCRIPT
2

10
illness and help-seeking. Unfortunately, most young people needing mental health care do not access
11,12
or receive services when needed. Stigma and social discrimination continue to be an ongoing problem
13,14
for those living with mental illness, especially youth. The fear of stigma associated with mental illness
15
is a common barrier preventing youth from seeking help and accessing services. These issues reinforce

PT
the need for community-based early identification and intervention as part of a comprehensive approach

to address the mental health needs of adolescents.8,11 Mental health literacy is a foundation to effectively

RI
12,16-17
address youth mental health promotion, early intervention, and ongoing care.

Mental health literacy encompasses knowledge and skills addressing biological and psychosocial

SC
aspects of mental health to improve understanding of mental health and mental disorders, reduce stigma,
18
help early identification of mental disorders, and facilitate help-seeking behaviours in youth. Just as

U
health literacy is key to recognizing physical symptoms and seeking help, mental health literacy is key to
12
early recognition of mental health problems and help-seeking.
AN
Schools are ideal venues for mental health promotion and education, where youth are easily

accessible, teachers and students are both familiar with classroom-based learning, and mental health
M

knowledge can be normalized as part of education. 19,20,21 In a systematic review, Franklin et al. found

effect sizes ranging from small to large for most studies where mental health interventions were delivered
D

22
by teachers. Optimally, schools should be able to incorporate mental health education into existing
TE

23
curricula, delivered by teachers, to obtain positive mental health literacy outcomes, though this is not
24
currently an evidence-supported standard practice.
EP

To date, there is minimal empirical evidence and no consensus on the effectiveness of school-
12,24-26
based educational programs providing mental health literacy for students. Substantive conceptual,
C

12,25,27
design, and methodological problems are found in the extant research. We have undertaken a large
AC

randomized controlled trial (RCT) to evaluate the effectiveness of a mental health literacy intervention for

adolescents, integrated into the classroom and delivered by teachers, on knowledge and stigma.28

In the present study, we proposed three hypotheses regarding the impact of a mental health

curriculum for those participants that receive it: 1) there would be a significant increase in mental health

knowledge; 2) there would be a significant increase in positive attitudes towards mental illness/reduction

in stigma; and 3) the increase in knowledge would significantly predict the increase in positive attitudes
 ACCEPTED MANUSCRIPT
3

towards mental illness/reduction in stigma. The secondary objective of the study was to determine

applicability of an educational resource for mental health in high schools.

METHOD

Participants and Procedure

PT
All high schools in the regional area of Ottawa, Canada were offered the opportunity to

participate in the study. Thirty schools that agreed to participate were randomized, by a research

RI
assistant using a random number generator, to one of three arms: teaching as usual (TAU, or control),

curriculum, and curriculum + follow-up eLearning modules. The Mental Health and High School

SC
Curriculum Guide (The Curriculum Guide) was required to be integrated within the grade 11 or 12 Ontario

Provincial Ministry of Education Healthy Living course. The existing Healthy Living course did not contain

U
any mandatory or standardized mental health content. As such, any delivery of mental health content was

at the individual teacher’s discretion. The Curriculum Guide replaced any discretionary content on mental
AN
health within the Healthy Living course. This course was open to all students regardless of educational

stream (i.e., university or community college). Students were not aware of the study prior to registering for
M

the course, nor were they aware of the likelihood of receiving The Curriculum Guide. The eLearning

modules were only accessible after completion of the Healthy Living course.
D

All participants under the age of 18 provided written informed assent; written informed consent
TE

was obtained from the parent or legal guardian. Students that were 18 years of age or older provided

written informed consent. Students with an English reading and comprehension level lower than the sixth
EP

grade were excluded from participation in the study. No personal identifying information was collected

from study participants. Ethics approval for the study was obtained from the Research Ethics Board of the
C

Royal Ottawa Mental Health Centre, and from each of the participating school boards: the Ottawa-
AC

Carleton District School Board, the Ottawa Catholic School Board, and the Renfrew County District

School Board.

The Curriculum Guide was designed to improve mental health literacy, increase the

understanding of mental illness, reduce stigma associated with mental illness, and promote students’

help-seeking behaviours. It is the first comprehensive secondary school mental health and illness

curriculum reviewed by an expert team of curriculum specialists across Canada to ensure all content was
 ACCEPTED MANUSCRIPT
4

classroom appropriate. The curriculum has been endorsed by several national partners, including the

Canadian Mental Health Association, the Canadian Association for School Health, Curriculum Services
29
Canada, and the Centre of Excellence for Child and Adolescent Mental Health. The Curriculum Guide

contains six distinct modules: 1) stigma of mental illness; 2) understanding mental health and mental

PT
illness; 3) information on specific mental illnesses; 4) experiences of mental illness; 5) seeking help and

finding support; and 6) the importance of positive mental health. These six modules each contain a

RI
complete lesson plan, with embedded classroom activities, core, and supplementary resources. Modules

1 and 4 address stigma reduction through digital stories and video interview of youth with mental illness.

SC
Module 2 addresses understanding of mental health and brain functions. Module 3 addresses mental

illnesses and treatments. Modules 5 and 6 address help-seeking behaviours and mental health self-help

U
competencies. A teacher self-study guide enhances teachers’ understanding of the materials. There have

been revisions to the curriculum since the study was conducted. For further details, the latest version of
AN
the freely available curriculum can be found online at: www.teenmentalhealth.org/curriculum. Information
30
on the curriculum’s development and field testing has been described previously.
M

For schools randomized to the curriculum arms of the study, the research assistant (N.F.) trained

teachers of the Healthy Living course on The Curriculum Guide content in a half-day session. Teachers
D

then proceeded with the implementation of The Curriculum Guide, which requires approximately 6 hours
TE

of classroom time. They were given pedagogical flexibility in the delivery of The Curriculum Guide while

instructed to maintain fidelity of content in an effort to preserve the classroom experience for both
EP

22
teachers and students. The Curriculum Guide contains standardized core content that teachers must

adhere to, as well as recommended lesson plans and procedures. Following implementation, teachers
C

were provided with questionnaires to evaluate The Curriculum Guide and provide feedback on their
AC

experience.

Control schools received teaching as usual of the existing course. All students, irrespective of

randomization, received a resource sheet on publicly available, regional child and adolescent mental

health services. Once their participation in the study was complete, all control schools were offered

training in The Curriculum Guide for implementation in their classrooms.

 ACCEPTED MANUSCRIPT
5

Changes in mental health knowledge and attitudes towards mental illness/stigma were measured

using pre- and post-test questionnaires specifically developed by study investigators to coincide with the

curriculum and address study outcomes. The pre-test (Time 1) questionnaire was administered to all

participants just prior to the start of the in-class delivery of The Curriculum Guide or TAU. The post-test

PT
(Time 2) questionnaire was administered shortly after the completed delivery of The Curriculum Guide or

TAU. This timeframe varied depending on the length of time teachers took to deliver The Curriculum

RI
Guide; however, all pre-post questionnaires were completed within a maximum of 2 weeks before and

after implementation. Certain psychosocial demographic questions were included for the pre-test package

SC
(see Table 1); otherwise questionnaires were identical. Research assistants administered the pre- and

post-test questionnaires.

U
Measures

To evaluate differences between students taught The Curriculum Guide and those in the TAU
AN
group, students completed structured pre (Time 1) and post (Time 2) questionnaires containing the

primary outcome measures of mental health knowledge and attitudes towards mental illness/stigma. To
M

our knowledge, there are no published, standardized, psychometrically sound instruments to measure
25
these study outcome variables in youth. Two authors involved in field-testing of The Curriculum Guide
D

(S.K., Y.W.) developed knowledge items based on material found in the resource. Items for the stigma
TE

scale were developed by two of the authors (R.M., N.F.) in collaboration with other experts in the fields of

mental health and education (S.L.).

EP

Mental Health Knowledge: The knowledge component of the questionnaire contained 15 multiple

choice questions about mental illness and the treatment of mental disorders. In order to be included in the
C

analysis, students must have completed at least 12 of 15 questions. Total scores were computed, with
AC

higher scores indicating greater mental health knowledge. The internal consistency for the mental health

knowledge items was found to be 0.40 at time 1 (n=475) and 0.54 at time 2 (n=448) using Cronbach’s

alpha.

Attitudes Towards Mental Illness (Stigma): This section of the questionnaire contained 8

statements, rated on a Likert scale, about the possible characteristics of individuals with mental illness. In

order to be included in the analysis, students had to have completed a minimum of 6 of 8 items. Total
 ACCEPTED MANUSCRIPT
6

scores were computed, with higher scores indicating more positive attitudes towards individuals with

mental illness. The internal consistency as measured by Cronbach’s alpha for the stigma items was 0.65

at time 1 (n=516) and 0.68 at time 2 (n=471).

Statistical Analysis

PT
A power analysis was conducted to determine appropriate sample size. Descriptive analyses

were performed to provide sample characteristics for study participants. There were no methodological

RI
differences between the curriculum and the curriculum + follow-up e-learning module groups. Therefore,

consistent with the scope of this study, both curriculum groups were collapsed for the purpose of pre/post

SC
analyses of the in-class delivery of The Curriculum Guide.

Multilevel model (MLM) analyses were conducted using full maximum likelihood estimation (see

U
Supplement 1, available online). First, we investigated the effects of time (pre vs. post) and condition

(curriculum vs. TAU) on the continuous outcome variables knowledge and stigma (Models 1 and 2 in
AN
Supplement 1, available online). A third model examined the relationship between students’ knowledge

and stigma scores (Model 3 in Supplement 1, available online). Students were set as a random factor
M

with repeated measurements nested within student and upper levels of the hierarchy (classroom and

school) controlled for at level 2 with the condition (curriculum vs. TAU) variable. Subgroup analyses
D

consisted of two models including school stream at level 2 with the outcome variables knowledge and
TE

stigma (Models 4 and 5 in Supplement 1, available online). Intraclass correlation coefficients (ICC) were

calculated to assess dependence within the data.

EP

Post hoc Bonferroni adjusted analyses were conducted using MLM to investigate the estimated

marginal means of knowledge and stigma, respectively, for each time point within each level of condition.
C

All analyses were conducted using Statistical Package for the Social Sciences (SPSS; Version 22).
AC

RESULTS

Participants

After agreeing to participate, 5 schools were unable to meet study requirements and withdrew

post-randomization, prior to initiation of the study. One school with a classroom of 25 students (4% of the

overall sample) was excluded due to students’ low reading level. This resulted in a total of 24 participating

schools.
 ACCEPTED MANUSCRIPT
7

All teachers that received training delivered the curriculum in full. After completing the study, a

total of 23 teachers returned satisfaction and evaluation surveys providing feedback on their experience

with The Curriculum Guide. Most teachers implemented the guide in 4 weeks or less (65.2%; n=15). The

remainder of the teachers implemented the guide in 4-8 weeks (34.8%; n=8), and no teachers required

PT
more than 8 weeks to implement the program in their classroom. The majority of teachers (73.9%, n=17)

reported that the classroom materials, provided for the presentations and activities within the curriculum

RI
guide, were engaging for the students. Nearly all of the teachers (95.7%; n=22) reported the content of

the curriculum guide was relevant and age appropriate for their students, with 73.9% (n = 17) reporting

SC
there was sufficient time to be able to cover one module per class (75 minutes). After implementing The

Curriculum Guide, 78.3% (n=18) of the teachers felt more comfortable talking about mental health with

U
their students. Teachers overwhelmingly reported that their expectations of the curriculum guide were met

(95.7%; n=22), with most rating The Curriculum Guide as very good or excellent (78.2%; n=18).
AN
Recruitment of students took place between September 2012 and March 2013. Seven of the 541

students that agreed to participate withdrew, resulting in a total number of 534 participants across 24
M

schools (curriculum n=362; TAU n=172). Of the 534 students who participated in the study, 87.8%

(n=465) completed both pre- and post-questionnaires, 10.1% (n=56) completed only the pre-
D

questionnaire, and 2.2% (n=12) completed only the post-questionnaire. Fifty-six of the 523 students that
TE

completed the pre-questionnaire did not complete the post-questionnaire, resulting in an attrition rate of

10.7% (Figure 1).

EP

The average age of participants was 16.5 years (SD=0.98). There were significantly more

females in the TAU group, and significantly more students in the TAU group were in the university stream
C

than expected (Table 1). No other potential sources of bias due to sample distribution differences
AC

between students in the curriculum and TAU groups were found. The power analysis demonstrated an

adequate sample size.

Primary Analyses

A one-way analysis of variance (ANOVA) with random effects was run for each of the dependent

variables to calculate their respective intra-class correlation coefficients (ICC). These null models
 ACCEPTED MANUSCRIPT
8

demonstrated that 45.3% of the total variance in knowledge, and 56% of the total variance in stigma, was
31
accounted for by between-participant differences, providing strong support for the use of MLM.

MLM analyses of Model 1 revealed a significant time by condition effect on knowledge (F (1,

521.74)=20.09, p<.001). Post hoc analyses showed a significant increase in knowledge from pre to post

PT
(F(1, 495.33)=25.78, p<.001, ß=.67, 95%CI=.41-.93) for the curriculum condition and a nonsignificant

change in knowledge over time (F(1, 484.28)=.55, p=.459, ß=-.14, 95%CI=-.51-.23) for the TAU

RI
condition. Students who received the curriculum showed an increase in mental health knowledge scores

from pre (M=8.12, SD=2.18) to post (M=8.82, SD=2.41) whereas students receiving TAU had a decrease

SC
in mental health knowledge from pre (M=8.69, SD=2.14) to post (M=8.51, SD=2.45) (Figure 2).

MLM analyses of the second continuous outcome stigma (Model 2), also demonstrated a

U
significant time by condition effect (F1, 479.96)=8.86, p<.01). Post hoc analyses showed significant

increases in positive attitudes from pre to post (F(1, 488.95)=11.33, p<.01, ß=.51, 95%CI=.21-.81) for
AN
students in the curriculum group whereas students receiving TAU did not have a significant change in

attitudes over time (F(1, 475.47)=1.58, p=.209, ß=-.27, 95%CI=-.69-.15). Students who received The
M

Curriculum Guide showed an increase in positive attitudes towards mental illness from pre (M=20.39,

SD=2.89) to post (M=20.93, SD=3). Students in the TAU group showed a decrease in positive attitudes
D

from pre (M=20.98, SD=2.61) to post (M=20.70, SD=2.96; Figure 2).

TE

Model 3 examined the association between knowledge and stigma. Results showed a statistically

significant relationship between knowledge and attitudes towards mental illness (F(1, 987.71)=84.26,
EP

p<.001, ß=0.34, 95%CI=.27-.41). This shows that for each unit increase in mental health knowledge

score, students had a corresponding increase in positive attitudes towards mental illness of 0.34.
C

Secondary analyses
AC

Secondary analyses were carried out as a significantly uneven distribution of students in

university and community college streams was observed (see Table 1). In this model, school stream was

included as a fixed factor with two levels representing students enrolled in the university stream (n=331)

and those enrolled in the community college stream (n=203).

This subgroup analysis consisted of a model testing the 3-way interaction of time by condition by

school stream on the outcome variable knowledge, which was significant (F(1, 513.91)=35.59, p<.001).
 ACCEPTED MANUSCRIPT
9

The same interaction was then tested with stigma as the outcome variable. The time by condition by

school stream interaction significantly predicted stigma (F(1, 503.64)=18.48, p<.001). Post-hoc pairwise

comparisons demonstrated a significant positive impact of the curriculum on mental health knowledge

(F(1, 489.62)=30.49, p<.001, ß=.95, 95%CI=.61-1.29) and stigma reduction (F(1, 482.38)=15.75, ß=.79,

PT
p=001, 95%CI =.40-1.18) for students in the university stream. All other pairwise comparisons were

nonsignificant (Figure 3).

RI
DISCUSSION

This study is the first RCT examining the effectiveness of a mental health literacy curriculum,

SC
developed for high school students, and delivered in the classroom. Students receiving The Curriculum

Guide showed significant improvements in mental health knowledge and a reduction in stigma compared

U
to those receiving TAU, with the magnitude of these findings being substantial. Additionally, we found that

improvement in mental health knowledge predicted a corresponding improvement in attitudes towards

AN
mental illness/reduction in stigma. The association between stigma and mental health knowledge is not

well understood in the literature. Research to date has not always supported a relationship between these
M

constructs.32,33, 34 Intuitively, it is reasonable to expect that improving mental health knowledge would help

reduce stigma surrounding mental illness. Our findings contribute to the emerging literature that
D

35,36
demonstrates the influential relationship between mental health knowledge and reducing stigma. The
TE

impact of knowledge on stigma demonstrated in the current study provides a promising solution to

addressing attitudes toward mental illness through education.

EP

Subgroup analyses revealed more robust improvements in knowledge and stigma for students in

the university stream. The trend towards improvement for those in the community college stream was not
C

statistically significant. This finding may be due to the unexpected and disproportionally low number of
AC

participants in the TAU group for community college stream students, magnified by the 2:1 randomization.

Further adaptation of The Curriculum Guide may be beneficial for students in the community college

stream.

The implementation of The Curriculum Guide was found to be readily adaptive, with classroom

teachers successfully integrating it as part of an existing high school course. The great majority of

teachers reported positive overall experiences delivering The Curriculum Guide. Our findings are
 ACCEPTED MANUSCRIPT
10

22
consistent with Franklin et al.’s review, supporting the effectiveness of classroom delivery of The

Curriculum Guide by teachers. Studies evaluating the impact of this approach with classroom teachers
28
have demonstrated substantial and significant improvement in the mental health literacy of teachers and
37,38
of students. Therefore, the application of programs such as The Curriculum Guide can positively and

PT
sustainably impact mental health literacy for both teachers and students alike.

In a non-randomized cluster-controlled trial of high school students receiving a mental health

RI
39
literacy program, Skre et al. found that younger students (ages 13-15) held greater prejudiced beliefs

and showed poorer retention of knowledge of mental health services. Age and maturity level may need to

SC
40
be considered, as these factors could impact the efficacy of programs like The Curriculum Guide. Our

study of adolescents did not include this younger age range and therefore cannot speak to the efficacy of

U
mental health literacy programs for younger students. However, delivery of The Curriculum Guide may

also be beneficial for youth entering community college and university, as recent research demonstrates
AN
40
a need for such programs.

This study presents with certain limitations that should be considered. Students were not
M

individually randomized or stratified, but rather randomization took place at the school level, which may

have introduced a source of bias. However, randomization of schools precluded selection of better
D

schools to receive the educational resource. The students were unaware of the pending study and were
TE

blinded to the arm in which they were enrolled. These study conditions reduce potential selection bias,

further strengthening the findings.

EP

It is recognized that the mental health knowledge and stigma measures used were not

standardized; to our knowledge there were no existing standardized tools for measurement of our study
C

41
objectives in youth. A recent scoping review examined current measures of mental health literacy,
AC

including measures of knowledge and stigma. The authors identified significant gaps in the psychometric

properties of these scales, especially for youth. No scales were found to concisely measure general
41
knowledge and stigma associated with mental illness in adolescents.

The stigma scale used in the present study demonstrated satisfactory levels of internal

consistency, whereas the knowledge scale was low to moderate. Internal consistency indices such as

Cronbach’s alpha test the homogeneity of a scale, indicating the items in the scale should be correlated
 ACCEPTED MANUSCRIPT
11

with each other. Our knowledge questionnaire intentionally includes items of a multidimensional structure,

testing different aspects of mental illness. Since we tested a heterogeneous group of items, we did not
42
expect high internal consistency on this measure. That said, future research may benefit from the use of

knowledge and stigma scales that have undergone rigorous psychometric development.

PT
In addition to the reported outcomes of improved knowledge and reduced stigma, one major goal

of The Curriculum Guide is to promote students’ help-seeking behaviours. Because the time frame for the

RI
pre-post measurements was relatively short (approximately 4-8 weeks), changes in help-seeking

behaviour were not the focus of this preliminary evaluation of The Curriculum Guide. It should be noted

SC
that this important outcome of behavioral change is the main focus of future research using longitudinal

follow-up data.

U
It has been shown that initiatives to increase knowledge and positive attitudes about mental
43
illness may improve individuals’ likelihood of disclosing and seeking help for mental illness, supporting
AN
the clinical relevance of this finding. Of particular importance is our finding that increasing mental health

knowledge can lead to improvements in attitudes towards mental illness, providing empirical evidence for
M

classroom-based mental health education, delivered by teachers, as a means to reduce stigma.

44
The many strengths of this large RCT support the effectiveness and applicability of The
D

Curriculum Guide for high school students. Notably, this representative sample of students spanned a
TE

diverse geographical region of Canada including urban, suburban, and rural communities, suggesting the

potential for broad application in high schools. Replication and further studies are warranted, especially
EP

those including socioeconomic status and cultural factors, which may further elucidate the findings

presented here.
C

In conclusion, our study is the first large RCT to demonstrate the effectiveness of a mental health
AC

educational resource on mental health literacy for high school students. The findings merit strong

consideration for the delivery, by high school teachers in the classroom, of a standardized mental health

curriculum.

Clinical Guidance
 ACCEPTED MANUSCRIPT
12

• Our randomized controlled trial of mental health literacy provides evidence for its role in improving

knowledge of mental illness, reducing stigma, and improving attitudes toward mental health

problems in high school students

• The positive relationship between improved knowledge of mental illness and the reduction in

PT
stigma is important to establish, as these enhanced elements of mental health literacy may

improve help-seeking efficacy and behaviours

RI
• The Curriculum Guide is an effective tool in improving mental health literacy in high school

students

SC
• The Curriculum Guide is readily adapted as delivered by teachers in the classroom and easily

integrated into an existing high school course

U
References
AN
1. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and

age-of-onset distributions of DSM-IV disorders in the national comorbidity survey replication. Arch

Gen Psychiatry. 2005;62(6):593-602.

M

2. Merikangas KR, He JP, Burstein M, et al. Lifetime prevalence of mental disorders in US

D

adolescents: results from the National Comorbidity Survey Replication–Adolescent Supplement

(NCS-A). J Am Acad Child Psy, 2010;49:980-989.

TE

3. Breslau J, Miller E, Joanie Chung WJ, Schweitzer JB. Childhood and adolescent onset

psychiatric disorders, substance use, and failure to graduate high school on time. J Psychiat Res.
EP

2011;45(3):295-301.

4. Kessler RC, Foster CL, Saunders WB, Stang PE. Social consequences of psychiatric disorders I:
C

Educational attainment. Am J Psychiat. 1995;152:1026-1032.

AC

5. Shochet IM, Dadds MR, Ham D, Montague R. School connectedness is an underemphasized

parameter in adolescent mental health: Results of a community prediction study. J Clin Child

Adolesc. 2006;35(2):170-179.

6. Mathers C, Fat DM, Boerma JT. The global burden of disease: 2004 update. Geneva,

Switzerland: World Health Organization; 2008.

 ACCEPTED MANUSCRIPT
13

7. Kessler RC, Avenevoli S, Costello J, et al. Severity of 12-month DSM-IV disorders in the national

comorbidity survey replication adolescent supplement. Arch Gen Psychiatry. 2012;69:381-389.

8. Kutcher S. Facing the challenge of care for child and youth mental health in Canada: A critical

commentary, five suggestions for change and a call to action. Healthc Q. 2011;14:15-21.

PT
9. Rutter M, Bishop DVM, Pine DS, et al, eds. Rutter's child and adolescent psychiatry (5th ed.).

Oxford, UK: Blackwell Publishing Ltd; 2010.

RI
10. Kelly CM, Jorm AF, Wright A. Improving mental health literacy as a strategy to facilitate early

intervention for mental disorders. Med J Australia, 2007;187:s26-s30.

SC
11. Burns BJ, Costello EJ, Angold A, et al. Children’s mental health service use across service

sectors. Health Affair. 1995;14(3):147-159.

U
12. Waddell C, Offord DR, Shepherd CA, Hua JM, McEwan K. Child psychiatric epidemiology and

canadian public policy-making: The state of the science and the art of the possible. Can J
AN
Psychiat. 2002;47(9):825-832.

13. Corrigan P. How stigma interferes with mental health care. Am Psychol, 2004;59:614-25.
M

14. Corrigan P, Dremming Lurie B, Goldman HH, Slopen N, Medasani K, Phelan S. How adolescents

perceive the stigma of mental illness and alcohol abuse. Psychiatr Serv. 2005;56(5):544-550.
D

15. Clement S, Schauman O, Graham T, et al. What is the impact of mental health-related stigma on
TE

help-seeking? A systematic review of quantitative and qualitative studies. Psychol Med.

2015;45:11-27.
EP

16. Jorm AF. Mental health literacy: Public knowledge and beliefs about mental disorders. Brit J

Psychiat. 2000;177:396-401.
C

17. Sharp W, Hargrove DS, Johnson L, Deal WP. Mental health education: An evaluation of a
AC

classroom based strategy to modify help seeking for mental health problems. J Coll Student Dev.

2006;47:419-438.

18. Wei Y, Kutcher S, Szumilas M. Comprehensive school mental health: An integrated "school-

based pathway to care" model for Canadian secondary schools. McGill Journal of Education.

2011;46(2):213-229.
 ACCEPTED MANUSCRIPT
14

19. Rones M, Hoagwood K. School-based mental health services: A research review. Clin Child Fam

Psych. 2000;3(4):223-241.

20. Ringeisen H, Henderson K, Hoagwood K. Context matters: Schools and the “research to practice

gap” in children’s mental health. School Psychol Rev. 2003;32:153-168.

PT
21. Mcluckie A, Kutcher S, Wei Y, Weaver C. Sustained improvements in students’ mental health

literacy with use of a mental health curriculum in Canadian schools. BMC

RI
Psychiatry. 2014;14:379.

22. Franklin CGS, Kim JS, Ryan TN, Kelly MS, Montgomery KL. Teacher involvement in school

SC
mental health interventions: A systematic review. Child Youth Serv Rev. 2012;34:973-982.

23. Kutcher S, Wei Y. School Mental Health Literacy. Education Canada. 2014;54:22-25.

U
24. Schachter HM, Girardi A, Ly M, et al. Effects of school-based interventions on mental health

stigmatization: a systematic review. Child Adolesc Psychiatry Ment Health. 2008;2:18.

AN
25. Wei Y, Hayden JA, Kutcher S, Zygmunt A, McGrath P. The effectiveness of school mental health

literacy programs to address knowledge, attitudes and help seeking among youth. Early Interv
M

Psychiatry. 2013;7(2):109-121.

26. Browne G, Gafni A, Roberts J, Byrne C, Majumdar B. Effective/efficient mental health programs
D

for school-age children: a synthesis of reviews. Soc Sci Med. 2004;58:1367-84.

TE

27. Weare K, Nind M. Mental health promotion and problem prevention in schools: What does the

evidence say? Health Promot Int. 2011;26(1):i29-69.

EP

28. Kutcher S, Wei Y, McLuckie A, Bullock L. Educator mental health literacy: A program evaluation

of the teacher training education on the mental health and high school curriculum guide.
C

Advances in School Mental Health Promotion. 2013;6(2):83-93.

AC

29. Willinsky C. A New Curriculum Resource from the Canadian Mental Health Association. Health

and Learning Magazine.2008;5:15-17.

30. Kutcher S, Wei Y. Challenges and solutions in the implementation of the school-based pathway

to care model: the lessons from Nova Scotia and beyond. Canadian Journal of School

Psychology. 2013;28(1):90-102.
 ACCEPTED MANUSCRIPT
15

31. Singer JD, Willett JB. Applied longitudinal data analysis: Modeling change and event occurrence.

Oxford, UK: Oxford University Press; 2003.

32. Li J, Li J, Huang Y, Thornicroft, G. Mental health training program for community mental health

staff in Guangzhou, China: effects on knowledge of mental illness and stigma. Int J Mental Health

PT
Sys. 2014;8:49

33. Schomerus G, Schwahn C, Holzinger A, et al. Evolution of public attitudes about mental

RI
illness: A systematic review and meta-analysis. Acta Psychiatr Scand. 202;125:440-52.

34. Pescosolido BA, Jensen PS, Martin JK, Perry BL, Olafsdottir S, Fettes D. Public knowledge and

SC
assessment of child mental health problems: Findings from the National Stigma Study-

children. J Am Acad Child Adoesc Psychiatry. 2008;47(3):339-349.

U
35. Busby Grant J, Bruce CP, Batterham PJ. Predictors of personal, perceived and self-stigma

towards anxiety and depression. Epidemiol Psychiatr Sci. 2015;Mar 20:1-8.

AN
36. Rodgers RF, Paxton SJ, McLean SA, et al. Stigmatizing Attitudes and Beliefs Toward Bulimia

Nervosa: The Importance of Knowledge and Eating Disorder Symptoms. J Nerv Ment Dis.
M

2015; 203(4), 259-263.

37. Mcluckie A, Kutcher S, Wei Y, Weaver C. Sustained improvements in students’ mental health
D

literacy with use of a mental health curriculum in Canadian schools. BMC

TE

Psychiatry. 2014;14:379-384.

38. Kutcher S, Wei Y, Morgan C. Successful application of a Canadian mental health curriculum
EP

resource by usual classroom teachers in significantly and sustainably improving student mental

health literacy. Can J Psychiatry. 2015;60(12):580-586.

C

39. Skre I, Friborg O, Breivik C, Johnsen LI, Arnesen Y, Wang CEA. A school intervention for mental
AC

health literacy in adolescents: effects of a non-randomized cluster controlled trial. Public Health.

2013;13:873-888.

40. Gruttadaro D, Crudo D. College students speak: A survey report on mental health. Arlington, VA:

National Alliance on Mental Illness; 2012.

41. Wei Y, McGrath PJ, Hayden J, Kutcher S. Mental health literacy measures evaluating knowledge,

attitudes and help-seeking: a scoping review. BMC Psychiatry. 2015;15:291.

 ACCEPTED MANUSCRIPT
16

42. Norman GR, Streiner DL. Biostatistics: the bare essentials. Hamilton: BC Decker; 2008: 317.

43. Rusch N, Evans-Lacko S, Henderson C, Flach C, Thornicroft G. Public knowledge and attitudes

as predictors of help seeking and disclosure in mental illness. Psychiatr Serv. 2011; 62(6): 675-8.

44. Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: updated

PT
guidelines for reporting parallel group randomised trials. J Clin Epidemiol. 2010;63(8), e1-e37.

Figure Legend

RI
Figure 1: Consolidated Standards of Reporting Trials (CONSORT) participant flow diagram. Note:

SC
TAU = teaching as usual.

U
Figure 2: Pre- and post-test mean knowledge and stigma (positive attitudes) scores by condition.
AN
Note: TAU = teaching as usual.
M

Figure 3: Pre- and post-test mean knowledge and stigma (positive attitudes) scores by school

program and condition. Note: TAU = teaching as usual.

D
TE
C EP
AC
 ACCEPTED MANUSCRIPT

17

Table 1: Sample Characteristics and Chi Square Comparisons by Group

Variables Overall Curriculum Control (tau)

2

PT
X DF P
% N Actual N (%) Expected N (%) Actual N (%) Expected N (%)
Gender

RI
Male
44.9 240 179 (49.4) 163 (45.0) 61 (35.5) 77 (44.8)

SC
9.2 1 .002**
Female
55.1 294 183 (50.6) 199 (55.0) 111 (64.5) 95 (55.2)

U
High School Stream

AN
University
62 331 204 (56.4) 224 (61.9) 127 (73.8) 107 (62.2) 15.1 1 .001***

M
College
38 203 158 (43.6) 138 (38.1) 45 (26.2) 65 (37.8)

D
Language Spoken at Home

English
90 441
TE
291 (90.7) 288.9 (90.0) 150 (88.8) 152.1 (90.0)
EP
.44 1 .53
Other
10 49 30 (9.3) 32.1 (10.0) 19 (11.2) 16.9 (10.0)
C

History of Mental Health Treatment

AC

Received 11.4 61 44 (12.2) 41 (11.3) 17 (9.9) 20 (11.6)

Did not receive 88.6 429 318 (87.8) 321 (88.7) 155 (90.1) 152 (88.4) .59 1 .44
Family received 105 19.7 76 (35.5) 71 (19.9) 29 (16.9) 34 (19.8)
Family did not receive 429 80.3 138 (64.5) 286 (80.1) 143 (83.1) 138 (80.2) 1.26 1 .26
Note: ** denotes statistical significance at the 0.01 level; *** denotes statistical significance at the 0.001 level
 ACCEPTED MANUSCRIPT
18

Supplement 1

Multilevel Models

Models 1 and 2: Two-level random intercept models.

PT
Level 1: Yij = ß0j + ß1j(TIME)ij + rij

Level 2: ß0j = γ00 + γ01(COND)j + u0j

RI
ß1j = γ10

SC
Model 3: Two-level random intercept model.

Level 1: Yij = ß0j + ß1j(MHK)ij + rij

U
Level 2: ß0j = γ00 + u0j
AN
ß1j = γ10

Models 4 and 5: Two-level random intercept models.

M

Level 1: Yij = ß0j + ß1jTIMEij + rij

D

Level 2: ß0j = γ00 + γ01CONDj + γ02STREAMj + u0j

ß1j = γ10
TE

Note. Models 1 and 2 include fixed effects of time and condition, and a crosslevel interaction between time
(level 1) and condition (level 2). The dependent variable for Model 1 is knowledge. The dependent variable for
EP

Model 2 is stigma. Model 3 uses the grand mean-centered variable knowledge as a predictor of the dependent
variable stigma. Models 4 and 5 include fixed effects of time, condition, and school stream, and a 3-way
interaction of time by condition by school stream on the outcome variables knowledge and stigma respectively,
with school stream added at level 2. COND = condition; MHK = mental health knowledge; STREAM = school
C

stream (university vs. college).

AC
 ACCEPTED MANUSCRIPT

PT
RI
U SC
AN
M
D
TE
EP
C
AC
 ACCEPTED MANUSCRIPT

PT
RI
U SC
AN
M
D
TE
EP
C
AC
 ACCEPTED MANUSCRIPT

PT
RI
U SC
AN
M
D
TE
EP
C
AC