Student and Teacher Perceptions of Student Engagement

By

Stephanie D. Sutherland

A thesis submitted in conformity with the requirements

for the degree of doctor of Philosophy

Department of Theory and Policy Studies in Education

Ontario Institute for Studies in Education

University of Toronto

©Copyright by Stephanie Dawn Sutherland (2010)

 Student and Teacher Perceptions of Student Engagement

Doctor of Philosophy
Stephanie Dawn Sutherland
Department of Theory and Policy Studies in Education
University of Toronto
2010

Abstract

This study will explore student engagement as a multidimensional construct through a

systematic comparative study from the views of students and teachers. While the construct of

engagement holds promise for addressing declining motivation and achievement of adolescent

students, the challenges associated with measuring a multi-faceted construct suggest the need for

integrative research methodologies. This study will utilize concept mapping methods in two

urban secondary schools. This methodology holds the potential to provide the tools for

structured „meaning making‟ between participants (students and teachers). This capacity to

„think together‟ is promoted through intentionally structured (i.e., concept mapping processes)

practice of discourse. As a direct result of this approach, data revealed the degree of

convergence and divergence in student and teacher definitions of student engagement. Areas of

student/teacher convergence included themes addressing „diversity/belonging‟, „student-teacher

interaction‟, and „variety in school policy/structure‟. Areas of divergence included, „aspects of

pedagogy‟, „students at the centre‟, and „professional educators‟.

ii
 In framing student engagement as a multidimensional construct, this study was able to

uncover complex nuances. For example, closer examination of the student data revealed a nested

and multi-faceted relationship to their sense of engagement. Students most strongly associated

engagement to their sense of belonging at school. In turn, this sense of belonging was directly

impacted by their relationships with peers, and this connection was viewed to directly affect on

motivation (and subsequent achievement).

Future research is needed so as to delve deeper into the nature of social connections

among teachers-students, and students-students as an approach to untangle and better understand

the multidimensionality of factors at play.

iii
 Acknowledgements

There are so many people to thank who encouraged and supported me through this

process. First, I want to thank my supervisor, Dr. Lorna Earl, for giving me the opportunity to

collect this data and to learn alongside her. Dr. Earl‟s commitment to student development in

scholarship is one of the reasons that this study could be possible.

I want to express my gratitude to Dr. Ben Levin who agreed to work with me as a

committee member. Dr. Levin was instrumental in helping to shape this work into a

manageable entity. His generous approach to helping others achieve their goals will always have

a special place in my heart. Thank you to Dr. Stephen Anderson who agreed to become a

committee member and help see this project to completion. Dr. Anderson‟s comments and

insights were critical in sharpening the research questions, and subsequently making this study a

contribution to the field. Lastly, thank you to Dr. Patricia Thompson for acting as my external

examiner.

Thank you to my mother, Mary Louise Commodore, who was always available for a

subtle yet gentle push when I needed it. I want to thank my husband, Bill McEvily, for his

support and encouragement through this long process, and to my daughters Ruby and Isabelle,

for their unconditional love and precious smiles.

Finally, I would like to express my gratitude to my late father, Paul Commodore, who did

live to see the completion of this work but who knew in his heart that I would finish. I may have

not said it enough but he was the most incredible inspiration anyone could have ever had. Thank

you for always seeing the best in others.

iv
 Table of Contents

Abstract / ii

Acknowledgements / iv

Chapter 1: Introduction / 1

Research Questions / 5

Chapter 2: Literature Review / 9

Student Engagement: An Antidote to Declining Academic Achievement / 10

Participation Withdrawal / 10

Definitional Ambiguity / 12

Multidimensionality of the Construct / 13

School Level Factors Influencing Engagement / 13

School Climate / 14

Student Voice / 17

Student-Teacher Relationships / 18

Peers / 20

Research Approaches to Student Engagement / 21

Challenges of Quantitative Approaches / 22

Prospects for Participatory Approaches / 22

Concept Mapping / 23

Sequential Stages in Concept Mapping / 24

Theoretical Foundations of Concept Mapping / 26

Chapter 3: Context / 32

School A / 35

School B / 36

Chapter 4: Methodology / 38

v
 Ethical Considerations / 38

Step 1: Sample / 41

Step 2: Generation of Student (Statement) List / 44

Generation of Teacher (Statement List) / 50

Step 3: Structuring of Statements / 58

Step 4: Representation of the Statements / 59

Step 5: Interpretation of the Maps / 65

Chapter 5: Results / 67

Concept Mapping Outputs / 67

Chapter 6: Discussion / 81

Research Questions / 82

Key Dimensions of Convergence / 84

Convergence Area 1: „Diversity/Belonging‟ / 85

Convergence Area 2: „Student-Teacher Interaction‟ / 88

Convergence Area 3: „Variety in School Policy/Structure‟ / 90

Key Dimensions of Divergence / 91

Divergence Area 1: „Aspects of Pedagogy‟ / 92

Divergence Area 2: „Students at the Centre‟ / 93

Divergence Area 3: „Professional Educators‟ / 95

Chapter 7: Conclusions / 106

Chapter 8: Implications for Practice and Research / 109

Limitations / 109

References / 117

vi
Tables

Table 1 Sequential Concept Systems Stages in Structured Conceptualization / 40

Table 2 Student Brainstorming Focus Group Characteristics – Schools A & B / 42
Table 3 Teacher Brainstorming Focus Group Characteristics – Schools A & B / 43
Table 4 School A – Student Brainstorming List / 45
Table 5 School B – Student Brainstorming List / 47
Table 6 School A – Teacher Brainstorming List / 51
Table 7 School B – Teacher Brainstorming List / 53
Table 8 Combined Student & Teacher Items / 57
Table 9 Student & Teacher Statements By Cluster Name / 68
Table 10 Mapping of Concept Mapping Items to Student Voice Literature / 101

Figures

Figure 1 Concept Mapping Stages / 25

Figure 2 Example of Cluster Rating Map / 63
Figure 3 Example of Pattern Match / 64
Figure 4 Point Map / 72
Figure 5 Conceptual Cluster Map / 74
Figure 6 Point Rating Map / 75
Figure 7 Cluster Rating Map / 76
Figure 8 Cluster Rating Map with Discourse Regions / 78
Figure 9 Teacher and Student Pattern Match / 80
Figure 10 Pattern Match as Discussion Generator (Points of Divergence) / 111
Figure 11 Creation of Specific Objectives for Ongoing Discussion / 112
Figure 12 Subdivide Objectives into Action-Oriented Tasks / 113

vii
Appendices

Appendix A Informed Consent Form / 135

Appendix B Interpretation Session Materials / 137

viii
 CHAPTER ONE: INTRODUCTION

Research in the past thirty years has consistently shown that the current model of

schooling no longer adequately meets the needs of young people or of contemporary Canadian

society. There is a growing concern about the number of students who are fading out or

dropping out of school (see for example: Bowlby & McMullen, 2002; National Research

Council, 2003; Willms, Friesen, & Milton, 2009). Although studies in early learning have

heightened efforts to improve learning environments for young children, thus far, less attention

has been focused on how to transform learning environments for adolescent learners. The

concept of student engagement is increasingly viewed as a possible antidote to declining student

motivation and achievement particularly among secondary school students (Fredricks,

Blumenfelf & Paris, 2004; National Research Council & Institute of Medicine, 2004). Part of

the reason that researchers are increasingly focusing on student engagement is that they deem

engagement to be malleable, responsive to contextual features and amenable to environmental

change (Fredricks et. al., 2004). By contrast, the dominant approach in Canada to engaging

students in learning has involved “streaming” or separating students based on their skill and/or

readiness.

Students, for example, who had low confidence in their skills and found school work to

be too challenging, were placed in classes that required lower skills and less challenge, while

students who required greater challenge were provided with enrichment projects or given more

assignments. Students who had given up on school because of low skills and/or low challenge

1
often found themselves in some type of remedial class or alternative school (OECD, 2003).

Structurally, Canadian secondary schools (beyond Grade 9) follow this type of thinking by

streaming students into separate tracks. Schwartz and Fischer (2006) suggest that this approach

may be too simplistic both in terms of pedagogy and generational differences in adolescent

learners. Many students today view schooling as boring or as a mere grade game, in which they

try to get by with as little effort as possible (Burkett, 2002; Pope, 2002: Twenge, 2009).

Numerous studies find steep declines in motivation across grade levels (see, for example: Eccles,

Midgley & Adler, 1984; Fredricks & Eccles, 2002; OECD, 2003). These findings are

particularly troubling in light of the fact that adolescents will find themselves in a new, global,

fast changing economy that requires knowledgeable workers who can synthesize and evaluate

new information, think critically and solve problems (Fredricks et. al., 2004). Although school

attendance is compulsory, a sense of commitment and motivation to schooling cannot be

mandated. Perhaps, this is why student engagement is often viewed as an antidote to student

lack of motivation and academic achievement.

Framing student engagement as a multidimensional construct is promising so as to better

understand the complex interactions between the student and the schooling environment. To

understand the „schooling environment‟ this study will include engagement in the classroom as

well as the larger school community. School reform researchers have well documented the

difficulties in making changes in classroom practice. That is, many initiatives often fail to reach

into the classroom to influence the dynamic between teachers and students (Cuban, 1989;

Elmore, 1996; Elmore, Peterson, & McCarthy, 1996; Sarason, 1971; Tyack & Cuban, 1995).

Fullan, Bennett, & Rolheiser-Bennett (1990) state that school improvement initiatives and

2
classroom improvement must be linked if substantial progress towards school-based reform is to

be achieved. Undeniably, the elements of the school and the classroom do affect one another,

and in „effective schools‟ they have been noted to work together (Fullan et. al., 1990). In this

vein, this study will approach the school environment as a whole rather than as a collection of

independent elements. This study seeks to understand student engagement from the views of

students and teachers, whereby the classroom is thought to be the most influential place where

student and teacher relationships are shaped.

In crafting new paradigms for learning, we must include students as key stakeholders in

the process. Rudduck et al., (1996) explain that the conditions of learning that are common

across contemporary secondary schools do not adequately take account of the social maturity of

young people, nor of the tensions and pressures they feel as they struggle to reconcile the

demands of their social and personal lives with the development of their identity as learners. The

authors also note that what students have to say about teaching, learning, and schooling is not

only worth listening to but provides an important - perhaps the most important - foundation for

thinking about ways of improving schools. While it is imperative to ascertain the views of

students, student engagement is a two-way street. That is, teachers (and administration) need to

be involved in the creation and sustainability of engagement. The challenge for school

administrators and teachers is to understand the nuances of engagement in their own context, a

process that requires the active involvement of students and teachers. This study will explore

student engagement as a multidimensional construct through a systematic comparative study

from the views of students and teachers.

3
 Prior work on engagement has subdivided the construct into three broad (and often

overlapping) domains of behavioural, emotional and cognitive engagement. While is important

to categorize the research into these three areas, it can also be problematic in terms of attempting

to bring conceptual clarity to this multifaceted construct. One of the challenges posed by the

multidimensional conceptualization is that the three domains of engagement often overlap with

constructs that have been studies previously. For example, research on behavioural engagement

is related to that on student conduct and behavior (Karweit, 1989; Peterson, Swing, Stark, &

Wass, 1984). Research on emotional engagement is related to work on student attitudes (Epstein

& McPartland, 1976; Yamamoto, Thomas, & Karns, 1969) and student interest and values

(Eccles at al., 1983). Research on cognitive engagement is related to research on motivational

goals (Boekarts, Pintrich, & Zeinder, 2000; Zimmerman, 1990). Although encompassing

portions of literature under the umbrella of „engagement‟ can be problematic in terms of overlap

with related constructs, uniting all three domains has the advantage of treating engagement as a

“meta” construct (Fredricks et. al., 2004; Guthrie & Anderson, 1999; Guthrie & Wigfield, 2000).

The fusion of behavior, emotion and cognition under a single umbrella has the potential to

provide a richer characterization of students than is possible in research focused in a single

domain.

Another closely related challenge associated with the multidimensional nature of

engagement concerns measurement and the variety of approaches adopted. Whereas some

researchers employ conceptually discrete scales for each domain of engagement (for example:

Miller et. al., 1996; Nystrand & Gamoran, 1991; Patrick, Skinner & Connell, 1993; Skinner &

Belmont, 1993); others combine the different domains into a single general engagement scale

(for example: Connel, Halpern-Felsher, Clifford, Crichlow & Usinger, 1995; Marks, 2000; Lee

4
& Smith, 1995). The practice of combining items into general scales precludes examining

distinctions and/or nuances among the types of engagement. As well, purely quantitative

measures, in essence, have a pre-set agenda for inquiry. That is, student engagement scales often

ask participants about specific aspects of engagement, for example, cognitive aspects. More

research needs to tap the qualitative differences in engagement so as to better distinguish the

degree of behavioural, emotional or cognitive dimensions as well as to better understand how the

various types of engagement develop and interact (Willms et al., 2009). While the concept of

engagement holds promise for addressing declining motivation and achievement of adolescent

students, the challenges associated with measuring a multi-faceted construct suggest the need for

integrative research methodologies. This will be facilitated by utilizing such a methodology -

concept mapping - in two urban high schools. This methodology holds the potential to provide

the tools for structured „meaning making‟ between participants (in the present example, students

and teachers). This capacity to „think together‟ is promoted through intentionally structured (i.e.,

the concept mapping process) practice of discourse. Thus, as students and teachers develop

shared understandings it is premised that shared commitment to results would be fostered. The

set of research questions guiding this study are as follows:

Research Questions

1. How do teachers‟ and students‟ define student engagement?

2. What are the key dimensions of convergence and divergence between students‟ and

teachers‟ definitions of student engagement?

3. How do student and teacher definitions of student engagement compare to existing

5
 theory and research on the nature of student engagement and the school level factors

influencing student engagement?

The research questions guiding this study were addressed within the context of a larger

ongoing evaluation focusing on school improvement and student engagement – the Manitoba

School Improvement Program (MSIP). MSIP provided an ideal context in which to situate this

work. MSIP is a non-governmental agency operating in the province of Manitoba since the early

1990s, and came into being as a result of the vision of the Walter and Duncan Gordon

Foundation (WDGF). The Foundation sought to support school-based improvement projects

designed to help students at risk remain in school and fulfill their individual potential. The

central goal of MSIP has been to improve the learning experiences and outcomes of secondary

school students by building schools‟ capacities to enhance student engagement and learning.

MSIP encouraged schools to develop goals with a particular focus on the needs of adolescent

secondary school students. As a precondition to joining the MSIP network of schools (schools

receive funding and network support), they must submit project proposals based on the following

criteria: be school-based, focus on the needs of adolescent secondary students, and produce

annual evaluation reports (Walter and Duncan Gordon Charitable Foundation, 1993).

In describing their school goals, many schools focused on the relationship between

student engagement and learning. The two schools included in the present study have been

involved in MSIP since its inception and both schools had increasing levels of student

engagement as a primary focus as detailed in their school improvement plans. That being said,

however, after some preliminary conversations with administrators and some teachers at each

6
school, it became apparent that they could not begin the discussion (and subsequent

measurement) of increasing levels of student engagement without first defining what the

construct meant to them. The principal at one of the sample schools was honest about this

dilemma and described it this way:

The big problem we ran into here was our goal was to increase student
engagement, and for three years we said, “well how can we measure
student engagement…it‟s different for every student, and you know, how
can that be our goal?” So, this year we had to say, “Okay, what is
student engagement, and we need to ask our students and teachers to
define it and what does it look like and how are we going to measure it?
In the past, we set a goal that we couldn‟t evaluate so we got off the hook,
but this year instead of getting off the hook we changed our focus and we
said, “Okay, we have to be able to measure this and what is it going to
look like for us?”

This problem is not unique to the MSIP sample schools, rather many initiatives aimed at

improving school climate and achievement; explicitly or implicitly focus on engagement as a

route to improvement without understanding the complexities and nuances of engagement

(Guthrie & Wigfield, 2000). In asking students and teachers to define what student engagement

means to them we can better understand the complexity of student experiences in school and

design more specifically targeted, locally appropriate, and nuanced interventions.

The remainder of this thesis is organized into four chapters. The second chapter reviews

the relevant student engagement literature related to the central research questions in this study.

7
The third chapter provides detail on the methodology and analysis techniques employed;

specifically Trochim‟s (1989) integrative concept mapping procedure. I will first describe the

general technique and then explain how I adapted it for this particular study. The fourth chapter

presents the results from this study consisting of graphic outputs from the concept mapping tool

described in the preceding chapter. The final chapter provides a discussion of the main findings

and how they related to the research questions, and will conclude with some final observations

and possible directions for future research.

8
 CHAPTER TWO: LITERATURE REVIEW

This literature review serves to highlight relevant research pertinent in addressing the

study‟s central research question - how teachers and students define student engagement. The

first section makes a case for the importance of studying student engagement. In light of

declining adolescent motivation and achievement in secondary schools, student engagement is

presumed to be responsive to contextual features and holds promise for change. The next section

of the review examines the literature with respect to attempts at defining this complex construct.

Broadly, student engagement has been described in three ways: behavioural, emotional, and

cognitive engagement. The third section of the review explores the factors that influence student

engagement from an educational environment perspective. That is, the school and classroom

factors that mediate engagement will be explored. Finally, the review will account for current

research approaches employed in studying student engagement. This will include limitations to

the use of quantitative approaches and will make the case for the need to employ more

qualitative, participatory methods as methodological tools to assist in addressing the definitional

problems association with student engagement. In this case, concept mapping will be presented

as a promising participatory approach for students and teachers to co-constructing a contextually

relevant definition of student engagement.

9
Student Engagement: An Antidote to Declining Academic Achievement?

Evidence is mounting (see, for example: Bowlby & McMullen, 2002; National Research

Council Institute of Medicine, 2004) to show many problems experienced by students in middle

and secondary schools – such as disengagement, dissatisfaction with their schooling experience,

and dropping out – are significantly linked to the learning environment (Pope, 2001). For

example, preliminary research (Willms, et. al., 2009) finds that levels of school participation and

academic engagement fall steadily from Grade 6 to Grade 12, while cognitive/ intellectual

engagement falls during the middle school years and remains at a low level throughout

secondary school. Current theory and research offer various explanations for such findings.

Chaplain (1996) finds that students make a strategic withdrawal in order to protect their self-

worth. Specifically, the author explains that the motivation to protect one's sense of self-worth

results in pupils using a range of tactics to avoid damage to their self-esteem. While such tactics

are effective in the short term, the enduring consequence is further withdrawal, and ultimately,

under-achievement (Thompson, 2002).

Participation-Withdrawal

Covington (1992) highlights the relationships between ability, the quality highly regarded

in education, and feelings of self-esteem and personal worth. Covington (1992, p.16) notes that

“it is not surprising that the student's sense of self-esteem often becomes equated with ability, for

to be able is to be valued as a human being but to do poorly is evidence of inability, and reason

to despair one's worth.” Conversely, many successful students, however, are not so much

enthusiastic as bored, indifferent, unconcerned or instrumentally focused on getting their

10
credential. And there are other students who, while physically attending school for at least some

of the time, are disengaged or marginalized by their school experience. As Hargreaves, Ryan &

Earl (1996, p.80) state, "perhaps secondary schools fail to retain students because they never

really engage them in the first place".

Many researchers have postulated that engagement be thought of as a continuum (Finn,

1989, 1993; Finn and Rock, 1997; Goodenow, 1993; Voelkl, 1995, 1996, 1997; Wehlage et al.,

1989). Newman (1990) states that when efforts to act competently are met with successes,

continued investment is generated and the cycle continues. Prior psychological research has

confirmed that the desire for competence (both emotional and practical) has been recognized as

one of the most powerful bases for human interaction and motivation. Hargreaves et al. (1996)

note that the absence of challenge is a clear example of how secondary schools often fail to

engage student's interests and involvement. Finn (1989, 1993) has criticized such research for

focusing on the amelioration of student deficiencies rather than the development of strategies

that will improve student engagement. He proposes the participation-identification model, which

specifies that student identification with school, a psychological condition, depends on

participating in it. Identification is defined by two primary dimensions: an internalized sense of

belonging and value for success in school-relevant goals.

Undeniably, the traditional approach to schooling and, in particular, engaging adolescents

in schools and classrooms is inadequate. The concept of student engagement has attracted

growing interest as a way to ameliorate low levels of academic achievement, student boredom,

11
and increasing dropout rates (National Research Council & Institute of Medicine, 2004). So,

what, then, do we mean by student engagement and what are the attributes assigned to it?

Definitional Ambiguity

Like many other social constructs, student engagement has no universally accepted

meaning (Smith et al., 1998). Given the complexities of the construct, the explanatory power of

student engagement is weak. Though the construct remains conceptually fuzzy, it has received

increased attention in the academic research literature, namely within the fields of psychology,

social psychology and sociology of education. In this literature, three general approaches to

student engagement can be identified: behavioural engagement pertains to participation in school

activities (see, for example: Finn, 1989, 1993; Finn & Rock, 1997; Goodenow, 1993; Goodenow

and Grady, 1993; Voelkl, 1995, 1996, 1997; Wehlage, Rutter, Smith, Lesko & Fernandez, 1989),

psychological/emotional engagement component pertains to students‟ sense of belonging at

school and acceptance of school values (see, for example: Brady, 2005; Osterman, 2000;

Solomon, Watson, Battistich, Schaps & Delucchi, 1996), and cognitive/intellectual engagement

pertaining to students‟ investment in learning (see, for example: Blount, Morse, Anderson,

Christenson & Lehr, 2004; Willms, Friesen & Milton, 2009).

12
Multidimensionality of the Construct

Many studies of engagement include one or two of these types, but rarely all three.

Fredricks, Blumenfeld & Paris (2004) suggest that to date, research has not capitalized on the

potential of engagement as a multidimensional construct that involves behaviour, emotion, and

cognition. According to Fredricks et. al., 2004, engagement has considerable potential as a

multidimensional construct that unites the three components (behavioural, emotional, &

cognitive) in a meaningful way. In this way, engagement can perhaps best be thought of as a

„meta‟ construct. Rather, than focusing on one or two of the categories, fusing them together as

a multidimensional construct, one that focuses on students‟ and their interaction with the

educational environment holds promise in helping to better understand the complexity of

students‟ experiences in school. In this way, this study will include research on engagement in

the school and the classroom.

SCHOOL-LEVEL FACTORS INFLUENCING STUDENT ENGAGEMENT

Although learning involves cognitive processes that take place within each individual,

motivation to learn also depends on the student‟s involvement in a web of social relationships

that supports learning (Cohen & Ball, 1999). The likelihood that students will be motivated and

engaged is increased to the extent that their schools, teachers, family, and friends effectively

support their purposeful involvement in school (National Research Council Institute of Medicine,

2004). Although there are numerous studies focusing on students‟ family/ethnicity

characteristics (see, for example, Coleman, 1966; Davidson, 1996; Epstein, 2001; Goslin, 2003;

13
Rumberger & Palardy, 2002), and social demographic factors (see, for example, Bascia &

Hargreaves, 2000; Bradley, Corwyn, McAdoo & Coll, 2001; Finn & Rock, 1997; Jordan &

Plank, 2000; Statistics Canada, 2003, 2004, Whelage et al., 1989), this review will focus on

engagement with the school environment (school and classroom). This will include a discussion

of engagement with the school as an organization, the rise in popularity of student voice, and

students‟ engagement with teachers and peers.

The research on classroom and broader school effects has identified a number of

classroom and school factors relating student engagement to students‟ academic achievement

including the climate of the school and expectations for academic success (see, for example,

Rutter, 1983; Scheerens, 1992), the benefits of smaller schools in terms of attendance and

retention (see, for example, Finn, 1989; Friedkin & Necochea, 1988; Gardner, Ritblatt, & Beatty,

2000; Lee, Bryk, & Smith, 1993; Leithwood & Jantzi, 2009), the quality of instruction and

teacher/student relationships (Battistich, Solomon, Watson & Schaps, 1997; Lee & Burkam,

2003).

School Climate

School climate refers to the values, norms, beliefs, and sentiments associated with routine

practices and social interaction in schools (National Research Council Institute of Medicine,

2004). Many studies (Bryk & Schneider, 2002; Marks, 2000; Lee & Smith, 2001) have found

trust to be essential to school improvement efforts. For example, in their study of Chicago

schools, Bryk and Schneider (2002) found that in schools with the highest achievement levels,

14
teachers reported strong „relational trust‟ with the principal, fellow teachers, and to a lesser

extent, parents. Building a climate of trust is particularly important in secondary schools

whereby adolescents need a supportive and caring environment. Thus, the climate of the school

is particularly important for cohorts of students who fall under the label of adolescents.

Prior psychological research has documented the distinctiveness of the stage in life called

adolescence. At this stage in life many youth are seeking greater autonomy and more

challenging learning material; however, schools are often structured to provide less of both.

Adolescents need support from non-parental adults such as teachers, they are typically met with

few opportunities to get to know and connect with these adults (Eccles & Midgley, 1989; Eccles,

Midgley, Wigfield, Buchanan, Reuman, Flanagan, & Mac Iver, 1993). Researchers and

educators are clear on the fact that schooling as is is not working for large numbers of young

people (Hill, Campbell & Harvey, 2000; Lippman, Burns, & McArthur, 1996). One place to

start is with the structure of the school itself. In addressing this issue, Bryk and Schneider (2002)

suggests that schools build a climate of trust and being to affirm the institutional legitimacy of

the school.

Generally, there is consensus amongst student engagement researchers that two central

variables influencing engagement in school are students‟ participation in school activities and

their sense of belonging (see, for example: CEA, 2006;Willms et al., 2009; Willms, 2000, 2003;

National Research Council Institute of Medicine, 2004; Osterman, 2000; Solomon, Watson,

Battistich, Schaps & Delucchi, 1996). Schools are most likely to cultivate a sense of belonging
15
and membership in students if they demonstrate clarity of purpose, equity and personal support,

provide frequent occasions for all students to experience educational successes, and integrate all

of these factors into a climate of caring.

Several studies show that supportive relationships with others are linked to students‟

internalization, self-regulation, and sense of community. Research is consistent in identifying

the psychological sense of belongingness as an important factor in participation, school

engagement, and dropout (Leithwood & Aitken, 1995). Finn‟s theory of school withdrawal

maintained that identification with the school was an important factor sustaining school

involvement and that participation in school activities contributed to identification (Finn, 1989).

More recent research (Leithwood, Jantzi & Haskell, 1997; Leithwood & Jantzi, 1999) supports

Finn‟s argument that higher levels of student identification with school lead to higher levels of

student participation. Leithwood and his colleagues‟ defined participation broadly in terms of

response to requirements; class related initiative, extra-curricular activities, and decision-making.

Their findings are important because they challenge an embedded assumption that students

develop a sense of community through their participation in extra-curricular activities or that a

strong extra-curricular program will satisfy student needs for a sense of community and

engagement. This finding suggests, however, that students‟ participation is also shaped by their

experience as being part of a supportive community. Efforts to transform the school community,

especially within the past decade, have included a focus on the notion of student voice.

16
Student Voice

Student voice refers to students themselves actively shaping their own learning, and more

generally about what happens in schools and classrooms (see, for example, Earl, Freeman,

Lasky, Sutherland & Torrance, 2002; Fielding, 2004; Levin, 2000; Levin & Pekrul, 2007; Mitra,

2003; Rudduck, 2002; Shultz & Cook-Sather, 2001; Willms et al., 2009). Prior research on

student voice has separated findings into one of four „clusters‟: the autonomy cluster (students

being able to make choices and decisions about their work), the pedagogy cluster (learning with

clear expectations and that is connected to daily lives), the social cluster (collaborative work and

being respected by teachers and peers), and the institutional cluster (understanding of school-

based procedures and policies) (McIntyre & Pedder, 2004; McIntyre, Pedder, & Rudduck, 2005;

Rudduck, 2007; Rudduck & Flutter, 2004; Shultz & Cook-Sather, 2007).

Specifically, in eliciting student perspectives about meaningful academic work, Willms

et. al., 2009 found students want the work they undertake to be relevant, meaningful and

authentic. This work builds on earlier research finding that students who feel their school work

is relevant, or more connected to their 'real world', find more identification with their educational

environment (Hargreaves et al., 1996; Newman, 1990). Rudduck et al., (1996, p.179) found the

best teaching strategies for engaging students were ones that made “clear links to the outside

world” and focused on “contemporary events of interest and meaning to students.” Newman

(1992) states that typically disengaged students behave well in school. They attend class and

complete work, but with little indication of excitement, commitment or pride in mastery of the

curriculum. In contrast, Newman (1990) has found that engaged students make a psychological

17
investment in learning. They try hard to learn what school offers. They take pride not simply in

earning the formal indicators of success (grades), but in understanding the material and

incorporating or internalizing it in their lives.

Undeniably, teachers remain the gatekeepers of school change. Steinhouse (1975, p.208)

said that only teachers could really change the world of the classroom and that they would do so

by understanding it. Teacher discussion and collaboration with students can help move towards

such understanding. That being said, however, providing all students with a change to negotiate,

plan and participate can be daunting, especially when added to an already overstretched teaching

staff (Jerome, 2001, p.9). Thus, building a school-wide framework for student voice and

student-teacher collaboration needs to become integrated in the climate of the school (Rudduck

et al., 1996). When student voice is successfully integrated into schools, the benefits,

particularly for teachers, can be numerous.

Student-Teacher Relationships

The Merriam Webster’s Collegiate Dictionary (11th ed.) lists “commitment” among the

most common usages of “engagement”. As defined in the New Oxford American Dictionary, to

engage is to “attract or involve”. Considering student engagement then the act of student

commitment and involvement in schooling is undoubtedly related to their relationships with

teachers. That being said, however, much of the prior research on student engagement has

traditionally dealt with students‟ ( see for example, Rudduck et al., 1996; Soo Hoo, 1993;

Willms, Friesen & Milton, 2009) or teachers‟ perspectives (see for example, Datnow &

18
Castellano, 2000; Fullan & Hargreaves, 1992) separately and has not addressed the similarities

and differences between the two in simultaneous manner. More recently, a growing number of

studies have constructed theoretical interpretations based on data-driven perceptions of students

and teachers (Cook-Sather, 2007; Earl, Freeman, Jardine, Clifford & Friesen, 2008; Lasky,

Sutherland & Torrance, 2002; Earl & Sutherland, 2003; Macbeath, Myers & Demetriou, 2001;

Rivière, Sotomayor, West-Burns, Kugler & McCready, 2008). This study seeks to build on this

growing body of research and contributing to a more multi-dimensional view of student

engagement by eliciting the views of students and teachers in a participatory method.

For students, teacher support and caring has been correlated with various aspects of

behavioural engagement, including higher participation in learning and on-task behaviour

(Battistich, Solomon, Watson & Schaps, 1997), lower disruptive behaviour (Ryan & Patrick,

2001), and a lower probability of dropping out of school (Croninger & Lee, 2001). Woods

(1996) noted the significance of the learning support provided by the social relationships in the

classroom. He argued that learning takes place most effectively when a mutually shared

understanding between teachers and students has been built up through 'negotiative discussion'.

For example, students can co-create assessment criteria with their teachers as a way to jointly

craft the learning experiences inside classrooms (Jardine, in press). These types of „knowledge-

building environments‟ ideas must be publicly available so that all members of the class can

build on ideas, improve them, challenge them, and justify them (Gilbert, 2005; Scardamalia &

Bereiter, 2001, 2003).

For teachers, student voice has been shown to provide teachers with a more open

perception of young people‟s capabilities, the capacity to see the familiar from a different angle,

19
and a readiness to change thinking and practice in light of these perceptions (Rudduck, 2007).

Researchers and teachers agree about how insightful young people are when asked about aspects

of teaching and learning but the ironic thing is that students themselves are often surprised that

anyone wants to hear what they think (Rudduck, 2007). Cook-Sather (2002), has said,

Decades of calls for educational reform have not succeeded in making

schools places where all young people want to and are able to learn. It is
time to invite students to join the conversation about how we might
accomplish that (p.9).

The next section of the review will highlight the issues associated with student-student

relationships and engagement in schooling.

Peers

Traditionally, researchers have focused less on the peer group than on teachers as a factor

in the socialization of engagement (Ryan, 2000). The bodies of literature on peer acceptance and

rejection have been used as theoretical justification for studying peer acceptance and

engagement. Peer acceptance in both childhood and adolescence is associated with satisfaction

in school, which is an aspect of emotional engagement, and socially appropriate behaviour and

academic effort, which are aspects of behavioural engagement (Fredricks, et al., 2004).

Studies of peer acceptance and friendship consistently show that high achievement is

correlated with peer acceptance and/or peer interaction (Jules, 1991; Ladd, 1990; Taylor, 1989;

Wentzel & Asher, 1995). Research on peer acceptance is important for a number of reasons.

First, the experience of belongingness is associated with important psychological processes. For

example, students perceive themselves to be more competent and autonomous and have higher

20
levels of intrinsic motivation. They have a stronger sense of identify but are also willing to

conform to and adopt established norms and values. On the other hand, feelings of

rejection/alienation are the flip side of the relatedness coin. Rejection or the sense of exclusion

and estrangement from the group is consistently associated with behavioural problems in the

classroom (either aggression or withdrawal), lower interest in school, and dropout (Bauermeister

& Leary, 1995; Connell & Wellborn, 1991; Goodenow, 1993).

Osterman (2000) argues that there is little research that provides us with a deep

understanding of the nature and quality of peer relationships within the school and classroom

context. What is needed are more qualitative studies that probe the nature of interpersonal

connections among students (Osterman, 2000).

Research Approaches to Student Engagement

Recall from the introductory chapter the description of the two sample schools in this

study who found themselves in a conundrum. On one hand, these schools put primacy on

„increasing levels of student engagement‟ as a goal in their school improvement plans, but on the

other hand, neither school was conceptually clear what engagement meant to them. Put another

way, these schools proceeded to put in place plans to increase student engagement implying that

they are able to somehow employ measurement practices without having a clear understanding

of the central components important in their respective buildings. It is clear from the literature

that numerous schools find themselves in a similar predicament as the sample schools (Wilms et

al. 2009). Indeed, within the field of educational research, a great deal of emphasis is often

placed on using external measurements of school and district to hold the system accountable for

21
student success, but these measurements do not always provide enough information to help local

decision makers focus their ideas, practices, resources, energy and leadership to improve

learning (Elmore, 2006).

Challenges of Quantitative Approaches

Many quantitative attempts to „measure‟ student engagement are often limited to the

manipulation of several dimensions of engagement. That is, some scholars include conceptually

distinct and discrete scales for each type of engagement (for example, Miller, Greene, Montalvo,

Ravindran, & Nichols, 1996; Nystrand & Gamoran, 1991; Skinner & Belmont, 1993). On the

flip side, other researchers combine many discrete scales to create a general engagement scale

(for example, Marks, 2000; Lee & Smith, 1995). The practice of combining the items into

general scales precludes examining distinctions among the types of engagement (Fredricks et al.,

2004). Finally, current measures often do not tap qualitative differences in the level of

engagement, making it difficult to distinguish between the degrees of behavioural, emotional, or

cognitive engagement (Fredricks et al., 2004).

Prospects for Participatory Approaches

This review has accounted for many of the key issues with regard to student engagement

in schooling, from why studying school engagement is important, to definitional ambiguities,

and critical aspects of schooling (climate, student voice, relationships with teachers and peers)

that shape our thinking about student engagement. Another key dimension that cannot be

22
overlooked is the research itself. That is, most research focused on student experience is focused

„on‟ not „with‟ students (Cook-Sather, 2007). More recently, Cook-Sather (2007) has called for

an increase in qualitative work, and for changes in the roles of both researcher and student. For

example, research „on‟ posits the researcher as distanced, authoritative and as the sole author of

meaning derived from qualitative research approaches such as observations and interviews. On

the other hand, research „with‟ calls upon both researchers and students to reconceptualize

themselves in new, more collaborative relationships (Cook-Sather, 2007).

To actively participate in accountable decision making, schools need access to fine-

grained data that can be collected, interpreted and acted upon in local settings. This study will

employ a participatory research method whereby students and teachers in two urban Canadian

high schools can themselves, construct a context-specific definition of what student engagement

means to them. This approach will be facilitated by concept mapping, an approach that creates

opportunities for “conversations across methods as ways of generating additional insights

regarding the phenomena under study as well as the methods that are being used” (Greene, 2008,

p.23).

Concept Mapping

The literature describes the use of concept mapping in two ways: that related to student

learning and curriculum development; and that related to program evaluation and planning.

Concept mapping is a graphic technique for promoting social interaction and exchange by

creating the conditions for the understanding of thoughts and how they might be linked with each

other (Khattri & Miles, 1994). In other words, concept mapping is a type of structured

23
conceptualization which can be used by groups to develop a conceptual framework which can be

used for program planning and development, as well as for evaluation purposes (Trochim, 1989).

Sequential Stages in Concept Mapping

Figure 1 depicts the stages, moving from left to right, in the concept mapping approach.

The process begins with an evaluation/research question. In the present study, the primary

research question seeks to have students (S) and teachers (T) produce a jointly authored

definition of student engagement (see again, Figure 1). To construct the map, “ideas first have to

be described or generated, and the relationships between them articulated” (Trochim, 1989, p.1).

This step is typically accomplished by holding focus groups with each stakeholder group and

engaging them in a brainstorming exercise. In this instance, students and teachers were asked to

generate short statements and/or phases about what student engagement meant to them. Once

the ideas have been generated they are subsequently sorted and rated, then entered into the

concept mapping software for multidimensional scaling and cluster analysis. Next, the system

produces a series of concept maps which are to be used with stakeholders in a process of joint

meaning making. Ideally, these interpretation sessions would produce group ownership and

commitment to ongoing strategic planning and improvement efforts.

24
 Figure 1 Concept Mapping Stages

QUALITATIVE QUALITATIVE

QUANTITATIVE

T S

Note: Sample (T) Teachers, (S) Students

The main difference between Trochim‟s concept mapping and other common mapping

processes is that the method described in this illustration is particularly appropriate for group use

and generates a group map which makes it attractive for use with different stakeholder groups.

According to Wandersee (1990, p.927), concept mapping “relates directly to such theoretical

principles as prior knowledge, subsumption, progressive differentiation, cognitive bridging, and

integrative reconciliation.” In other words, mapping is a theory of meaningful group learning.

This is echoed by Novak (1990) who contents that concept mapping is an effective method for

teaching and learning to become more fully understood by both teachers and students as the

entire process is premised on group understanding.

25
Theoretical Foundations of Concept Mapping Methodology

An understanding of the psychological, sociological and philosophical foundations from

which the concept mapping process is derived is important to the use of its application in this

study. These foundational understandings assist in clarifying how concept mapping evolved. It

is precisely these evolutionary tenants from which contributions to ongoing research and practice

can be elicited. Work in cognitive theory by Ausubel (1968) played a key role in establishing the

psychological foundations from which contemporary concept mapping theory and methods

evolved. Like pictures, concept mapping today produces a visual representation of accessible

information in a specific orientation. Generally, theory in cognitive mapping emphasizes

humans‟ systematic acquisition, storage access and utilization of knowledge (Golledge, 1986).

In schemata theory, discussed by Milligan (1979) and Sholl (1987), concept mapping processes

directly parallel the schema system.

Schema systems are active processes by which mental pictures or understanding of

something learned is developed (Smilkstein, 1991). People acquire knowledge only to the

degree by which they have constructed schemas from learning experiences. Concept mapping

technologies are embedded in cognitive learning theory. In general, the acquisition and storage

of knowledge delineated in cognitive learning theory directly parallels concept mapping steps

defined by Trochim (1989). In learning theory, learners are stimulated to activate related

knowledge in a particular area. Similarly, participants in concept mapping processes are

encouraged to access related knowledge on the area under focus during the brainstorming phase.

26
Cognitive learning processes of guiding learners to develop new structures or knowledge about

the structures is paralleled by the processes in concept mapping of generating and developing

items and interconnections.

Like cognitive learning theory, concept mapping processes consolidate new structures

and knowledge. In cognitive learning theory, under appropriate conditions, learners acquire a

more unified, complex understanding of the phenomena in question. In concept mapping the

consolidation of information is demonstrated by the aggregation of information displayed using

individual or group maps. These maps help participants develop broader and more common

understandings of the information displayed.

While cognitive theory provides structure for the perceived acquisition and integration of

knowledge, sociological principles provide the processes for understanding the connections in

terms of “social processing” (Garling, 1984). Huberman and Cox (1990) contend that the

acquisition of knowledge is an interactive process between and within the environment. It is

these interactive networks that are precisely the foundations on which concept mapping

variations rely. In other words, group and individual constructs are established during an

interactive process in conjunction with individual experiences and strategies. Concept mapping

relies heavily on these interactions in creating construct maps that reflect these communications.

This position is consistent with a long line of psycho-sociological research emphasizing the

importance of socially constructed thoughts that make learning at both individual and group

levels possible (Bandura, 1986).

27
 In social processing, the acknowledgement and rationalization of thought construction is

defined through the interactions of people. Similarly, in concept mapping, knowledge is

constructed through an interactive link between participants. Open discussions, interviews and

focus groups generate the items to be used in the application. In concept mapping, people

construct and understand the maps as networks among thoughts of individuals within groups. In

essence, the final group map is a visual representation and acknowledgement of thoughts

constructed through social interactions between people.

As in other research, concept mapping processes can be situated in a variety of

epistemological stances. Traditionally, cognitive theory relied heavily on “scientific” inquiry

and was immersed in decidedly positivistic or logical empiricist approaches and principles. This

stance allowed the social sciences to simulate the apparent objectivity assigned to the natural

sciences. Historically, interpretivist epistemology arose from the critiques of positivism in the

social sciences. In particular, interpretivists disagreed with social science attempts to import

standards and procedures of natural sciences in order to study human beings in society (Swandt,

1994). Philosophically, interpretivist researchers construe meaning as the primary focus in

exploring the nature of social reality. “Facts” are not entities waiting to be discovered in the

natural, objective world. Instead, they are social constructions of the ways human beings

experience actions through interpretive activities. Concept mapping methodological techniques

and strategies fit with the tenets of interpretivism established here. Since qualitative methods are

often associated with interpretivist research, concept mapping approaches that reflect these

structures support the interpretivist paradigm effectively.

28
 While interpretivisits emphasized the world of experience as it is lived and felt through

social interactions, constructivists stressed the construction of knowledge (Swandt, 1994). At the

risk of oversimplifying, constructivists assume that the terms by which the world can be

understood is predicated by social interchanges among people. Variations in concept mapping

approaches that surfaced in constructivist approaches, adhered to these more adaptive social

functions of cognition. More interactive processes with individuals and groups were evidenced

in these concept mapping applications. While concept mapping as a tool tends to be used in

ways consistent with interpretivist and constructivist paradigms, as a methodology it is free of

ties to any particular philosophical orientation to knowledge and ways of knowing (Rizzo-

Michelin, 1998). In this respect, it is similar to several qualitative methodologies that could be

applied in preordinate (Miles & Huberman, 1994) or emergent (Lincoln & Guba, 1985) ways.

From another epistemological perspective, concept mapping can also be employed as a

critical theoretical tool and as an aid to encourage participants to gain a deeper understanding of

their circumstances thereby fostering self-determination and responsibility. Such approaches

would be highly participatory and engaging for participants (Fetterman, 1994). As a critical tool,

participants commit to social justice principles that are inevitably value centered. Within this

frame, concept mapping can be used as a tool to shape knowledge in an emancipatory context.

Interpretivist, constructivist and critical theory research paradigms are the foundations form

which variations in concept mapping, and subsequent analyses of concept mapping output, arise.

Despite epistemological differences, concept mapping applications are flexible and adaptable

processes. It is however, the underlying paradigm that researchers bring to the inquiry process

that guides which concept mapping approach will be adopted.

29
 This review has highlighted many of the key ideas important in studying student

engagement in school. Given that the literature pertaining to student engagement is broad the

attempt here was to highlight important dimensions that would serve to inform the central

research question of this study. That is, how do teachers‟ and students‟ define student

engagement? As such, the review began by highlighting prior work that has established a solid

base linking engagement to achievement. To be successful at school, students need to be – at

some level – engaged with learning. If engagement can be established as critical to student

learning, then many would question why so many students are not being served by our schools.

Perhaps part of the problem lies in the definitional ambiguity of the construct of student

engagement itself. Many definitional attempts have produced three categories of engagement,

but it has been suggested that instead of examining engagement in silos the focus should be on

positing engagement as a multidimensional construct. In this way, attempts to define

engagement would capture its richness, depth, and complexity.

Next, the review highlighted factors that act to mediate student engagement within

schools. In this study, engagement in schooling includes a focus on schools and the classroom.

In this light, issues such as school climate, student voice, and students‟ relationships with their

teachers and peers were covered. Clearly, there is an incredible amount of research documenting

the factors that mediate student engagement in schooling, yet the field of educational research

appears to have little influence on the way schools operate. Perhaps researchers need to put

more focus on altering current research methods and approaches. The final section of the review

30
explored the challenges of quantitative research approaches, and the prospects for participatory

methods. Concept mapping, as used in this study, was reviewed and presents as an alternative

approach in providing students and teachers with tools for inquiry.

31
 CHAPTER THREE: CONTEXT

In 1991 the Walter and Duncan Gordon Foundation (WDGF) began a program of school

improvement in Manitoba, Canada. It was conceived of as a pilot project to develop and test a

Canadian school improvement model, with an emphasis on improving schooling particularly for

students at risk. Manitoba was chosen for the pilot because it was an appropriate but manageable

size for the planned initiative with a government that was open to foundation involvement (Earl,

Torrance, Sutherland, Fullan, & Ali, 2003). Throughout the life of MSIP, the Walter and

Duncan Gordon Foundation have commissioned a number of evaluations to describe the role and

influence of MSIP (see Earl & Lee, 1998, Earl et. al., 2003; Fullan, Kilcher 1996). MSIP has

outlasted many other school improvement networks around the world and has become an integral

part of education in Manitoba.

In this study, MSIP served as a context to investigate schools that were embarking on

school projects aimed at increasing levels of student engagement within their buildings. From

the outset, MSIP has had two core goals, 1) improving student learning, and 2) increasing student

engagement. Student learning has been viewed as the heart of school improvement. In MSIP

schools, learning is viewed as not strictly academic, but includes a broad range of knowledge,

skills and attitudes. As Earl et al., (2003, p.15) note, “learning doesn‟t just happen. It requires

intentional and sustained efforts by teachers and students.” MSIP engaged in a variety of

activities all aimed at improving the experiences of secondary students in Manitoba high schools.

MSIP build networks of schools and districts interested in improving the outcomes for students.

32
A key component of this networking activity included MSIP acting as “critical friends” to

schools within this network. That is, MSIP would provide both pressure and support to schools

as they engaged in school-based planning, and data-driven analysis of issues and findings (Pekrul

& Levin, 2007).

School-based evaluation was a central focus of the MSIP initiative at the school level.

Participation in the MSIP network required schools to produce an annual evaluation report based

on data collection throughout a given year (Katz, Sutherland & Earl, 2003). MSIP advocated a

distributed leadership model, and thus felt the involvement of students in school improvement

efforts was critical for success. From its inception MSIP placed importance for students to act as

partners in educational change. Programs designed to include students included the following:

Students at the Centre (SAC), student conference “Make a Choice! Raise Your Voice”, and an

advocacy program “Student Voice: Voices of Today and Tomorrow (SVT) (Pekrul & Levin,

2007).

In the ongoing evaluation of MSIP, student engagement was identified as a precursor to

learning. Attempts to gauge the “levels” of student engagement occurred on two levels: 1)

students‟ relationship with the learning environment (school atmosphere/climate, student voice

in decision-making, student participation in school activities, and student relations with teachers,

and 2) students‟ relationship to their own learning (motivation to learn, confidence in their own

ability to succeed, relevance of course/curriculum, interest in course/curriculum). This study, in

part, extends the broader work of MSIP, and the site schools specifically, by probing deeper in

the student engagement construct.

33
 Two schools within the MSIP network were selected for inclusion in this study. These

schools were included in this research to maximize the numbers of focus groups with both

students and teachers while keeping constant several important features of the school. That is,

the goal was to try and reflect diverse settings in a way that was realistic for a research study of

this nature. Thus, limited capacity both in terms of “person-power” and resources meant the

study had to be limited to two secondary schools. The schools were chosen for their similar

characteristics in that both schools are in the same school division, had a focus on engagement in

their school improvement plan, and had been involved with MSIP for over five years. Also,

principals at both schools noted that they had allocated resources to the MSIP sponsored

Students at the Centre initiative, a program formed to explore the potential of student voice in

supporting positive change in high schools.

Further, the schools are relatively similar in terms of demographics and socio-economic

status. Both schools offer instruction in English only. School A, is an 8-S4 (12) structure,

School B is an S1 (grade 9) – S4 (grade 12) structure. The two high schools are both located

within a city and service a mainly working class environment with many parents working several

jobs. Teen pregnancy is a serious concern within the province and manifests itself in the study

schools with the inclusion of infant labs (day care) on school premises. During the time of data

collection (2002), the principals at each school were female, in their mid forties, and had taken

over their administrative positions within the past three years (both women had been former

teachers in different buildings). At the time of this research, School A‟s population was

approximately 1,200 and School B‟s was roughly 950.

34
School A

For the two years prior to collecting data for this study, School A had been undergoing a

process of amalgamation with the adjacent junior high school. The principal described this

process as difficult. She explained that it meant that over 30 teachers had to leave the senior

high school. She added that due to the instability, school planning as a whole had been difficult.

At the time of my initial interview (September 2002), the principal appeared optimistic that the

staffing and administration were finally stabilized so as to move forward as a S1 to S4 school.

The school‟s improvement initiatives were described as containing many programs under

one larger project. A significant part of the larger project‟s aim had been making high school a

more personal experience for each student. The principal put it this way, “having each student

feel that staff have a genuine interest in knowing and understanding their goals, objectives and

individual needs is integral to creating this culture.” After additional probing regarding the

specific programs in place within the school, the principal explained that one of their goals is to

explore „their understanding‟ of student voice, and to support more opportunities for student

directed activities in the classrooms. One aim, she continued, was that such a forum would allow

students to share their educational experiences with community including the provision of some

response and dialogue.

Evidence of this focus included the implementation of the teacher advisor model, schools

within school to create smaller settings for student learning, and implemented “self-directed days

35
Wednesdays.” Students determine their own timetable by selecting form a menu of activities,

opportunities for assistance and course assignments). The school has remained consistent in

their school improvement approach as evidenced by the principals‟ comments in 2002. This is

how the principal described their current focus towards increasing levels of student engagement:

I think on a personal level that for students in this school there are a lot of opportunities
for them to connect with some really caring adults, informally and formally, which
weren‟t here before. I think that the self directed Wednesday has been a really important
piece in terms of its impact on kids. Some of the data that we‟ve collected or that the
school has included comments from previous students who have commented that self-
directed days were really very valuable. They just weren‟t sure that they recognized it at
the time, but they did learn how to use their time. And when they went to university and
had a free afternoon or whatever, they did have a sense that they were accountable and
they were used to an agenda book – the kids all have an agenda book that is checked by
their TA.

The principal described her understanding of student engagement as students being meaningfully

involved in school life. She acknowledged the complexity of the engagement construct as she

said, “we all come to view things through our own experience-coloured lenses.” When asked to

describe the level of success of the school improvement initiative, the principal noted that

structural and programmatic changes had taken place but concurred that the classrooms had

remained largely untouched by their school improvement focus.

School B

The focus of the improvement program at School B has been described by the principal

as focusing on student engagement and best learning practices. When asked for specific

examples, the principal noted that they were working on the development of teacher advisor

36
system1 and ongoing curricular developments. She explained that the school

development/improvement committee meets two Tuesdays per month (after school) and has

roughly equal participation from teachers and students. The principal at School B described the

culture of the school in this way, “this is the kind of place that everyone who walks through the

door loves to work at right away, and we‟ve got a warm caring staff”. When asked to describe

what student engagement meant to her, and more broadly for the school, she noted “providing

multiple opportunities for student involvement & looking at impact in the classroom”

Changes in school structure that have resulted from the school improvement initiative

include the implementation of “Wacky Wednesdays”. The principal explained that each

Wednesday the students experience a “scrambled” schedule. The pedagogical rationale for this

initiative is to give students a change of routine. The principal also noted that she would initiate

a complete timetable change for the 2003-2004 academic year. According to the principal, “I

compiled in-class time and in reviewing the data I realized that we could add more instructional

time by altering the timetable.” At the time of data collection, the principal agreed that no

specific classroom-related changes had occurred as a result of their school improvement project.

The next chapter – methodology – will serve to explain how the study data was collected.

The chapter is sub-divided into the six subsequent steps involved in the concept mapping

method. Throughout this chapter, illustrative tables and diagrams will be presented to assist the

reader in orienting them to the method.

1
The Teacher Advisory system was being implemented slowly at School B. During data collection, 2001-2002 the
school had begun professional development with the teachers who would be mentors.

37
 CHAPTER FOUR: METHODOLOGY

Ethical Considerations

This study was cleared by the Human Ethics Committee of the Ontario Institute for

Studies in Education at the University of Toronto (2003). In addition, project approval was

granted by the participating school division as well as the principal‟s at the two school sites.

Oral and written communication with each of the participants detailed the purpose of the

research and discussed confidentiality, anonymity, and withdrawal rights. Participants in the

study individually indicated informed consent by signing a release form (see Appendix A).

Undeniably, student engagement is a construct fraught with definitional

ambiguity. As a participatory process, concept mapping holds the potential for group members

to foster some ownership in the school improvement agenda. Specifically, Trochim and Linton‟s

(1986) general framework for structured conceptualization was utilized as a tool so as to

facilitate group conceptualization. That is, the authors propose a specific type of structured

conceptualization process which they term “concept mapping” (Trochim & Linton, 1986). In

concept mapping, ideas are presented in the form of a picture or map. To construct the map,

ideas first have to be described or generated, and the interrelationships among them articulated.

Multivariate statistical techniques – multidimensional scaling and cluster analysis – are then

applied to this information and the results are depicted in map form. Table 1 illustrates the six

sequential steps in Trochim‟s model. Of course, Trochim and Linton‟s model is not the only

way to accomplish concept mapping, as others have advocated that maps can be drawn free hand

(Novak & Gowin, 1984; Rico, 1983), but the major difference from these approaches and

38
Trochim‟s is that the latter method is particularly appropriate for group use. That is, this method

generates a group aggregate map, and thus is better suited as a tool to investigate teachers‟ and

students‟ views of student engagement as used towards a group (teacher and students) meaning

making.

39

Step 1
Preparation
There are two major tasks which
must be undertaken prior to the
start of the actual group
process: (a) facilitator decides
who will participate in the
process, and (b) facilitator
works with the participants to
decide on the specific focus for
the conceptualization.
A. Selecting the participants
works best when it includes a
variety of relevant stakeholders.
Broad heterogeneous
participation helps to insure
that a wide variety of
viewpoints will be considered
and encourages a broader range
of ‘buy in’ to the resulting maps.
There is no strict limit to the
number of people who take part
in the process. According to
Trochim (1989, p.3), “one might,
for instance have a relatively
small group do the statement
generation (e.g., brainstorming)
and a much larger group
perform the sorting and rating.
In general, however, we have
found that concept maps are
better understood by people
who have participated in all
phases of the process than by
those who have only taken part
in one or two stages.
B. In general, it is best if the
final set of brainstormed
statements are ‘of a kind’, that
is, share the same level of
conceptual generality and
grammatical structure.
Table 1 Sequential Concept Systems® Stages in Structured Conceptualization
Step 2
Generation of
Statements
This step signals the beginning
of the actual concept mapping
process with the generation of a
set of statements which ideally
should represent the conceptual
domain for the topic of interest.
Typical of brainstorming
exercises, people are
encouraged to generate lots of
statements and are told that
there should be no criticism or
discussion regarding the
legitimacy of statements that
are generated during the group
sessions.
Theoretically, there is no limit to
the number of statements
which can be generated, though
large numbers of statements
can impose serious practical
constraints. According to
Trochim (1989, p.5), “based on
our experience, we now limit
the number of statements to
one hundred or less.” Once the
final set of statements has been
generated it is valuable for the
group to examine the
statements for editing
considerations.
Step 3
Structuring of
Statements
The statements are then
entered into the Concept
Systems software and decks of
cards are produced (each card
in the deck contains one of the
statements). This can also be
done manually be using 3 X 5
cards and writing out a
statement on each card.
Next, the brainstorming groups
(step 2) are reconvened and
each person in the group is
given a deck of cards. Each
person is instructed to sort the
cards in a way that makes sense
to you. There are several
restrictions: (a) all statements
cannot be put into a single pile,
and (b) all statements cannot be
put into their own pile.
The final activity in the
structuring step involves
participants rating each
statement in each pile.
Typically, this is accomplished
using a Likert-type response
scale (e.g., 1-to-5) to indicate
how much importance, priority,
effort or expected outcome is
associated with each statement.
Step 4
Representation of
Statements
Step 4 tasks: (a) data entry, (b),
system generation of concept
maps, and (c) decisions on final
cluster solutions.
A. All sorted and rated cards are
entered into the software.
B. The Concept Systems
software performs two-
dimensional multidimensional
scaling analysis to map the
brainstormed statements into a
two-dimensional plot. To
represent the conceptual
domain the system performs
hierarchical cluster analysis.
This analysis is used to group
statements which presumably
reflect similar concepts.
C. All hierarchical cluster
analysis procedures give as
many possible cluster solutions
as there are statements. The
task for the analyst (and/or
facilitator) is to decide on how
many clusters that statements
should be grouped into for the
final solution. According to
Trochim (1989, p.8), “there is no
simple way to accomplish this
task, essentially, the analyst
must use discretion in
examining different cluster
solutions to decide on which
makes most sense for the
project at hand. Usually,
assuming a set of a hundred or
fewer statements, we begin by
looking at all cluster solutions
from about 20 to 3 clusters.”
Step 5
Interpretation of Maps
The outputs from step 4 are then
assembled. These outputs consist of:
1. The statement list
2. The cluster list
3. The point map
4. The cluster map
5. The point rating map
6. The cluster rating map
7. Pattern match
Each of these maps is a concept map,
and each tells something about the
major ideas and how they are related.
That being said, however, it is the
cluster map that is usually the most
directly interpretable map.
Step 5 involves reconvening the
brainstorming groups or broader
groups of similar stakeholders for the
interpretation exercise. The
facilitator begins by giving the group
the final set of brainstormed
statements. Participants are then
reminded that these statements were
sorted and rated. Next, the cluster
map and cluster rating maps are
distributed. Each person is asked to
read through the set of statements
for each cluster and come up with
name that seems to describe it.
Then, the group works cluster-by-
cluster in an attempt to achieve
group agreement on an acceptable
name for each cluster.
Step 6
Utilization of Maps
At this point in the process
the group(s) is redirected
back to the original
purpose for conducting
the structured
conceptualization. A
number of suggestions for
ongoing planning and use
have been recommended:
*discuss how the final
concept map might be
used to enhance the
planning and/or
evaluation effort
*Each cluster could be
assigned to a sub group to
closely examine issues
*the map can be displayed
and prioritized in terms of
immediate, short and
longer term goals based
on the priority ratings
*for planning purposes the
concept map can also be
used as the framework for
an outline of a planning
report (clusters could be
used as sub headings)
*concept map could assist
with measurement
development (i.e.,
statements within clusters
could be considered as
survey items).
40
The remainder of this chapter will outline how the concept mapping process was conducted at

the sample schools. As outlined in table 1 the first step in the concept mapping procedures

involves convening a sample.

Step 1: Sample

The sample of student and teacher participants was drawn from two MSIP secondary

schools. As described in chapter one, this study was part of a larger MSIP study. The two

schools in the present study have been involved with MSIP since its inception and both schools

had increasing levels of student engagement as a primary focus as outlined in their school

improvement plans. Due to their interest in student engagement and their willingness to work on

additional research, each of the high school principals were contacted and invited to participate

in the present study. After numerous conversations both principals enthusiastically agreed and

assisted in the creation of the student and teacher focus groups. Ideally, the student groups

would contain a mix of gender, grade representation, ethnicity and academic achievement level

so as to better represent the demographics of the school. Similarly, the teacher group would

contain a mix of gender, grade level expertise, and teaching experience. Another requirement of

the sample was that the student group be subdivided to ensure representation from both grade 10

and grade 12 students.2 Students in grade 10 were selected as they are at the legal age whereby

they must remain in school. Grade 12 was used as these students, for one reason or another,

have made a choice to stay in school.

2
Grade determinations were most important for the overall evaluation work so as to utilize a targeted survey
approach. For more information see Earl & Lee (1989). Further, these grade breakdowns are commonly used by
large-scale student engagement survey work (Frase, 1989; Rumberger, 1995; NSSE, 2007).

41
 The teacher sample varied with respect to gender, age, grade level (range S1 – S4), and

years of teaching experience (range 2 yrs – 15yrs). Teachers varied with respect to their subject

of expertise as focus groups were comprised of teachers from across teaching disciplines (e.g.,

mathematics, English, history, science, Phys Ed). All teachers were Caucasian, and taken

together, these characteristics produced a „teacher‟ sample that accurately reflected the wider

teaching faculty at both schools. Principals at each school requested teacher volunteers via email

requests and posters in the staff room asking for volunteers. Students were asked to volunteer by

the principals and through teacher nominations. Ultimately, the student and sample was a mix of

random, purposive, and self-selection which produced a final sample of 33 participants arranged

in 3 groups of 8 and 1 group of 9 (see Tables 2 & 3 for brainstorming focus group

characteristics).

Table 2 Student Brainstorming Focus Group Characteristics – Schools A & B

School A B

 Total participants 8 9

 Gender breakdown 2 female, 6 males 4 females, 5 males

 Ethnicity Caucasian, Aboriginal, Eastern Caucasian, Aboriginal, Eastern

European & East Indian European & African American

42
Table 3 Teacher Brainstorming Focus Group Characteristics – Schools A & B

School A B

 Total participants 8 8

 Gender breakdown 6 female, 2 male 4 females, 4 males

 Ethnicity Caucasian Caucasian

 Teaching experience Mixed (range 1 year – 18 years) Mixed (2 years – 10 years)

The preliminary concept mapping activity required members of each group to generate

statements in response to a focus instruction regarding their own definition(s) of student

engagement. I facilitated four sessions, two at each school.3 Upon meeting the student and

teacher groups, I spent some time introducing myself, having participants introduce themselves,

explaining the nature of the study, and then each person was briefed about the specific

requirements of the research. Next, all individuals were given a letter of informed consent and

asked to sign it (see again Appendix A). Students and teachers at both schools were already

familiar with the term student engagement, and appeared both curious and interested in the

processes of the study. One student queried, “so, like, do I get to be in this for a few years?”

The initial focus group meeting with teacher and student groups followed Trochim‟s

concept mapping closely and consisted of brainstorming/statement generation sessions (Trochim,

3
To supplement the data obtained during the concept mapping brainstorming sessions, permission was
obtained from respondents to audio tape these meetings. Each of the sessions lasted from sixty to 90
minutes, subsequently the tapes were transcribed verbatim.

43
1998). These sessions took on the nature of a free flowing focus group whereby participants had

the opportunity to engage in discussion around the statements, and more general dialogue about

what student engagement meant to them. This preliminary concept mapping activity included

giving the participants the instruction: generate statements (short phrases or sentences) that

related to [teacher/student] perceptions of student engagement. My role during these sessions

was to keep individuals on task, and to record all statements generated on chart paper. The

following section will highlight the student lists, the teacher lists, and finally the unified “student

engagement” list.

Step 2: Generation of the Student (Statement) List

The School A student focus group was held over their lunch period with the permission

(from the principal and relevant teachers) to retain the students into the next period as necessary.

Students at School A produced an initial statement list comprised of forty-three statements (see

Table 3). At first, students were surprised that their opinions were being sought. They

repeatedly asked me thinks like, “so you really want to know what we think” and “this is really

cool that you are asking us.” The session lasted for approximately sixty minutes, and as more

time went on students would ask, “what are you going to do with this stuff again” and “are they

[administration] going to make any changes as a result of this research?”

44
Table 4: School A - Student Brainstorming List
Item Short phrase or sentences that relate to Student Perceptions of Student Engagement
Number

1 Involvement in school activities (i.e., sports, robotics)

2 Student council promotes decision making

3 Phys Ed promotes leadership qualities

4 School courses that enable you to “build something from nothing”

5 Availability of extra-curricular programs (i.e., infant lab – child care)

6 Hockey team has great school spirit

7 Playing sports reduces boredom in school

8 Increased teacher involvement in school activities (i.e., art and drama teacher)

9 Style of the teacher (i.e., if the teacher is open and allows you to have your own ideas,
relaxed class, feel more at home in this class)

10 Students and teachers meeting each other half way (i.e., students show initiative and
teachers will respond positively)

11 Personality of teacher is important

12 Older teachers have more difficulty relating to students – they grew up in a different time

13 Strict teachers make you work harder as they have higher expectations

14 I love teachers who try to make school fun

15 Teachers who explain information, talk to you, and go out of their way to help you are
critical for engagement

16 Some teachers are hypocritical because they expect students to make school their first
priority but teachers don‟t (i.e., teachers do not return assignments promptly)

17 Teachers who are empathetic to students lives (i.e., some students have kids of their own)

18 Variety in teaching style

19 Lots of course options

45
20 Teachers who are understanding and want to help you with your problems

21 School diversity is important – school is accepting of different social groups

22 School safety is important

23 School atmosphere is relaxed and comfortable (i.e., no gangs here)

24 Student in this school stick together and help each other out

25 The community is tolerant/understanding of the students here – we‟ve come a long way

26 Parental involvement is not a big factor in school life

27 Parents are concerned only at report card time - assessment

28 Students dislike the principal

29 Some teachers here are mean

30 Older teachers are more set in their ways – don‟t want to change with the times

31 Younger teachers can relate to students – make you want to come to school more

32 It is important how teachers make you feel when it comes time to graduate

33 Teachers understand the parent-student report card relationship – teachers don‟t want to
screw you

34 The student body at this school is great

35 Student attitudes in this school are amazing

36 This school provides lots of different types of field trips

37 It helps if students are involved in school life (i.e., MSIP conference crew, plays, student
council) as it makes you more motivated to come to school

38 This school gives student parents another chance at an education (i.e., infant lab) – open
minded to realities of the world

39 Internship program is good and the teacher who runs it will bend over backwards for you

40 The principal is too judgmental, too strict and doesn‟t try to get to know the students

41 Wacky Wednesdays are dumb. Nobody likes to go and it‟s really confusing

46
 42 This school has no “student space”. We have nowhere to go when we are not in class

43 The school policies at this school are unreasonable as we aren‟t allowed to sit in the halls

A total of nine students from School B participated in the initial focus group. As with the

group at School A, this session was held during the lunch period, and, was also granted

permission to remain in the group into the next period if necessary. This group convened for

approximately sixty minutes and produced an initial statement list of forty-six statements (see

Table 5).

Table 5: School B Student Brainstorming List

Item Short phrase or sentences that relate to Student Perceptions of Student Engagement
Number

1 Student involvement (i.e., student council, leadership) is important

2 Having a say in classroom work is important

3 Smaller classes make you feel more comfortable answering questions

4 The teachers characteristics (i.e., caring, passionate) are critical to student engagement

5 Classes that are meaningful and relevant to my life and interests

6 The most important part of school is a teacher who cares about what he is teaching

7 Teachers who let us think for ourselves

8 Teachers who act as coaches/mentors

9 A teacher who can change their teaching style to suit different learning styles

47
10 Teachers who compromise with students

11 Self-directed days are great ways to let students structure their days (note – this idea was
originated by a suggestion from a former student)

12 This school listens to students – student voice

13 The school council here does a good job at involving lots of students (i.e., planning
dances and pep rallies)

14 Younger teachers are more flexible and can relate better to students

15 Lots of selection regarding sports

16 The guidance department is knowledgeable and helpful

17 This school used to have a bad reputation but the bad students are gone now

18 The community is tolerant of the students here but students at other schools still don‟t
respect us

19 The grade 12s at this school are more responsible

20 As you progress from grade 9 to 12 you just become more involved naturally

21 Balance of student and teacher ideas

22 Teacher reputation is important

23 If you are doing bad in a class you are unmotivated

24 I enjoy seeing my friends in my classes

25 Friendships are the most important thing in school. If you have a fight with your friends,
it affects everything

26 Student-teacher relationships are key to engagement

27 Students care about what teachers think, although teachers don‟t often realize it

28 Smaller classes are better

29 Should have choice in class work distribution

30 Course selections are difficult at this school because the best courses fill up quick

31 Important to build yourself a good reputation with teachers

48
 32 There has been a high administration turnover here which makes student-admin
relationships harder

33 We have a 10-80-10 rule here (10% of the students won‟t do anything, 80% are willing,
and 10% do everything)

34 You used to be considered a nerd if you did choir, now it is a cool thing to do

35 There isn‟t much programming for students who fall in the middle, lots for top and bottom
end students

36 Teacher reputation is the most important thing. Teachers need to have a connection with
the students

37 This school needs more younger teachers

38 The older teachers need re-training, they often forget they are here for the students

39 Curriculum needs to be studied. The differences between the three levels of math
(academic, applied and consumer) are too severe

40 This school is very multi-cultural and we learn a lot from the other cultures

41 This school is safe because the people are caring

42 We need more support for student ideas (i.e., student voice and MSIP)

43 We need better organization of scheduling at this school

44 Students who are more involved in school will be better, more rounded persons

45 Involvement in school is rewarding

46 Engagement is an individual choice as some students are loners by nature

Like the group at School A, this group was surprised that their input was being sought.

However, it quickly became apparent that they did not hold the principal in high regard. I spent

some time assuring students that their comments would be kept confidential. A common

sentiment from these students was that the principal did not care what they thought and they felt

49
she was “trying to get them” in some way. After some discussion, we were able to get past their

negative feelings toward the administration and have a productive brain storming session.

The next step in the concept mapping process is to unify a stakeholder group statement

list (Trochim, 1989, 1993). That is, this application of the concept mapping design sought to

discern student and teacher perceptions of student engagement for the purposes of better

understanding. Individual student-level (and teacher-level) differences were not a concern for

this study, the two student statement lists were merged one statement list representative of the

views of “students” as a unified stakeholder group.

Generation of the Teacher (Statement) List

A total of eight teachers from School A participated in the initial brainstorming/

statement generation session. This group was held after school with food and beverages

provided. These teachers seemed to genuinely take pleasure in the opportunity for dialogue as

one could easily see that they enjoyed being together, and worked well as a team. For example,

they engaged in an enthusiastic and lively discussion, and at times debated their perceived

notions of student engagement (as well as approaches to pedagogy). The session went on for

over two hours. At end of the session the teachers had produced a list containing twenty-seven

statements (see Table 6).

50
Table 6: School A - Teacher Brainstorming List
Item Short phrase or sentences that relate to Student Perceptions of Student Engagement
Number

1 Exploration of ideas

2 Balancing of student and teacher ideas

3 Make material personal by establishing relationships

4 Make material relevant by linking class work to society and community. Use practical
examples

5 Students who are engaged outside the classroom will likely be engaged while inside the
classroom – individual nature

6 Offer lots of extra-curricular activities as potential incentives to “hook” kids

7 The school climate can act to engage kids even if they are not academically strong

8 There exist many levels of engagement. The kids who are characterized as extreme
attendance problems haven‟t found their “hook” yet

9 Staff engagement with school

10 Different types of engagement. Some kids are highly academically oriented and some are
more athletic

11 Government policies influence engagement in schools (i.e., schools of choice legislation)

12 Students need to take ownership in their learning

13 The challenge for teachers is to appeal to a broad range of students and still make class
work challenging without being unmotivated

14 Types/levels of engagement. Highly motivated, overachievers need to learn how to

prioritize

15 Balance workload and comfort zones. You want all kids, at some point during a given
week, to be in and out of their comfort zones.

16 Encourage all students to do their best and become independent learners

17 Need to balance school policy and the individual needs of students (i.e., if you haven‟t
attended a number of classes you can‟t play sports but what if sports if the “hook” that

51
 brings this student to school in the first place?)

18 Teacher knowledge. What do teachers need to know to engage students?

19 Teacher needs to facilitate an environment where a multitude of emotions can be

expressed (i.e., anger, sadness, happiness)

20 Students need to be able to trust teachers

21 Teachers need to ensure that students feel they can take risks with their learning

22 Teachers need to be completely involved in student learning

23 School must be supportive of parental roles

24 Unique course offerings

25 Schools needs to be a place of strong support for students and their families (i.e., notion of
neighbourhood school)

26 School needs to be a safe place

27 Staff should be visible within the community (i.e., live and coach)

52
 At School B eight teachers volunteered to participate in the focus group. This session

was conducted after school (again food and beverages were provided) and lasted approximately

sixty minutes. The School B group was not as “lively” as the teachers at School A. Here,

teachers seemed to want to get on task so they could go home. Teachers at school B did not

appear as enthusiastic about the discussion. Instead, they stuck to their task of producing a list

by calling out items to be recorded on chart paper. The dynamics of this group could have been

affected by the principal insisting on being a part of the focus group discussion. Despite my

„gentle‟ urging her not to attend the group, she insisted that she would add to the discussion.

This group produced a list containing forty-six statements (see Table 7).

Table 7: School B - Teacher Brainstorming List

Item Short phrase or sentences that relate to Student Perceptions of Student Engagement
Number

1 School should act as a “home away from home”

2 Students need to be interested in some facet of the school

3 Engagement takes on many forms (i.e., being present does not necessarily constitute
engagement)

4 Various levels of engagement within the school (i.e., socializing with peers can take on
different forms of engagement)

5 Unique school programming (i.e., MAP)

6 School needs to offer social aspects

7 School has a wide variety of program offerings (i.e., academics, athletics, extra-curricular)

8 Large school can offer more choices

9 Large school has more staff and therefore more chances of students connecting with an

53
 adult in the building

10 Students want to be in a safe and comfortable place

11 Class material must make sense to them

12 Teachers need to offer incentives and instill competition

13 Phys Ed program promotes life-long activity (i.e., engage in activities outside the
curriculum – skateboarding)

14 Variety of exposure to teaching styles

15 Students who make a presence in your room are more engaged

16 Cooperative learning is important

17 Make your classroom an interesting place to be. Teach for a wider majority of students

18 Seek out individual talents

19 Gauge teaching to needs (i.e., In Senior 4 most students don‟t need teachers, they need
mentors and guides)

20 Ownership and relevance of tasks

21 Find something students care about

22 Place high expectations on students to encourage them to feel more valued

23 Recognize “mark-oriented” Senior 4s (i.e., competition for scarce scholarships)

24 Success breeds success. Get marking back to students in a timely manner

25 Often the “mark” gets in the way of engagement. I don‟t believe in averaging marks

26 Encourage engagement by using different testing strategies

27 Trigger students imagination

28 Make the material relevant. Put it in teenager terms

29 Be aware and empathetic to student life styles

30 Extra-curricular activities encourage closer student-teacher relationships

31 Make a human connection. Show students that teachers are humans too!

54
 32 Parents need to take responsibility for student learning

33 Set standards based on ability

34 Understand society we live in today

35 Students need to be hungry for learning

36 Students need a solid base for learning

37 Teach skills only when students need it

38 Teachers should emulate the behaviour they want to see

39 Teachers need to be qualified as students can quickly pick up when you are not

40 Shared teacher-student decision making to a point. Students need reference points to be

able to make judgments

41 Different levels and types of engagement

42 Trust building is important

43 Ask kids what they need

44 Structure of class can help engagement levels (i.e., proper gender mix or not at all)

45 Look for “hook”. Get one student on board and you will quickly have seven more. Be
entertaining

46 Force students to share

As with the two student lists, the teachers lists were merged into a single list representing

a unified „teacher‟ stakeholder group. The next step was to merge the unified student and unified

teacher lists into one master „student engagement‟ list. Developers of the concept mapping

software (Trochim, 1989; Trochim & Linton, 1986) recommend that a list of no more than one

hundred statements be used with the ideal statement list at 60. Based on prior concept mapping

studies, a statement list of sixty statements appeared to be both representative of all stakeholder

55
views as well as manageable (Cousins & Sutherland, 1998; Gans, 2002; Kane & Trochim, 2007;

Michalski & Cousins, 2000; Sutherland & Katz, 2005; Trochim, 1989). This step was undertaken

by the researcher and involved removing similarities and obvious redundancies within the larger

item list. This step is akin to a type of formal content analysis whereby the researcher needs to

ensure all main content themes are represented (Krippendorf, 2004). Every effort was made to

preserve a balance between student and teacher generated ideas. That is, approximately thirty

statements came from teachers and thirty from students.

Throughout this editing and reduction process, the proportionality of the original

set of raw statements was preserved so that approximately a quarter of the 60 statements

came from each of the four focus groups (see Table 8). This final list edit process is

commonly done by the researcher/evaluator in the concept mapping process (Gans, 2002;

Kane & Trochim, 2007; Krippendorf, 2004; Michalski & Cousins, 2000; Sutherland &

Katz, 2005; Trochim, 1989). Though not recommended by the developers of the concept

mapping system, the researcher reviewed this data reduction process with two doctoral

students familiar with this research. Given that the researcher/evaluator was also the

facilitator in the focus groups, they are intimately familiar with the content and context

for each session.

56
Table 8: Combined Student & Teacher Items
Student Engagement Brainstorming Items (Teachers & Students)

1. Guidance Support 31. High teacher expectations of students

2. Parental involvement 32. School safety

3. Timely grading/assessment of student work 33. Need for student space

4. Opportunities for student leadership 34. Compromise

5. Active role in classroom discussions 35. School spirit

6. Teacher as friend 36. Doing work in groups

7. Teacher is passionate about teaching 37. Student ownership in learning

8. Lots of program/course options 38. Balance of teacher & student ideas

9. Student independence in learning 39. Community involvement

10. Variety in teaching style 40. Different types of school involvement

11. Relevance of class material 41. School policies

12. Incentives to promote student learning 42. Teacher job satisfaction

13. Extra-curricular activities 43. Student input into learning

14. Changes in regular timetable 44. Trust

15. Small classes 45. Having the same teacher/student more

than once

16. Involvement in student council 46. Competition

17. Hiring of younger teachers 47. Student attitudes/interest in classes

18. School reputation 48. Have fun in class

19. Feel comfortable at school 49. Experience success in learning

20. Teachers‟ reputation 50. Different assessment methods

21. Importance of friends/socializing 51. Student motivation to learn

22. Changes in administration 52. Demanding curriculum

57
23. Student-teacher relationships 53. Opportunities for teacher-student

decision-making

24. Teacher professional development 54. Teacher is entertaining/interesting

25. School is accepting of different social groups 55. Mixed ability classes

26. School is multi-cultural 56. Challenging class projects

27. Support for student ideas 57. Respect for others

28. Effective communication 58. Supportive principal/vice-principal

29. Interesting classes 59. Smiling

30. Involvement in sports 60. Being recognized

Step 3: Structuring of Statements

Next, these sixty statements were entered into the Concept Mapping System Software

package to produce decks of cards. That is, each card in the deck contained one of the

statements from the list, thereby producing a deck of sixty cards. The researcher produced a

deck of cards for each participant in the focus groups so as to engage in the next step in the

process. Student and teacher focus groups were reconvened (separately) and each participant

was handed a deck of cards. They were asked to independently read each statement and sort

them „in a way that makes sense to you‟. That is, place cards that they perceived to be similar in

groups. The step had two rules: 1) participants could not place all cards in one pile, and 2)

participants could not end up with sixty cards laid out. The students were particularly good at

focusing on this task Teachers seemed to take longer to understand the nature of the activity, and

one teacher in particular had difficulty sorting her cards. She continually wanted to overlay (as

opposed to sort into piles) various cards that she felt were „interrelated‟. Once the statements

58
were sorted, participants were asked to individually rate each statement in each pile on a five-

point scale in terms of their perceived importance (5 = extremely important; 4= very important;

3= moderately important; 2= somewhat important; 1 = not important). The meaning of the rating

scale had to be repeated several times for each group and was ultimately written on the

blackboard for easy reference. Overall, the sorting and rating process ran smoothly for all

groups. The sorted, rated cards were placed in sealed packages, labeled as T (teacher) or S

(student), and subsequently entered into the concept mapping software.

Step 4: Representation of the Statements

All sorting and rating data provided by each of the 33 respondents were analyzed as a

single project using Concept Systems Software. The software provided a convenient means to

perform the statistical calculations used to generate the initial concept maps, the refined maps

based on stakeholder input, and to generate the pattern matches. The major calculations

performed by the software include data aggregation, multidimensional scaling, cluster analysis,

bridging analysis, and sort pile label analysis. Such statistical analyses are akin to a simplified

version of hierarcial linear modeling. One could feasibly use various other statistical software

such as SPSS or SAS to perform these analyses. That being said, however, the concept mapping

system provided a user-friendly, windows-based application. The intent of this study is not to

focus on the statistical properties of the method but rather to highlight one way in which

stakeholders can collaboratively engage in this type of methodological process. The

aforementioned statistical procedures – multidimensional scaling (MDS) and cluster analysis,

and the application of such concept mapping techniques have been well described elsewhere

59
(see, Anderberg, 1973; Davison, 1983; Everitt, 1980; Gans, 2000; Kruskal & Wish, 1978;

Trochim, 1989)

Each individual‟s sort data was used to generate all of the Concept Map results in the

Concept System. First, each participant‟s unstructured similarity sort piles are concerted into a

binary matrix that is as large as the statement set itself. In this case, there were 60 statements in

the set, so the matrix was 60 rows by 60 columns. For illustration purposes only, Figure 3

provides an example of this concept for a single participant and a 10-statement project.

Following this example, if two statements were grouped together into a pile the corresponding

row and column intersection has the entry „1‟ to indicate the relationship. Otherwise, a „0‟ is

place into the row-column intersection to indicate that there is no relationship. The matrix is

perfectly symmetrical along the diagonal axis because each statement must be sorted with itself.

This explains why the value „1‟ appears at every statement row-column intersection. Thus, each

participant‟s sort information is converted into an N x N (where N = the number of statements in

the set) binary symmetric matrix of similarities (for example, see Gans, 2000). By transforming

a participant‟s sort data in to a binary square similarity matrix, a common data structure is

created that can be replicated for all participants. This allows each participant‟s sort data to be

aggregated with other project participants.

Using hierarchial cluster analysis procedures, the Concept Systems Software initially

produces a concept map with a default number of six clusters. All hierarchial cluster analysis

procedures give as many possible cluster solutions as there are statements. According to

Trochim (1989), these clustering methods begin by considering each statement to be its own
60
cluster (i.e., an N-cluster solution). At each stage in the analysis, the algorithm combines two

clusters until, at the end, all of the statements are in a single cluster. The task for the analyst is to

decide how many clusters the statements should be grouped into for the final solution. There is

no simple way to accomplish this task. Essentially, the analyst must use discretion in examining

different cluster solutions to decide on which makes sense for the case at hand (Gans, 2000;

Penny 2002; Trochim, 1989).

The goal is to examine which statements make sense in the various groupings. The

recommended cluster size is typically a six to twelve cluster solution (Gans, 2000). Based on

this knowledge, I examined all cluster solutions from six to twelve. Ultimately, I found that the

nine cluster solution was conceptually clear (i.e., the cluster sizes and importance ratings were

visually and statistically clear). Also, I asked several doctoral cohort members to look at the

maps and try to determine which would be easiest to work with. Undeniably, the task of

deciding upon a cluster solution requires the discretion of the analyst and would be ideal if

participants could be involved in this decision-making process. The selection of the cluster

configuration is most often done by the analyst (Trochim, 1989, 1993), however more recent

web-based applications of concept mapping as well as the use of internet-based software (e.g.,

Skype) enable this step to be more collaborative.

The next step in the data analysis is to incorporate the rating information into the map.

Until this point, the only data used as input for the analysis are each participant‟s sort data.

These sort data enable the Concept Systems to generate two-dimensional representations of the

61
brainstormed statement set. The rating information provides depth to those two-dimensional

graphics. As described earlier, each participant rates their perceived importance of every

statement in the brainstormed statement set on a Likert scale. Specifically, the rating measures

importance on a 1 to 5 scale where 1 = unimportant and 5 = extremely important. This

information is averaged for each statement in the set and represented in a “point rating” map. As

Gans (2000) describes it, the average rating for a point falls within the range represented by the

number of blocks associated with a statement. This ultimately enhances interpretation sessions

and orients participants to the next graphic, the “cluster rating” map.

The cluster rating map shows the final cluster solution and adds the same depth provided

by the rating data in a similar manner to the point rating map. The rating value for a cluster is

the average rating across all the statement ratings in the cluster. The cluster rating is represented

as layers varying in height from 1 to 5. Each rating constitutes a rating range whose value is

reported in the legend. The point cluster-rating map presents an image that has tremendous

impact on groups during the interpretation session. Figure 2 provides an example of the point

cluster-rating map for a project focused on what constitutes a fun, exciting and successful

science-based learning experience for young people (see www.joe.org/joe/2007august/tt2.php).

In this example, the thickest clusters (as indicated by the layer value) are deemed to be most

important factors contributing to a fun and exciting science-based learning experience (for

example, “characteristics of a good leader”, “creating a supportive environment”, and “fostering

respect/group dynamics”).

62
Figure 2: Example of Cluster Rating Map

The final analysis to be performed is the pattern match, which is essentially a graphic

comparison of the cluster rating maps for two demographic sub-groups – in this case students

and teachers. Pattern matching is powerful in its implications, particularly as a measure of

stakeholder consensus regarding their views of statement importance within specific cluster map

clusters. The results of a pattern match are represented both graphically (as a ladder graph) and

numerically (as a correlation coefficient) between measures. The ladder graph is comprised of

two vertical scales, one for each stakeholder group and is joined by sloping lines each

corresponding to a labeled concept map cluster. The correlation coefficient associated with each

pattern match ranges between –1 and +1. Values near 0 indicate the absence of a match; values

63
closer to either pole indicate stronger matches. Negative values imply an inverse relationship

(when one measure is high, the other is low and vice versa). Positive values imply a synchronic

relationship (high with high and low with low). It is a standard Pearson r (product moment

correlation) between the average ratings of the two variables, and it is useful to describe the

strength of the relationship between them (see Figure 3 for an example of a study on program

evaluation competencies – www.social researchmethods.net/…/AEAaccr.html). Both the

theoretical basis and practical application of pattern matching have been well-described

elsewhere (McLinden & Trochim, 1998; Trochim, 1985, 1989, 1990).

Figure 3: Example of Pattern Match

64
 The slope of the lines in Figure 3 illustrate that, for example, students and non students

were in agreement with the importance of field experience and in disagreement with items such

as program philosophy and curriculum philosophy.

Step 5: Interpretation of the Maps

Ideally, the next step in this procedure involves conducting interpretation sessions with

all participating stakeholder groups. Unfortunately, obtaining access to all four groups for a third

time proved impossible. As such, I was only able to conduct interpretation sessions with one

teacher group and one student group at each school. As well, not all of the original participants

could attend. Interpretation session participant totals were, teachers (6) and students (5). Each

session began with a brief recapitulation of the study to reorient and/or orient participants to

purpose. Next, the point cluster rating map and the pattern match were distributed to all

participants.

Due to time limitations, I had previously assigned cluster labels based on both focus

group discussions and my own reading of the maps. Respondents were asked to challenge the

cluster labels and offer new cluster names. The teacher group recommended that the cluster

“engagement as a feeling” sounded too similar to the cluster “emotions”, as such “engagement as

a feeling” was changed to “engagement as a habit of mind”. Their rationale was that

engagement was more “a frame of mind”, and “a state of being that could be determined and

often observed”. For their part, the student group felt that the cluster “education for students”

should be changed to “students at the centre”. Both name changes were subsequently approved

by the alternate group. A discussion ensued about the rating maps. Once explained, both groups

65
seemed pleased that the Pearson r at .8 indicated they were quite similar in their views. As one

student commented, “this is a really good sign, right?!”

In an attempt to reach more participants, copies of the interpretation session materials

along with a brief report were forwarded to each of the principals in hope that they would

distribute amongst staff and students (see Appendix B). Approximately, six months after the

distribution attempt, I posted the results of the concept mapping processes on a web site

(http:www.StephanieSutherland.com) to illicit further feedback and comments from participants

and other interested school members. The final step (Step 6) in Trochim‟s concept mapping

process concerns using the data for ongoing planning and improvement purposes. Principals and

various teachers at schools indicated that they would utilize the maps (specifically the cluster

rating maps) for their ongoing school improvement work. The next chapter will outline the

results of the concept mapping method. The outputs of the method will be presented in

increasing complexity and detail.

66
 CHAPTER FIVE: RESULTS

CONCEPT MAPPING OUTPUTS

The constructed maps are presented in order of increasing complexity and detail. As

described above, (see again, Table 8) represents the total sixty statement list previously

generated by students and teachers. In addition to generating the statements to describe their

perceptions of student engagement, students and teachers also rated the relative importance of

each item. Importance ratings allowed for pattern matching analyses to be performed between

the two groups. The results of these analyses illustrated in Table 9 are the average ratings and

bridging index scale. The bridging index scale is from 0.0 to 1.0. Lower values imply that a

statement is sorted primarily with statements that are close to it on the maps and therefore more

similar. It is useful to keep Table 9 on hand when examining the maps that follow.

67
Table 9: Student and Teacher Statements by Cluster Name

Cluster Name Statement Average Bridging

Rating Index *

3) Timely grading/assessment of student work 3.83 0.65

Aspects of pedagogy
10)Variety in teaching style 4.04 0.95

15)Small classes 3.65 0.20

28)Effective communication 4.30 0.07

31)High teacher expectations of students 3.70 0.47

50)Different assessment methods 3.65 0.20

52)Demanding curriculum 3.00 0.30

Cluster Average: 3.74 0.28

1) Guidance support 3.60 0.74

Professional
educators
8) Lots of program/course options 3.70 0.47

17)Hiring of younger teachers 2.91 0.39

20)Teachers‟ reputation 3.30 0.43

24)Teacher professional development 4.13 0.58

42)Teacher job satisfaction 3.83 0.65

58)Supportive principal/vice-principal 3.74 0.65

Cluster Average: 3.60 0.56

68
Variety in school 2) Parental involvement 3.52 1.00
policy/structure

14)Changes in regular timetable 2.78 0.64

22)Changes in administration 2.22 0.47

40)Different type of school involvement 4.04 0.95

Cluster Average: 3.14 0.77

4) Opportunities for student leadership 3.83 0.54

Beyond the classroom
13)Extra-curricular activities 4.00 0.50

16)Involvement in student council 3.04 0.52

18)School reputation 3.52 0.45

30)Involvement in sports 3.17 0.53

35)School spirit 4.09 0.38

39)Community involvement 3.52 0.75

41)School policies 3.52 0.42

Cluster Average: 3.59 0.51

21)Importance of friends/socializing 4.04 0.54

Diversity Belonging
25)School is accepting of different social groups 4.61 0.39

26)School is multi-cultural 3.83 0.42

32)School safety 4.43 0.43

33)Need for student space 3.52 0.44

Cluster Average: 4.09 0.44

Student-Teacher 5) Active role in classroom discussions 3.74 0.11

interactions

7) Teacher is passionate about teaching 4.65 0.21

9) Student independence in learning 4.00 0.13

69
 11)Relevance of class material 4.43 0.00

29)Interesting classes 4.17 0.04

37)Student ownership in learning 4.04 0.12

38)Balance of teacher & student ideas 3.52 0.06

54)Teacher is entertaining/interesting 4.00 0.06

56)Challenging class projects 3.74 0.09

Cluster Average: 4.03 0.09

12)Incentives to promote student learning 4.04 0.11

Students at the centre
27)Support for student ideas 4.00 0.09

43)Student input into learning 3.35 0.16

45)Having the same teacher/student more than once 3.17 0.06

55)Mixed ability classes 2.96 0.16

Cluster Average: 3.50 0.11

Engagement as a 36)Doing work in groups 3.35 0.22

habit of mind

47)Student attitudes/interest in classes 4.39 0.21

48)Having fun in class 3.91 0.21

49)Experience success in learning 4.30 0.07

51)Student motivation to learn 4.09 0.16

53)Opportunities for teacher-student decision-making 3.65 0.20

Cluster Average: 3.95 0.18

6) Teacher as friend 3.35 0.42

Emotions
19)Feel comfortable at school 4.70 0.56

23)Student-teacher relationships 4.26 0.48

70
 34)Compromise 3.65 0.50

44)Trust 4.48 0.34

46)Competition 2.91 0.52

57)Respect for others 4.74 0.34

59)Smiling 3.78 0.36

60)Being recognized 3.83 0.54

Cluster Average: 3.97 0.45

The first map that the concept mapping software generates is the Point Map (see Figure

4). The numbered point map illustrates the sixty statements (master list) as they were placed by

multidimensional scaling. Figure 4 illustrates that statements that were sorted together more

frequently by participants (students and teachers) are closer to each other on the map. For

example, looking at the bottom right corner of the map (see arrow) there are many statements

that have been sorted in a similar manner by participants. Specifically, statement numbers 5

“active role in classroom discussions”, 7 “teacher is passionate about teaching”, 9 “student

independence in learning”, 11 “relevance of class material”, 29 “interesting classes”, 37 “student

ownership in learning”, 38 “balance of teacher and student ideas”, 54 “teacher is

entertaining/interesting”, and 56 “challenging class projects” are located in close proximity.

In contrast, looking at the far left side of the map, there are statements such as 2 “parental

involvement”, 40 “different types of school involvement”, and 4 “opportunities for student

leadership” that remain quite isolated indicating that these statements were not sorted in a similar

manner by participants.

71
 Figure 4 Point Map

35 25
39 30 26
16 18 32
13 21
33 5759
4 41
19
40 46 44 60

8
22 6 34
2 14
20 23
1 24 17 48
43 55 36
58 12
52 53
42 27 45 51
28 15 49 47
1154
38
31 37 7 29 5
3 9 56
10 50

The concept mapping software also organizes the points into conceptual clusters as represented

by Figure 5. The nine-solution cluster map visually portrays the same clustering that appears on

the point map in Figure 4. Like the points on the point map, the smaller clusters contain

statements that are, from the participants‟ perspective, more conceptually similar while clusters

that are farther apart reflect conceptual difference. The closer the clusters are together on the

72
map, the more similar respondents felt the items to be. The clusters located at the bottom left

side of Figure 5 “Students at the centre”, “Engagement as a habit of mind”, and “Student-

Teacher interactions” are good illustrations of clusters that participants perceive to be similar.

The size of the cluster itself also indicates how conceptually similar or dissimilar the

individual statements were perceived to be by students and teachers. For example, larger more

elongated clusters (see for example, “professional educators”) indicate that both students and

teachers did not think that many of the items (i.e., #1 “guidance support”, #8 “lots of

program/course options, #17 “hiring of younger teachers, #20 “teachers‟ reputation”, #24

“teacher professional development”, #42 “teacher job satisfaction” and #58 “supportive

principal/vice-principal”) were conceptually similar. Conversely, the cluster labeled “diversity

and belonging” is relatively compact indicating that both students and teachers perceived the

items (i.e., #21 “importance of friends/socializing”, #25 “school is accepting of different social

groups”, #26 “school is multi-cultural”, #32 “school safety, and #33 “need for student space”) to

be similar.

73
 Figure 5 Conceptual Cluster Map

Beyond the classroom

Diversity/Belonging

Emotions

Variety in school policy/structure

Engagement as a habit of mind

Professional educators Students at the centre

Student-Teacher interactions
Aspects of pedagogy

The next set of maps integrates participant rating data into graphic outputs. The point-

rating map in Figure 6 illustrates the average item ratings by all respondents. The square „piles‟

beside each of the item numbers indicates average importance assigned to that item by

participants. Recall that statements were to be sorted from one (not very important) to five (very

important). The legend located on Figure 6 demonstrates that, at the low end, items rated less

than 2 are denoted by one box and items that were rated higher than four are denoted by five

stacked boxes. For example, items such as 57 “respect for others” and 19 “feel comfortable at

school” were perceived to be very important for student engagement by both students and

74
teachers. Conversely, items 22 “changes in administration” and 52 “demanding curriculum”

were not perceived by participants to be of central importance for student engagement.

Figure 6: Point Rating Map

Figure 7 displays the same data from Figure 6 in a two-dimensional visual cluster format.

Similar to the point-rating map, this graphic illustrates the average ratings by all respondents in a

cluster format. The legend on Figure 6 indicates that the lowest rated items (i.e., 3.14 to 3.33)

are denoted by a single „layer‟. Conversely, the highest rated items (i.e., 3.90 to 4.09) are

75
denoted with five „layers‟. The highest rated cluster by student and teacher groups was

“diversity/belonging” (cluster rating = 4.09), followed closely by “student-teacher interactions”

(cluster rating = 4.03). On the contrary, the lowest rated clusters were “variety in school

policy/structure” (cluster rating average = 3.14) and “students at the centre” (cluster rating

average = 3.50).

Figure 7 Cluster Rating Map

Diversity/Belonging

Beyond the classroom

35
30 2625
3916 18 32
13 21 Emotions
33
4 41 57
59
19
40
Variety in school policy/structure 46 44 60

8
2 22 6 34
14
20 23
Engagement as a habit of mind
Professional educators
1 24 17 Students at the centre
48
43 55 36
58 12
Aspects of pedagogy 52 53
42 27 45 51
Layer Value 15 49 47
28
1 3.14 to 3.33 38 11 29
2 3.33 to 3.52 31 37 754 5
3 3.52 to 3.71 3 9 56 Student -Teacher interactions
4 3.71 to 3.90 10 50
5 3.90 to 4.09

76
Figures 4 – 7 represent the foundation for students and teachers to build a shared foundation of

meaning making (Fullan; 2001; Senge, 2000). That is, both stakeholder groups had input into

defining what student engagement means to them. Throughout these processes, both students

and teachers (within their respective groups) gained a sharpened understanding of their own

orientation to student engagement by virtue of having to explain their thinking to others, and at

the same time being exposed to the views of others.

After the maps were produced, another meeting with participants was convened. During

this session, students and teachers were given a chance to examine the maps that they produced,

they were able to ask questions, and ultimately were asked to give each cluster a label (name).

This meeting is often called an “interpretation session” for it gives the participants an

opportunity to analyze and discuss their maps. Ideally, when conducting interpretation sessions

one would invite the respective stakeholder groups to meet. Unfortunately, due to time

constraints, only the students were able to undertake this broad naming task.

During the interpretation sessions, 2 broad “discourse regions” of similarity were

identified (see Figure 8). The clusters labeled “diversity/belonging”, “emotions”, “students at

the centre,” “engagement as a habit of mind”, and “student-teacher interactions” were referred to

by the students as “internal (or intrinsic) characteristics of student engagement”, clusters

“beyond the classroom”, “variety in school policy/structure,” “professional educators,” and

“aspects of pedagogy” were referred to by the students as “extrinsic characteristics of student

engagement”. Clusters labeled “beyond the classroom”, “variety in school policy/structure”,

„professional educators‟, and „aspects of pedagogy‟ were referred to as external (or extrinsic)

77
characteristic of student engagement. As well, the clusters in the former region are typically

larger and elongated indicating that they are less conceptually clear in the minds of students and

teachers than the clusters on the later region. Looking at the item and cluster ratings of the entire

map, what becomes apparent is that the items/clusters within the extrinsic characteristics

discourse region of Figure 8 are rated lower than those items/clusters within the intrinsic

characteristics discourse region. As well, the clusters in the extrinsic region are typically larger

and elongated, thus indicating in an effective visual that they were less conceptually clear in the

minds of students and teachers than the clusters in the intrinsic region.

“
Figure 8 Cluster Rating Map with Discourse Regions

Beyond the classroom Diversity/Belonging

Emotions

Variety in school policy/structure

Engagement as a habit of mind

Professional educators
Students at the centre
Aspects of pedagogy

Layer Value
1 3.14 to 3.33
2 3.33 to 3.52
3 3.52 to 3.71
4 3.71 to 3.90 Student-Teacher interactions
5 3.90 to 4.09

Extrinsic Characteristics Intrinsic Characteristics

78
 One of the pivotal questions arising in the context of this study is the extent to which

students‟ and teachers‟ views about student engagement converge. Do students‟ and teachers‟

perceptions of student engagement differ? If so, in what ways? The final graphic output is the

pattern match (see Figure 9) which represents a direct comparison of the student and teacher

group ratings of statement clusters. This pattern matching technique permitted the identification

of consensus and disagreement among the two stakeholder groups. Figure 9 shows fairly strong

agreement between students and teachers (r = .8) on the general importance ratings of the student

engagement items. The clusters that were considered to be most important for student

engagement by students and teachers were ones that related to the individual, or for intrinsic

purposes. This finding supports earlier work by Cousins and Sutherland (1998) who found

individual characteristics to be most important to students and teachers with regards to student

engagement.

With respect to student engagement, both stakeholder groups had similar clusters rated in

terms of importance. Students rated “diversity/belonging”, then “student-teacher interactions”,

followed by “emotions” as being very important for student engagement. Generally, teachers

were in close agreement with the students as to the most important rated clusters. Like the

students, teachers rated “diversity/belonging” as most important with regard to student

engagement”, followed by “engagement as a habit of mind”, and “student-teacher interactions.”

As well, students and teachers were similar in their views toward items that related to “beyond

the classroom”. These “beyond the classroom” items were rated as relatively important by both

groups, but the teacher group rating this cluster only slightly less important that did the students.

Another area of stakeholder convergence centered on the “variety in school policy/structure”

items. Teachers gave this cluster an average rating of 3.27, whereas the students gave it a 3.02.

79
Where there was divergence between the stakeholder groups, it was most evident within the

“aspects of pedagogy”, “students at the centre”, and “professional educators” clusters.

Figure 9: Teacher & Student Pattern Match

Teacher Student

Diversity/Belonging Diversity/Belonging
4.07 4.1
Engagement as a habit of mind Student-Teacher interactions

Student-Teacher interactions Emotions

Aspects of pedagogy Engagement as a habit of mind

Emotions
Professional educators

Students at the centre

Beyond the classroom
Beyond the classroom
Professional educators Aspects of pedagogy

Students at the centre

Variety in school policy/structure Variety in school policy/structure

3.27 3.02
r = .8

The remaining chapters will serve to integrate the findings of the study and offer some

implications for practice and policy. This will be accomplished with particular reference the

research questions guiding this work.

80
 CHAPTER SIX: DISCUSSION

This purpose of this study was to explore how students‟ and teachers‟ at two mid western

Canadian high schools engaged in a process of co-constructing a definition of student

engagement. As noted in preceding chapters, the Manitoba School Improvement Program

(MSIP) functioned as a „natural experiment‟ within which to investigate the research questions

posed in this study. Student engagement, as noted in chapter two is often held out as a key goal

in school improvement efforts. MSIP, being a school improvement organization with student

engagement as an espoused outcome, provided the context within which a participatory method

such as concept mapping could be explored.

This chapter serves to integrate the findings of the overall study. This is done with

reference to the research questions and methods employed. The remainder of this chapter is

organized into three major subsections. The first section addresses the first research question

raised in chapter one - how students and teachers at two Canadian high schools defined student

engagement. The second subsection provides commentary on the second research question

regarding the extent to which this study has been able to uncover areas of convergence and

divergence in students‟ and teachers‟ perceptions of student engagement. The final subsection

will address the third and final research question examining how student‟ and teachers‟

definitions of student engagement as uncovered here, compare to existing theory and research on

the nature of student engagement and the mediating factors at school.

81
Research Questions

RQ #1: How do teachers’ and students’ define student engagement?

Prior research has established the need for student engagement to be redefined in a

contextually relevant environment (Riviere, Sotomayor, West-Burns, Kugler, & McCready,

2008; Vibert & Shields, 2003; Willms, 2009). That is, an ongoing challenge for school

administrators and policy makers is to understand the nuances of student engagement in their

own context, a process that requires the active involvement of students and teachers who make

up that context in schools, and perhaps more importantly, in classrooms. In this study, the

concept mapping tool facilitated a process whereby the two primary stakeholders of student

engagement – students and teachers – participated in a process of definitional clarity. Concept

mapping provided a process whereby each individual developed a sense of personal insight or

vision with respect to student engagement, then after becoming aware of this mental

representation, they exposed it to the influence of others. Figures 4-7 represent the work of

students and teachers in building a shared foundation. Throughout these processes, students and

teachers (within their respective brainstorming groups) gained a sharpened understanding of their

own orientation to student engagement by virtue of having to explain their thinking to others, and

at the same time being exposed to the views of others. This capacity to „think together‟ is

promoted through the intentionally structured discourse. Such discourse opportunities allow for

the creation of a shared „picture of the future‟ that can foster genuine commitment by key

stakeholders, as participants in the process, developing ownership over the vision (Greene, 2008;

Mertens, 2009; Sutherland & Katz, 2005).

82
 The preliminary activity in co-constructing a definition of student engagement included

two groups of teachers and two groups of students to engage in separate brainstorming statement

generation sessions. Lively and animated discussion ensued during these sessions in both the

teacher and student groups. Students seemed particularly enthusiastic that their opinions were

being sought, and that the administration had deemed these sessions important enough so as to

permit them to miss their first class after lunch. The next task in the concept mapping process of

working towards a shared definition of student engagement involved reconvening these groups

so as to sort and rate the generated items. Upon receiving instruction on the purpose and

mechanics of the sorting and rating task both teacher and student groups had little difficulty

creating conceptual groupings of engagement items and subsequently assigning rating values.

The final activity involving teachers and students required participants to take part in

interpretation sessions. Throughout these processes, the teachers and students involved were

continuously engaged in an open dialogue whereby they shared their own interpretations and

began to come together as a group in working towards a shared picture of what student

engagement meant to them.

The final task for the participants involved interpretation sessions whereby the concept

mapping outputs (maps) are reviewed, discussed, and assigned cluster names. Ideally, when

conducting an interpretation session, the facilitator would invite the respective stakeholder

groups (teachers and students) to meet together. Unfortunately, due to time constraints, only the

two student groups were able to undertake this naming task. Teachers, however, when presented

(at a later meeting) with named maps agreed and endorsed the student cluster names.

Next, and perhaps the pivotal question arising in the context of this study, is the extent of

convergence and divergence between students and teachers of the co-constructed definition of

83
student engagement. Specifically, the pattern matching analysis and corresponding graphical

representation illustrated in Figure 9, identifies the issues that are most amenable to action or

further exploration.

RQ #2: What are the key dimensions of convergence and divergence between students’

and teachers’ definitions of student engagement?

The concept mapping software provided for the direct comparison of stakeholder

convergence and divergence by virtue of a pattern match (see again, Figure 9). As previously

described in chapter 3, the software calculates an overall correlation coefficient, then proceeds to

break down, cluster by cluster, the level of stakeholder convergence and divergence. The

brainstorming group sessions served to provide descriptive detail to the key dimensions.

Overall, the students and teachers in this study had a fairly strong agreement on the general

importance ratings of the student engagement items (r = .8). Generally, the findings here are

consistent with prior research but perhaps what is new and useful is the ability to simultaneously

examine areas of convergences, and more importantly divergence in student and teacher

definitions of student engagement.

Key Dimensions of Convergence

In this study students and teachers most strongly converged on three clusters

“diversity/belonging‟, „student-teacher interactions‟, and „variety in school policy/structure‟ as

evidenced by the lack of slope or little slope of the lines in Figure 9. I discuss each of these areas

of convergence in turn.

84
 Convergence Area 1: „Diversity/Belonging‟

The convergence observed in the area of „diversity/belonging‟ supports earlier findings

by Willms et al., (2009) who found by asking students directly that “a sense of belonging”

ranked among the most important variables with respect to students‟ feelings of engagement with

school. This is also consistent with the extensive body of work that finds students‟ sense of

belonging to the school community is critical for their engagement with school (National

Research Council Institute of Medicine, 2004; Osterman, 2000; Solomon et al., 1996; Willms,

2000; 2003).

Interestingly, the nature of the concept mapping tool provided for a forward and

backward analysis of the data (Greene, 2008). Recall, for example, that students and teachers

were asked to generate a list of student engagement statements – separately – within

brainstorming groups. Analysis of the brainstorming sessions (see Tables 3 – 6) revealed that the

items relating to diversity and social awareness came from the student groups only. It was not

until after they were presented to teachers during the sorting and rating exercise that they were

rated highly. One student spoke of diversity in this way:

This is a very multi-cultural school, like if you‟re white you‟re almost a

minority and that‟s awesome because I can learn more if I‟m in a
classroom that has Filipino, Japanese, Chinese and people that are of
African and Native decent. And, there can be people from Europe and it‟s
really cool because you get to understand different traditions.
And

Overall, our school is very accepting…like everybody feels like they

85
 belong. If there are fights it‟s usually, like, boyfriend/girlfriend stuff.
It‟s almost never, like, I‟m white and you are black and I want to beat
you up. It‟s more about who cheated on so and so…

These findings support the broad base of prior work examining students‟ ethnicity and their

engagement in school (for example, Coleman, 1996; Davidson, 1996; Epstein, 2001; Willms,

2000). The qualitative descriptions of the positive associations between students‟ perceptions of

belonging and the importance of the school community support earlier research linking the need

to develop schools as communities (Baumeister & Leary, 1995; Furman, 1998; Osterman, 2000,

Solomon et al., 1996). One student equated her sense of belonging to a powerful school

community:

I like people at this school. Compared to other schools and the people I
know there, the people in our school are probably the best. Physically,
our school looks like crap. Really, it is just horrible and it‟s falling apart
but the community here is probably the strongest that I know about.

Consistent with the literature on the importance of peers with respect to student

engagement (Bauermeister & Leary, 1995; Fredricks et al., 2004; Goodnenow, 1993; Ryan,

2000), many students explicitly linked their friendships and socialization in terms of their sense

of belonging at school. One student described the impact her friendships had with respect to

learning this way:

If you get in a fight with your friends it affects everything…all day. Like
if me and [Laura] got into a big fight…seriously…so if I went to World
Issues class and her and [Steven] were mad at me…I would not learn. I‟d

86
 just shut down and it would ruin everything. Relationships are such a big
thing to us.

The links between peer acceptance and behavior in classrooms has been well established within

the literature (see for example, Osterman, 2000; Ryan, 2000; Solomon, Battistich, Watson,

Schaps & Lewis, 2000; Wentzel & Asher, 1995; Wentzel & Caldwell, 1997). That being said,

however, there is little research that provides a deep understanding of the nature and quality of

peer relationships within the schools. What little we do know comes in scattered pieces of

information gleaned from a variety of sources (Osterman, 2000). The present study contributes

to our understanding of the nuances of the relationship between student engagement, sense of

belonging and peer relationships. It appears, albeit from this small sample, students most

strongly associate engagement to their sense of belonging at school. In turn, this sense of

belonging is directly impacted by their relationships with peers. It is as though students

negotiate a multitude of critical incidents (e.g., conflicts/issues with peers and friends) as they

navigate through their school day with respect to their peers. Students reported that their

relationships and interactions with peers directly affected the degree of motivation they had for

school work.

Teachers also described the sentiment that the school fosters a sense of connection and

belonging for the students. Teachers understood that the school itself was often viewed as a safe

place for students. One teacher put it like this:

87
 We talk about students who feel extremely safe here. I spoke with one
student today who told me that this is the very best place he has ever been
but he still struggles to make it to class. Probably his wellness and how
he feels about himself is a priority for him and that‟s just fine with me.

The implication for student engagement research is the recognition that in order to capture the

complexities and nuances embedded in this construct, engagement should be studied as a meta

construct (Fredricks et al, 2004). That is, it often becomes difficult to make distinctions between

the nuances of the behavioural, emotional, and/or cognitive dimensions of student engagement.

Convergence Area 2: „Student-teacher interaction‟

Another important dimension of student-teacher convergence focused on student-teacher

interactions. Much of the discussion in the brainstorming sessions, particularly from students,

supported Woods (1996) findings in linking the degree of learning support and degree of

mutually shared understandings between teachers and students in the classroom as critical in

setting the stage for learning to occur within classrooms. The following exchange is illustrative

of the varying perspectives teaches held about „negotiative discussion‟ (Woods, 1996) within the

classroom:

Researcher: Do you make visible your expectations for students? If so,

Do they ever challenge you?

Teacher 1: Sometimes they challenge me

Teacher 2: No, not often, but sometimes

Teacher 3: I think…I think for the most part we do what we do

88
 because that‟s what we think is in their best interest

even though they [students] may not think so. And, so

we just smile and tell them „no‟.

Teacher 4: [Laugher]…Yeah… they‟ll thank you later.

Teacher 1: Actually, I have to say that I enjoy when it happens, you

know, when they ask about my expectations and I like to

talk about it with them and make them feel as though they

have a right to ask.

This quote is illustrative of the range of teacher responses in creating an atmosphere whereby

students and teachers can interact in mutually beneficial ways (National Research Council

Institute of Medicine, 2004; Gilbert, 2005; Scardamalia & Bereiter, 2001, 2003). Perhaps most

importantly, this quote speaks to the necessity that teachers need to be open to, and willing to

listen to what students have to say (Cook-Sather, 2002; Rudduck, 2007), and to be prepared to

modify pedagogical approaches so as to include the voices and opinions of students so as to

jointly author student learning (Fielding, 2001). Unfortunately, cross sectional studies such as

the present one offer little insight into the longer term impact of realizing student voice within

classrooms. Future research should include a longitudinal dimension to follow classrooms so as

to document the degree of movement along the student voice continuum, with a goal of having

every teacher make some movement in terms of nurturing student voice within their classrooms.

This notion of student voice permeated the discussion around the „student-teacher

interactions‟ cluster. Many students in both statement generation groups discussed how having a

89
„say‟ in their education was important to them, and that they wished teachers would listen to

them more”

If I could change something about this school I think it would be that we

could maybe be heard a bit more. You know, teachers blame us. Really,
they say that we don‟t voice our opinions and that we are not involved
whatsoever, but really, if they would listen to us they‟d probably find out
a lot of us do have something meaningful to say. I think we need more
guidance on how to do that. And, teachers should reach out a little bit too
if they really want to know what we are thinking.

Convergence Area 3: „Variety in School Policy/Structure‟

Finally, students and teachers strongly converged on the idea that the cluster they called

„variety in school policy/structure‟ held little relative importance to student engagement in their

schools (Fielding, 2004). Recall from the context section that both schools had altered schedules

on Wednesdays. According to the principals at each school, the intent was to promote student

choice and increased flexibility in the students‟ schedules. When asked what the students

thought of this policy, students at School A were in unanimous agreement when they called it

“stupid.” Students at School B mentioned that they felt these “special” days were “not well

organized” and “the administration hasn‟t been able to figure out how to make it work.” For

their part, teachers at both schools felt the change in schedules was either a “waste of time” or

was viewed as “a pet project of the principal.” This is consistent with a study of school reform

by Datnow, Borman and Stringfield (2000) that found that school-based reform implementation

faltered when there had not been careful attention to a school‟s climate and the program‟s

90
specific fit within that schools reform agenda. Scholarship in evaluation has provided a solid

knowledge base that without participation in an initiative, there will be little chance of buy-in,

and thus long term sustainability (see, for example, Cousins & Earl, 1995; Cousins & Whitmore,

1998).

In sum, in this study I observed the most significant convergence between teachers‟ and

students‟ perceptions of engagement in the areas of „diversity/belonging‟, „student-teacher

interactions‟, and „variety in school policy/structure‟. Taken together, these three areas of

convergence suggest that students and teachers are most in sync with the emotional, and to a

lesser degree the cognitive dimensions of student engagement. Counterbalancing these areas of

convergence, however, are several points of divergence.

Key Dimensions of Divergence

Teachers and students most strongly diverged on three clusters, „aspects of pedagogy‟, students

at the centre‟, and „professional educators‟ as reflected in the steep slope of the lines (see again, Figure

9). The concept mapping outputs facilitates analysis in several ways. Perhaps the easiest determination

of a cluster that is conceptually unclear is by looking at its‟ shape and layer value. The Cluster Rating

Map (see again, Table 7), illustrates that the „professional educators‟ cluster is large and elongated,

displaying relatively lower rating values. That is, participants, on average, rated these items not to be of

high importance. The other concept mapping output that is useful to examine is the slope of the lines in

Figure 9. This information should be particularly useful to school administrators, faculty and students in

terms of ongoing school-based improvement efforts. Recall from chapter three, that the greater the

slope of the line in Figure 9 is indicative of the greater the amount of convergence on a given cluster by

students and teachers. Thus, the clusters with the greatest slope may be an ideal place for ongoing

91
planning and improvement discussions to begin to bring the views of teachers and students more in line

with each other.

Divergence Area 1: „Aspects of Pedagogy‟

For their part, teachers rated „aspects of pedagogy‟ higher than did the students. In their

brainstorming sessions teachers spoke a lot about the importance of a challenging curriculum, and the

need to set high expectations for students. More recently this thinking, though, can have detrimental

consequences for student engagement, particularly among lower achieving students when not

accompanied by high levels of support (see, for example, Galloway, Pope & Osberg, 2007; Pope, 2001;

Willms, 2000). This body of research illustrates that high academic expectations can lead to increased

student stress, and ultimate disengagement (Luthar & Latendress, 2005; Luthar & Becker, 2003). Thus,

teachers who overemphasize aspects of pedagogy, particularly to low achievers, may actually cause

these students to further disengage from schooling. Indeed, the divergence in thinking between students

and teachers concerning various aspects of pedagogy is worthy of ongoing and deeper inquiry.

Another area of dissonance within the „aspects of pedagogy‟ cluster revolved around the issue of

student assessment. The concept mapping method permits the researcher to trace the discussion of

assessment (cluster phase) back to the brainstorming sessions which enables the analyst to determine the

source of dissonance. For example, student assessment was given a lot of attention only in the student

sessions. Students brought up a wide range of issues related to assessment. The following interchange

92
between two students provides details to the statement „grading/assessment of student work‟ found

within the „aspects of pedagogy‟ cluster:

Student 1: With my science teacher, if you give him a short little paragraph,

it will take him about four or five weeks to mark it.

Student 2: Yah, and in the meantime you have other stuff to do, other stuff

to worry about but you don‟t really know if you‟re doing the stuff

before right or not…

Student 3: I get really excited to get a mark back because then I am better able

to focus my studying so I can do better on my next test or

assignment.

This quote is illustrative of the recommendations by the National Research Council Institute of Medicine

(2004) in fostering high school students‟ motivation to learn that teachers should make a practice of

routinely providing assessment feedback to students so as to promote behavioural, cognitive, and

emotional engagement.

Divergence Area 2: „Students at the Centre‟

The concept mapping method permits the backward mapping of data analysis so as to „drill

down‟ to specific discussions of themes. This can be particularly useful in ongoing school improvement

planning as related to student engagement so as to provide concrete examples for ongoing planning and

discussions. The „students at the centre‟ cluster was rated significantly higher by students than by

teachers. It appears that teachers viewed this cluster as less important in terms of student engagement

within their respective schools. This finding is important, particularly for school administrators, as both

schools allocated resources to various MSIP sponsored “students at the centre” initiatives. As

93
previously mentioned, the students at the centre initiative was formed as part of a larger effort aimed at

exploring the potential of student voice in supporting positive change in high schools (Pekrul & Levin,

2007). Although the benefits of having students play a central role in school reform are numerous and

have been well documented elsewhere, the fact is that many student focused initiatives often fail to

achieve a substantial degree of spread within school-based reform efforts (Coburn, 2003; Pekrul &

Levin, 2007).

In the present study it could be that teachers attending the brainstorming sessions had little or no

knowledge of these types of activities being offered to students. Or, perhaps the organizers of the

MSIPs did not adequately inform and/or involve teachers in the student-based initiative.

Undeniably, the placing „students at the centre‟ of education runs the risk of not being inclusive

to teachers. And, as Rudduck et al (2007) points out, teachers are the gatekeepers of authentic student

voice in schools. If teachers are not given the information and tools to: 1) understand the value student

voice (Cook-Sather, 2002; Levin, 2000; McIntyre & Pedder, 2004; McIntyre et. a., 2005), 2) nurture an

environment whereby student voice can grow (Fielding, 2001; Osterman, 2000), and 3) sustain a culture

of student voice (Fielding, 2001; 2004), the chances for authentic student voice permeating a school

culture are narrow at best.

Finally, the cluster average itself (3.50) is indicative that this cluster was not deemed as

important – overall – in comparison to other clusters by both students and teachers. Analysis of the

within cluster items shows that the average per item ratings vary from 2.96 (mixed ability classes) to

4.04 (incentives to promote student learning). Two issues are noteworthy here. First, the title of the

cluster may be misleading. That is, the items do not necessarily relate to students being at the centre of

learning, nor do they directly compare to the MSIP‟s Students at the Centre‟s goals. Second, this cluster

94
may have performed better statistically if a different factor configuration were selected. Recall from

chapter three that the analyst must decide the cluster solution. This decision is not performed on a per

cluster basis but rather on how all of the clusters spread on the map. This is precisely the kind of cluster

that would be beneficial for use in ongoing school improvement planning efforts, efforts that attempted

to involve more stakeholders (in this case students and teachers) from the school community.

Divergence Area 3: „Professional Educators‟

The third significant area of divergence in student and teacher thinking is found in the

„professional educators‟ cluster. Figure 9 shows that students rated teacher characteristics as an

important aspect of their feelings of engagement in school. During the brainstorming sessions, students

spoke of their teachers‟ personal attributes such as: involving students in class-based decisions/planning,

expectations of students, and the importance of teacher professional development. The following

student exchanged is an example of how students perceive involvement in the classroom:

Student 1: I relate engagement to involvement. Like, class involvement

means having a say in what kind of work you do.

Student 2: Yah, even if you have a really bad course or something that is

really hard like chemistry or physics…if you have a really cool

teacher it just makes the class more fun…someplace you want to

be.

Student 3: Uh huh. It‟s almost like you are teaching yourself but they

[teachers] are there to just open that door for you. You know?

I know that they have a hard job because they try to make their

teaching style exciting for so many different learning styles.

95
Other students spoke of student engagement as co-constructing and collaborating on classroom

activities and routines in this way:

Our English teacher is very open. He‟ll give us a basic idea of

what to do but it‟s really your own ideas that you need to come
up with. Also, it‟s a relaxed atmosphere in that class and you feel
more at home going there. He really meets the students half way.

The feeling of „meeting half way‟ was elaborated on by another student who stated:

A lot of teachers expect us to make school our first priority but

a lot of them don‟t make it their first priority. I see lots of them
leaving the parking lot at 3:30pm

For their part, teachers did not speak of professional development or personal satisfaction from teaching,

but mainly discussed their strategies to engage students in the classroom. Some teachers noted that it

was their job to “move students in and out of their comfort zones” and “hooking a few students will

allow you to bring in seven more.” Another teacher described his personal „student engagement‟

strategies this way:

Good teachers integrate the everyday world into a microcosm in

their classrooms, but for many teachers to step outside of their

96
 door and see the totality of improving the school is completely
foreign to them.

What is clear from this discussion of the „professional educators‟ cluster is that students value the role

teachers play in their sense of engagement to school, and it is absolutely necessary for teachers to listen

to what students have to say about their feelings of engagement (Cook-Sather, 2002; Cook-Sather, 2007;

Evans, 2002; Rudduck et al., 1996; Rudduck, 2007). Yet, to the extent that the student comments

relayed above are indicative of a broader sentiment among students, they suggest that involving students

in class decisions and planning, conveying expectations, and demonstrating that school is a priority for

teachers as well, are central to student engagement.

Teachers need to be the primary link between the classroom-level and the school-level. One

teacher noted that “we need to stop taking a linear approach to school reform, an approach that goes

school – classroom – teacher – students to one that looks more circular. We can do this by creating the

structures for dialogue.” This finding is strongly supported in the student engagement engagement

literature (Cook-Sather, 2002; 2007; Rivière et. al., 2008; Rudduck, 2007). The participatory nature of

the concept mapping design is one solution in overcoming this problem in more traditional research

designs. For example, the method itself is participatory and as such permits the examination of more

than one stakeholder simultaneously. Also, the concept mapping approach provides graphic displays of

the degrees of similarity and differences in stakeholder thinking. Moreover, by taking part in this

research, student and teachers were given the time and space to grapple with issues such as concept

mapping in a structured and systematic way.

97
RQ #3: How do student and teacher definitions of student engagement compare to
existing theory and research on the nature of student engagement and the
school-level factors influencing student engagement?

Within the engagement literature, three types of approaches to student engagement can be

identified: behavioural, emotional, and cognitive. Many studies of engagement include one or

two of these approaches but rarely all three (Rivière et. al., 2008). Fredricks et al., (2004)

suggest that to date, research has not capitalized on the potential of engagement as a

multidimensional construct. The findings from this study support the multidimensional nature of

student engagement. For example, students viewed their engagement in school (and classrooms)

to their sense of belonging in school. That being said, however, engagement as tied to a sense of

belonging at school was hinged in a large part to students‟ peer relationships. Uncovering yet

another layer to the complexity of engagement, students viewed their peer relationships to have a

direct impact on their levels of motivation and academic achievement in school. Because there

has been considerable research on how students behave, feel and think, attempts to conceptualize

and examine portions of the literature under various „engagement‟ labels can be potentially

problematic by resulting in a proliferation of different constructs, definitions and measures that

differ slightly, thereby doing little to improve conceptual clarity (Fredricks at al., 2004; Rivière

et. al., 2008).

Participatory methods such as concept mapping provide a tool for examining student

engagement in a multidimensional manner. In particular, the qualitative nature of the

brainstorming sessions permits a discussion that can accommodate the breadth and depth

necessary for this type of construct. For example, students spoke passionately about the

importance of friendships and peers as fostering a sense of belonging to school, yet this

98
discussion also touched on issues such as school climate, student-teacher interactions and student

voice.

Another relevant example of the difficulties in attempting to separate the dimensions of

student engagement can be found in the survey instruments used for the overall evaluation of

MSIP. The larger MSIP project, from which this study was spurred, began with the premise that

student engagement is a precursor to learning and as such the evaluation work poised the student

engagement construct along two broad dimensions: 1) students‟ relationship with the learning

environment (school atmosphere/climate, student voice in decision-making on school directions,

student participation in school activities, and student relations with teachers) and, 2) students‟

relationship with their own learning (motivation to learn, confidence in their own ability to

succeed, relevance of courses/curriculum, interest in courses/curriculum) (Earl et al., 2003).

This approach to measuring student engagement was broad in scope with regards to the

scales embedded within the student engagement survey. The problem with this type of general

scaling, however, is one of clarity. That is, the practice of combining items into general scales

precludes examining distinctions among the types of engagement (Fredricks et al. 2004). In

addition, conceptual distinctions are blurred because similar items are used to assess different

types of engagement. For example, questions about motivation and ability are included as

indicators of both behavioural engagement (Finn et al. 1995) and cognitive engagement (Connel

& Wellborn, 1991; Newmann, Wehlage & Lamborn, 1992). In an attempt to overcome the

problems with a lack of clarity in defining student engagement, this study employed a

participatory approach that permitted treating student engagement as a multidimensional

construct (Fredricks et al. 2004; Guthrie & Wigfield, 2000; Rivière et. al., 2008). The

qualitative dimension of this study (i.e., brainstorming focus groups) permitted for discussion

99
that could serve as a starting point to begin to make distinctions between the degrees of

behavioural, emotional, and/or cognitive dimensions of engagement. By doing so, this study

builds on and extends prior research on student engagement.

The findings from this study directly supported Bryk and Schneider‟s (2002) findings that

building a climate of trust is particularly important in secondary schools because adolescents

need a supportive and caring environment. Students and teachers strongly converged on the

importance of student-teacher interactions as very important to student engagement. Similarly,

teachers and students were similar in their orientations to the importance of student voice as

related to student engagement in high school. Prior research on student voice separated findings

into four clusters: autonomy, pedagogy, social and institutional (McIntyre & Pedder, 2004;

McIntyre, Pedder & Rudduck, 2005; Rudduck, 2007; Rudduck & Flutter, 2004; Shultz & Cook-

Sather, 2007). Interestingly, items from the master brainstorming list (see again Table 8) can be

inserted into one of the four categories as related to how students and teachers view the issue of

student voice. Table 10 is not meant to be a complete exercise in mapping all 60 concept

mapping brainstormed items into a single table, but rather provides an example of how study

findings map onto the four clusters of student voice as represented in the literature.

100
Table 10: Mapping of Concept Mapping Items to Student Voice Literature

Student Voice Cluster Concept Mapping Items

Autonomy Cluster 9) Student independence in learning

Students being able to make choices and 27) Support for student ideas
decisions about their work
37) Student ownership in learning

43) Student input into learning

Pedagogical Cluster 11) Relevance of class material

Learning with clear expectations and that 31) High teacher expectations of students
is connected to daily lives

Social Cluster 6) Teacher as friend

Collaborative work and being respected by 23) Student-teacher relationships

teachers and peers
34) Compromise

38) Balance of teacher & student ideas

44) Trust

53) Opportunities for teacher-student

decision making

57) Respect for others

Institutional Cluster 14) Changes in regular timetable

Understanding of school-based procedures 22) Changes in administration

and policies
41) School policies

101
 This table is useful in explaining several features of study findings as they relate to the

literature. The table is a good example as to how student and teacher generated items can fit into

pre-existing clusters of student voice as related to promoting student engagement. Probing

deeper, the table is useful in highlighting the multidisciplinary nature of the student engagement

construct. For example, the concept mapping items generated by students and teachers were

often given different labels within different categories. That is, the social cluster in Table 10

contains items that were separated into cluster names such as “emotions”, “student-teacher

interactions”, and “engagement as a habit of mind.” And, the concept mapping approach

facilitates probing the data further to make determinations as to which stakeholder group

initiated an agenda item, and in turn, analyze the degree of support given by both groups. For

example, the notion of diversity/belonging was rated by both groups as the most important aspect

of student engagement. Yet, by tracing back the origin of this construct one finds it was initiated

only by the student group, then strongly supported by the teacher group. Without further study,

one can only postulate as to why only the student groups initiated this idea. Perhaps it was due

to the fact the both schools were considerably multi-cultural in their student population but the

teaching staff was entirely comprised of white, middle class staff members. The strong

divergence on the belonging/diversity cluster is well documented in the literature as important to

nurturing and sustaining student engagement (CEA, 2006; Willms et al., 2009; Willms, 2000,

2003; National Research Council Institute of Medicine, 2004; Osterman, 2000; Solomon,

Watson, Battistich, Schaps & Delucchi, 1996). What is different in the present study is the

finding that it was the student stakeholder group that put the item on the student engagement

agenda in the first place.

102
 Overall, this study supported earlier work that speaks to the complexity of the student

engagement construct (Cook-Sather, 2007; Cousins & Sutherland, 1998; Fredricks et al. 2004;

Riviere et al. 2008; Smith et al. 1998; Willms et al., 2009). In particular, a multidimensional

construct such as student engagement required a multidimensional methodological approach so

as to include the voices of key stakeholders (students and teachers) in a participatory process. In

this study, concept mapping provided a window to show how students and teachers articulate

their perceptions about student engagement in a participatory approach, thereby permitting the

investigator to „watch‟ the unfolding of this process in sequential stages. Perhaps the greatest

benefit to be derived from using the concept mapping approach was that it permitted student and

teachers to simultaneously co-create a definition of student engagement.

In addition to advancing our understanding of student engagement, another contribution

of this study is to illustrate the value of using the concept mapping methodology to study student

engagement. Participatory approaches strive to provide a shared knowledge base through direct

involvement of stakeholders in the research process. Stakeholders build on collaborative insights

in order to promote change within an organization, and thus the people affected by the change

must be involved in creating that change. Participation and collaboration have been found to

produce long-term commitment by building a culture of learning among those involved

(Trochim, Marcus, Masse, Moser & Weld, 2008; Cousins & Earl, 1992; Cousins & Leithwood,

1986; Whitmore & Cousins, 1997). Conversely, several of the study findings that were either

not reviewed or supported by the literature review are discussed below.

103
 Students and teachers at the sample high schools were in strong agreement that issues

concerning school policy and structure had little to do with feelings of student engagement.

Perhaps this was due to the fact that there appeared to be little school wide buy-in for reforms

initiated by the principals in both schools. For example, both schools were experimenting with

altered schedules so as to permit students more flexibility in their daily schedule. One could

postulate that attempts to alter the structure and/or policies as the school level were unsuccessful

because they did not achieve buy in from a large segment of the school population (both students

and teachers). Or, the lack of interest in this issue could be due to the fact that students and

teachers felt student engagement to be more of a classroom level issue. Nonetheless, there is a

broad and interesting literature that discussed the organization of the school in terms of

promoting student engagement (Darling-Hammond, Ancess & Ort, 2002; McLaughlin &

Talbert, 2001; National Research Council Institute of Medicine, 2004). This research illustrates

the powerful way in which school structures can either facilitate or inhibit student engagement.

Thus, future studies of student engagement need take the school organization literature and use it

to better inform data collection activities so as to better understand from the views of students

and teachers the importance of this area as related (or not) to student engagement.

Students spoke of classroom assessment as related to their emotional and behavioural

sense of engagement in school work (Eccles, Alder, Futterman, Goff, Kaczala & Meece, 1983).

That is, students mentioned that frequent and consistent feedback from teachers served as a

motivator for continued cognitive effort. In their review, Hattie and Timperley (2007) note that

feedback is one of the most powerful influences on learning and achievement but evidence

shows that the type of feedback and the way it is given can be differentially effective. That is,

104
feedback has not effect in a vacuum: to be powerful in its effect, there must be a learning context

to which feedback is addressed (Hattie & Timperley, 2007). The authors stress that the climate

of the classroom is critical, particularly if disconfirmation and corrective feedback at any level is

to be welcomed and used by the students (Hattie & Timperley, 2007). Moreover, research

confirms the finding that feedback on learning occurs too rarely, and needs to be used in

effective and constructive ways (Hattie & Timperley, 2007; National Research Council Institute

of Medicine, 2004). Teachers should continually monitor the effectiveness of the curriculum and

instructional practices so as to make determinations as to whether students are staying engaged

behaviourally (e.g., attendance, completion of work), cognitively (e.g., efforts to understand and

apply new concepts), and emotionally (e.g., enthusiasm for learning activities) (Timperley,

2003).

105
 CHAPTER SEVEN: CONCLUSIONS

This study employed a participatory approach – concept mapping – for the purpose of

illustrating how students and teachers could collaborate on a definition of student engagement.

This capacity to „think together‟ was promoted through an intentionally structured practice of

discourse. Thus, as students and teachers developed shared understandings it was postulated that

they would foster increased commitment to the results and subsequent school improvement

planning efforts. As a direct result of this approach, we were able to probe more deeply into the

complex student-teacher relationship in an effort to reveal key characteristics of student

engagement. In this way, students could take a genuine and authentic role as co-collaborators in

the process. For example, this is evidenced by student initiated items such as diversity and

belonging, being taken up and embraced by all stakeholders.

The impact of the interpretation sessions (both after the maps were constructed and at

future times so as to include a wider stakeholder group) highlights the contribution of the

concept mapping approach as contributing to ongoing school improvement dialogue and

planning. In other words, after the research and analysis processes are complete, the resultant

maps can act as the „overall plan‟ for ongoing student engagement work. The cluster maps, for

example, can act as strategic goal areas with specific tasks assigned to it. Teams or individuals

can be assigned to specific tasks to facilitate ongoing school-based improvement-oriented action.

The graphic representation of convergence and divergence in stakeholder perceptions is useful

for ongoing planning and school improvement work. In particular, the areas of divergence could

be immediately taken up and assigned to a sub-committee for further attention and exploration.

106
 This study adds to the student voice literature that speaks specifically to the necessity for

teachers to listen to students (McIntyre, Pedder & Rudduck, 2005; Pekrul & Levin, 2007;

Rudduck, 2007). In her own work, Rudduck (2007) concluded that when student voice was

thoughtfully introduced, with respect for its fundamental principles, there can be considerable

benefits for schools, for teachers, and for students (see also, McIntyre et al, 2005; Osterman,

2000; MacBeath, Demetriou, Rudduck & Myers, 2003). This process takes time, not only to

create organizational structures but also to nurture a sympathetic professional culture to sustain it

(Fielding, 2001). This new paradigm is challenging in that it pushes hard against current

traditions and practices currently at play in high schools. Another key area linking students and

teachers can be found with regard to the hidden/informal curriculum. Prior work and experience

has shown that students have a more difficult time navigating the informal curriculum (i.e.,

relationships with teachers and peers) than the mandated one (Rivière et al., 2008). This finding

has direct implications for the importance of belonging/diversity and friendships as uncovered in

this study.

Prior research has found a strong link between students‟ sense of belonging and academic

achievement (see for example, Willms, 2003, Osterman, 2000; Rivière et al. 2008). Many

students who have a low sense of belonging tend to cluster into two groups, one that has

relatively high academic achievement, and another that has exhibits performance indicators.

This study suggests that one approach to addressing the issue would be to target this second

group and possibly use the concept mapping tool to assist in uncovering deeper sources of

disengagement, and areas of perceived lack of support. For instance, this study showed that

teachers do not always have an accurate and complete understanding of how students view their

107
role as professional educators (in terms of involving students in class decisions and planning,

convey expectations, and demonstrating that school is a priority for teachers too). Thus,

educators may need more specifics about each group of students and what different factors are

related to engagement.

108
 CHAPTER EIGHT: IMPLICATIONS FOR PRACTICE AND RESEARCH

This final chapter will serve two functions. First, commentary will be provided on

limitations of this study. The second section will address some implications for practice and

further research in this area.

Limitations

Several limitations that apply specifically to the concept mapping approach, and more

generally to the study itself will be discussed. The concept mapping method presented several

challenges in terms of organizing and coordinating follow up sessions with study participants.

That is, it would have been ideal to hold the interpretation sessions in a whole-group (i.e., both

students and teachers) setting. Unfortunately, time and logistical constraints did not permit this

aspect of the „group think‟ process to occur. To overcome barriers of time and location, the

developers of Concept Systems now have a web-based version. Other possibilities to host

brainstorming type focus groups would be to hold virtual meetings with the use of current

commercially available software.

Aside from the logistical constraints of the concept mapping method, there are several

programmatic challenges associated with this tool. The determination of a „cluster solution‟

currently resides with the analyst. Revisions to the method could include a participatory element

whereby stakeholders could have input into the cluster decisions. Critics of the concept mapping

approach have claimed that the rating task can fall prey to a „group think‟ situation. This can

occur with any sit down survey/test administration and it would be the facilitators‟ role to ensure

individuals work independently. The study itself was conducted with a small number of

participants, and as such is open to problems of generalizability. As well, data collected herein

109
represent a cross section in time and as such this study represents but a snap shot of student

engagement in one place and time. As mentioned previously, future research should include a

longitudinal dimension to follow classrooms so as to document the degree of movement (if any)

teachers make with respect to student voice. A goal would be to have every teacher make some

movement in terms of accepting and/or nurturing student voice within their classrooms.

In addition to the methods and software limitations previously mentioned, there are a

number of conceptual limitations to this work that should be noted. This study focused on

achieving some degree of conceptual clarity from two key stakeholder groups (students and

teachers) on the construct of student engagement. That being said, however, the study had to be

bounded in terms of a conceptual frame and associated literature review so as to be manageable

and focused. As such, the study had to be narrowed to discuss some relevant literature while not

including others. Literature that was not intentionally a focus of this work but that is informative

to our understanding of student engagement includes the work on student disengagement, power,

pedagogy, and the broader work in organizational behavior that includes school organization and

structure. Concerning the limitations discussed and the results of this study, several areas of

future research can be identified.

This final section is subdivided into two parts. The fist will detail implications for

school-level practice in terms of ongoing planning and school improvement work. The second

section will provide insight into possible future directions research could take in the area of

student engagement. In addressing the former point, three maps have been identified as assisting

in ongoing planning work. Figures 10 though 12 below provide examples of how school

administration, teachers and students can collaborate on their school engagement agendas.

Figure 10 illustrates how school administration and/or school improvement sub committees

110
could use the concept mapping outputs for ongoing discussion and planning. Specifically,

school-based working groups might consider starting with the points of greatest divergence

between stakeholder groups as places where the most work in terms of consensus building needs

to occur.

Figure 10: Pattern Match as Discussion Generator (Points of Divergence)

111
Figures 11 and 12 utilize the cluster maps as areas for ongoing planning and discussion. Figure

11 provides an example where each cluster is used as a strategic goal area. Figure 12 presents

similar information albeit in a more finely grained manner. That is, Figure 12 suggests breaking

down each cluster into its statement set for discourse purposes.

Figure 11: Creation of Specific Objectives for Ongoing Discussion

112
Figure 12 Subdivide Objectives into Action-Oriented Tasks

In terms of ongoing research, this kind of study would benefit from several

methodological approaches. Repeating the concept mapping process with adequate resources so

as to ensure all steps were followed in a sequential manner would assist with the generalizability

of future student engagement findings. Also, repeating the concept mapping process in multiple

high schools within a given local would permit for broader participation of both students and

teachers and thus ensuring the brainstormed items are indeed representative of the context.

113
Another interesting opportunity for the context of the present study would be to do a concept

mapping study with schools both within and outside of the MSIP program.

As a research approach, concept mapping is not without limitations as noted in the

section above. That is, the phases of the design are often time consuming and can present

logistic issues, particularly when the study is undertaken in vast geographical regions, and the

software is not always intuitive. Taken together, though, this method is an effective way to

obtain high quality data, in this case from students, because of its directive process. Students

were able to participate equally in all phases of the research from conceptualization to analysis

and interpretation. This approach proved to be an effective vehicle for student voice in

educational research whereby students could present their ideas as a unified stakeholder group

free from the influence of others (e.g, teachers and/or administrators) who may appear to hold

more power in the school improvement agenda.

The present study uncovered some interesting findings with respect to the importance of

student friendships and peer groups as related to students‟ sense of student engagement. The

literature reviewed revealed that there is little research in this area that provides us with deep

understandings of the nature and quality of peer relationships within the classroom context

(Ryan, 2000; Osterman, 2000). Future qualitative research could delve deeper into the nature of

social connections among students as an initial approach to untangle and understand the

multidimensionality of factors at play.

Overall, research in the student engagement domain could include longitudinal and

observational dimensions. A longitudinal design is important so as to capture over multiple time

points, the degrees to which students and teachers move closer on points of divergence. Also, a

114
longitudinal design is imperative to be able to document the movement of teachers towards a

more collaborative approach to student-teacher interactions and student voice. Observational

techniques are often overlooked as time consuming and labour intensive but this type of data

collection would enhance the ability to „see‟ distinctions in teachers‟ differing approaches to

student voice.

Two fundamental issues need to be addressed so as to permit authentic student voice to

be nurtured within contemporary high schools: a change in mindset and changes in the structure

of educational institutions. Traditionally educational agenda‟s have been formed and maintained

by adults but it is time to rethink this approach and include students as co-collaborators in

authoring their education. It would be instrumental to this process for teachers to understand the

benefits that can be accrued when students have an authentic voice. That is, giving students a

voice can improve current educational practices when teachers listen to, and learn from student,

then they can begin to „see‟ the world from the students‟ perspectives. This study has added to

the understanding that students have unique perspectives on what happens in schools and

classrooms. For example, in the present study students were the drivers of the of the

diversity/ethnicity agenda as related to student engagement.

Undeniably the world is changing and there exist growing generational differences

between students and their teachers. One thing is clear, schooling „as is‟ needs to change so as to

be current and effective. One promising approach is to involve students in innovative

approaches to student voice. For example, students could be drivers of peer-led retreats, research

projects, and internet-based student forums. A critical aspect of engaging students as

collaborators is that they perceive themselves as partners. Evidence in other fields such as

medicine can provide insight into how this transformation can occur. Medical education research

115
informs us that doctors who have been found to nurture positive patient-provider relationships

and patient satisfaction are positively associated with quality care (Meredith, 2002). For their

part, teachers need to assist in teaching students how to be partners in their education. This joint

learning could help in moving teachers from their role as gatekeeper to one of collaborator.

116
 References

Anderburg, M. R. (1973). Cluster analysis for applications. New York: Academic

Press.
Ausubel, R. (1968). Cognitive learning theory. Beverly Hills, CA: Sage.
Bandura, A. (1986). Social foundations of thought and action. Englewood Cliffs, NJ:
Prentice-Hall.
Bascia, N., & Hargreaves, A. (2000). Teaching and learning on the sharp edge of
change. In A. Hargreaves (ed), The sharp edge of educational change: Teaching,
leading, and the realities of reform (pp.3-26). London, UK: Falmer Press.
Battistich, V., Solomon, D., Watson, M., & Schaps, E. (1997). Caring school
communities. Educational Psychologist, 32(3):137-151.
Bauermeister, R. F., & Leary, M. R. (1995). The need to belong: Desire for
interpersonal attachments as a fundamental human motivation. Psychological
Bulletin, 117(3):497-529.
Blount Morse, A., Anderson, A. R., Christenson, S. L., & Lehr, C. A. (2004).
Promoting school completion. Principal Leadership 4(6):9-13.
Boekarts, M., Pintrich, P. R., & Zeidner, M. (eds) (2000). Handbook of self-regulation:
Theory, research and applications. San Diego, CA: Academic Press.
Bowlby, G., & McMullen, K. (2002). At a crossroads: First results for the 18 to 20-
year old cohort of the Youth in Transition Survey. Ottawa: Statistics Canada.
Accessed at
http://www.statcan.gc.ca/bsolc/olc-cel/olc-cel?catno=81-591-X&lang=eng.
Bradley, R. H., Corwyn, R. F., McAdoo, H. P., & Coll, G. C. (2001). „The home
environments of children in the United States Part I: Variations by age,
ethnicity, and poverty status. Child Development, 72:1844-1867.
Brady, P. (2005). Inclusionary and exclusionary secondary schools. The effect of
school culture on student outcomes. Interchange 36(3):295-311.

117
Bryk, A. S., & Schneider, B. (2002). Trust in schools: A core resource for improvement.
New York: Russell Sage Foundation.
Burkett, E. (2002). Another planet: A year in the life of a suburban high school. New
York: Harper Collins.
Canadian Education Association (CEA). (2006). An agenda for youth – our thoughts so
far. Toronto:CEA.
Accessed at http://www.cea-ace.ca/media/en/AnAgendaforYouth.pdf
Chaplain, R. (1996). Pupil behaviour in primary schools. London: Pearson.
Coburn, C. E. (2003). Rethinking scale: Moving beyond numbers to deep and lasting
change. Educational Researcher, 32(6):3-12.
Cohen, D. K., & Ball, D. L. (1999). Instruction, capacity and educational improvement.
(Rep. No.RR-43). Philadelphia, PA: Consortium for Policy Research and
Education.
Coleman, J. S. (1996). Foundations of social theory. Cambridge, MA: Harvard
University Press.
Connell, J. P., Halpern-Felsher, B. L., Clifford, E., Crichlow, W., & Usinger, P. (1995).
Hanging in there: Behavioral, psychological, and contextual factors affecting
whether African American adolescents stay in school. Journal of Adolescent
Research, 10:41-63.
Connell, J., & Welborn, J. (1991). Competence, autonomy, and relatedness: A
motivational analysis and self-system process. In M. Gunnar and L. Srufe (eds),
Self processes in development: Minnesota symposium on child psychology,
volume 23 (pp.43-77). Hillsdale, NJ: Lawrence Erlbaum.
Cook-Sather, A. (2002). Authorizing students‟ perspectives: Towards trust, dialogue,
and change in education. Educational Researcher, 31(4):3-14.
Cook-Sather, A. (2007) Translating researchers: Re-imagining the work of investigating
students‟ experiences in school (pp711-726) . In D. Thiessen and A. Cook-Sather

118
 (eds), International handbook of student experience in elementary and secondary
school. Dordrecht, Netherlands: Springer.
Cousins, J. B., & Earl, L. M. (1992). The case for participatory evaluation.
Educational Evaluation and Policy Analysis, 14:397-418.
Cousins, J. B., & Earl, L. M. (1995). The case for participatory evaluation: Theory,
research, and practice. In J.B. Cousins & L.M. Earl (eds), Participatory
evaluation in education: Studies in evaluation use and organizational
learning (pp. 3-18). London: Falmer.
Cousins, J. B., & Leithwood, K. C. (1986). Current empirical research on evaluation
utilization. Review of Educational Research, 36(3):331-364.
Cousins, J. B., & Sutherland, S. (1989). Understanding connectedness in high school:
A concept mapping study. Paper presented at the annual American Educational
Research Association, San Diego, CA.
Cousins, J. B., & Whitmore, E. (1998). Framing participatory evaluation. In E.
Whitmore (ed), Understanding and practicing participatory evaluation.
New Directions for Evaluation, No. 80 (pp. 3-23). San Francisco: Jossey-Bass.
Covington, M. V. (1992). Making the grade: A self-worth perspective on motivation
and school reform. New York: Cambridge University Press.
Croninger, R. G., & Lee, V. E. (2001). Social capital and dropping out of high school:
Benefits to at-risk students of teachers‟ support and guidance. Teachers College
Record, 103(4):548-581.
Cuban, L. (1989). At-risk students: What teachers and principals can do. Educational
Leadership, 46(5):29-32.
Darling-Hammond, L., Ancess, J., & Ort, S. (2002). Reinventing high school:
Outcomes of the coalition campus schools project. American Educational
Research Journal, 39:639-673.

119
Datnow, A., Borman, G., & Stringfield, S. (2000). School reform through a highly
Specified curriculum: Implementation and effects of the CoreKnowlegeSequence.
The Elementary School Journal, 101(2):167-191.
Datnow, A., & Castellano, M. (2000). Teachers‟ response to Success for All: How
Beliefs, experiences, and adaptations shape implementation. American
Educational Research Journal, 37(3):775-799.
Davidson, M. L. (1983). Multidimensional scaling. New York : John Wiley and Sons.
Davidson, A. (1996). Making and molding identity in schools: Student narratives on
race, gender, and academic achievement. Albany: State University of New York
Press.
Earl, L., & Lee, L. E. (1998). Evaluation of the Manitoba of the Manitoba School
Improvement Program. Toronto: Walter & Duncan Gordon Charitable
Foundation.
Earl, L., Freeman, S., Lasky, S., Sutherland, S., & Torrance, N. (2002). Policy,
politics, pedagogy, and people: Early perceptions and challenges of large-scale
reform in Ontario secondary schools. Ontario Secondary School Teacher‟s
Federation: Toronto, ON.
Earl, L., & Sutherland, S. (2003). Student engagement in times of turbulent change.
McGill Journal of Education, 38(2):329-343.
Earl, L., Torrance, N., Sutherland, S., Fullan, M., & Ali, A. S. (2003). The Manitoba
School Improvement Program (MSIP) Final Evaluation Report. Toronto, ON:
Ontario Institute for the Studies in Education of the University of Toronto.
Eccles, J., Alder, T., Futterman, R., Goff, S., Kaczala, C., & Meece, J., & Midgley, C.
(1983). Expectancies, values, and academic behavior. In J.T. Spence (ed),
Achievement and achievement motives: Psychological and sociological
approaches (pp. 75-146). San Francisco: Freeman.
Eccles, J., & Midgley, C. (1989). State environment fit: Developmentally appropriate

120
 classrooms for early adolescents. In R. Ames and C. Ames (eds), Research on
motivation in education: Goals and concerns volume 3 (pp. 139-186). New York:
Academic Press.
Eccles, J. S., Midgley, C., & Adler, T. F. (1984). Grade-related changes in school
environment: Effects on achievement motivation. In J.G. Nicholls (ed),
Advances in Motivation and Achievement (pp. 283-331). Greenwich, CT: JAI
Press.
Eccles, J., Midgley, C., Buchanan, C., Reuman, D., Flanagan, C., & MacIver, D.
(1993). Development during adolescents. American Psychologist 48(2):90-101.
Elmore, R (1995) Structural reform in educational practice. Educational Researcher 24
(9):23-26.
Elmore, R. (2006). Leadership as the practice for improvement. Paper presented at the
International conference on Perspectives on Leadership for Systematic
Improvement, sponsored by the Organization for Economic Cooperation and
Development (OECD). London. Accessed at
http://www/oecd.org/dataocd/2/8/37133264/pdf
Elmore, R. F., Peterson, P. L., & McCarthy, S. J. (1996). Restructuring in the
classroom: Teaching, learning, and school organization. San Francisco:
Jossey-Bass.
Epstein, J. L. (2001). School, family, and community partnerships: Preparing educators
and improving schools. Boulder, CO: Westview Press.
Epstein, J. L., & McPartland, J. M. (1976). The concept and measurement of the quality
of school life. American Educational Research Journal, 13(1):15-30.
Evans, K. (2002). Fifth-grade students‟ perceptions of how they experience literature
discussion groups. Reading Research Quarterly, 37:46-49.
Everitt, B. (1980). Cluster analysis (2nd ed). New York: Halstead Press.
Fetterman, D. M. (1994). Steps of empowerment evaluation: From California to Cape

121
 Town. Evaluation and Program Planning, 17:305-313.
Fielding, M. (2001). Students as radical agents of change. Journal of Educational
Change, 2(2):123-141
Fielding, M. (2004). Transformative approaches to student voice: Theoretical
underpinnings, recalcitrant realities. British Educational Research Journal,
30(2):295-311.
Finn, J. (1989). Withdrawing from school. Review of Educational Research 59:
117-142.
Finn, J. (1993). School engagement and students at risk. Washington, DC: National
Center for Education Statistics.
Finn, J., Pannozzo, G. M., & Voelkl, K. E. (1995). Disruptive and inattentive withdrawn
behavior and achievement among fourth graders. Elementary School Journa,l 95:
421-454.
Finn, J & Rock, D. (1997). Academic success among students at risk for school failure.
Journal of Applied Psychology, 82:221-234.
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential
of the concept, state of the evidence. Review of Educational Research, 74(1):59-
109.
Fredricks, J. A., & Eccles, J. S. (2002). Children‟s competence and value beliefs from
childhood to adolescence: Growth trajectories in two “male-typed” domains.
Journal of Developmental Psychology, 38:519-533.
Friedkin, N. E., & Necochea, J. (1988). School system size performance: A
contingency perspective. Educational Evaluation and Policy Analysi,s 10:237-
249.
Fullan, M. (2001). The new meaning of educational change. New York: Teachers
College Press.
Fullan, M. G., Bennett, B., & Rolheiser-Bennett, C. (1990). Linking classroom and

122
 school improvement. Educational Leadership, May:13-19.
Fullan, M., & Hargreaves, A. (1992). What’s worth fighting for in your school?
Buckingham: Open University Press. (First published in 1991 by Ontario Public
School Teachers‟ Federation).
Fullan, M., Lee, L., & Kilcher, A. (1996). Lesson learned: The Manitoba School
Improvement Program (MSIP). Toronto: Walter & Duncan Gordon
Charitable Foundation.
Furman, G. C. (1998). Postmodernism and community in schools: Unraveling the
paradox. Educational Administration Quarterly, 34(3), 298-328.
Galloway, M., Pope, D., & Osberg, J. (2007). Stressed-out students-SOS: Youth
perspectives on changing school climates (pp. 611-635). In D. Thiessen and
A. Cook-Sather (eds), International handbook of student experience in
elementary and secondary school. Dordrecht, Netherlands, Springer.
Gans, J. (2000). Unpublished doctoral dissertation. Ithaca, NY: Cornell University.
Gardner, P., Ritblatt, S. N., & Beatty, J. R. (2000). The impact of school size on high
school student‟s outcomes and parent involvement. High School Journal, 83(2):
21-27.
Garling, A. (1984). Cognitive mapping. Environment and Behaviour, 1-34.
Gilbert, J. (2005). Catching the knowledge wave? The knowledge society and the
future of education. New Zealand: NCZER Press.
Golledge, R. (1986). Multidimensional analysis. New York: Plenum Press.
Goodenow, C. (1993). Classroom belonging among early adolescent students:
Relationships to motivations and achievement. Journal of Early Adolescence,
13(1):21-43.
Goslin, D. A. (2003). Engaging minds: Motivation and learning in American’s schools.
Lanham, MD: Scarecrow Education.
Greene, J. C. (2008). Mixed methods in social inquiry. San Francisco: Jossey-Bass.

123
Guthrie, J. T., & Anderson, E. (1999). Engagement in reading: Processes of motivated,
strategic, and knowledgeable social readers. In J.T. Guthrie & D.E. Alverman
(eds), Engaged Reading: Process, practices, and policy implications (pp.17-46).
New York: Teachers College Press.
Guthrie, J. & Wigfield, A. (2000). Engagement and motivation in reading. In M.
Kamil, P. Mosenthal, P.D. Pearson, and R. Barr (eds), Handbook of Reading
Research, volume III (pp.403-424). Mahwah, NJ: Lawrence Erlbaum.
Hargreaves, A., Earl, L., & Ryan, J. (1996) Schooling for change: Reinventing
education for early adolescents. London: Falmer Press.
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational
Research, 77(1):81-112.
Hill, P.T., Campbell, C., & Harvey, J. (2000). It takes a city: Getting serious about
urban school reform. Washington, DC: Brookings Institute.
Huberman, M., & Cox, P. (1990). Evaluation utilization: Building links between action
and reflection. Studies in Educational Evaluation, 16(1):157-179
Jardine, D. (2010). Ecopedagogy. In C. Kridel (ed), Sage Encyclopedia of Curriculum
Studies. New York: Sage Publications.
Jardine, D., Clifford, P., & Friesen, S. (2008). Back to the basics of teaching and
learning: Thinking the world together (2nd ed). New York: Routledge.
Jerome, L. (2001). Teaching citizenship: From rhetoric to reality. Education Today,
51(1):8-12.
Jordan, W., & Plank, S. (2000). Talent loss among poor students. In M.G. Sanders (ed),
Schooling students placed at risk (pp. 83-108). Mahwah, NJ: Lawrence Erlbaum.
Jules, V. (1991). Interaction dynamics of cooperative learning groups in Trinidad‟s
secondary schools. Adolescence 26(104):931-949
Kane , M., & Trochim, W. M. K. (2007). Concept mapping for planning and evaluation.
London: Sage Publications.

124
Karweit, N. (1989). Time and learning: A review. In R. E. Slavin (ed), School and
Classroom Organization (pp. 69-95). Hillsdale, NJ: Lawrence Erlbaum.
Katz, S., Sutherland, S., & Earl, L. (2002). Developing an evaluation habit of mind.
The Canadian Journal of Program Evaluation, 17(2):103-119.
Khattri, N., & Miles, M. B. (1994). Cognitive mapping: A review and working guide.
Sparkhill, NY: Center for Policy Research.
Krippendorf, K. (2004). Content analysis: An introduction to its methodology (2nd ed).
Newbury Park, CA: Sage.
Kruskal, J. B., & Wish, M. (1978). Multidimensional scaling. Beverly Hills, CA: Sage.
Ladd, G. W. (1990). Having friends, keeping friends, making friends, and being liked by
peers in the classroom: Predictors of children‟s early school adjustment. Child
Development 61:1081-1100.
Lee, V. E., Bryk, A. S., & Smith, J. B. (1990). The organization of effective secondary
schools. In L. Darling-Harmmond (ed), Review of Research in Education
(pp. 171-267). Washington, DC: American Educational Research Association.
Lee, V., & Burkam, R. (2003). Dropping out of high school: The role of school
organization and structure. American Educational Research Journal 40:353-393.
Lee, V. E., & Smith, J. B. (2001). Restructuring high schools for equity and excellence:
What works. Sociology of Education Series. New York: Teachers College Press.
Lee, V. E., & Smith, J. B. (1995). Effects of high school restructuring on size on early
gains in achievement and engagement for early secondary school students.
Sociology of Education, 68:241-270.
Leithwood, K. & Aitken, R. (1995). Making schools smarter. Thousand Oaks, CA:
Corwin Press, Inc.
Leithwood, K. & Jantzi, D. (1999). The relative effects of principal and teacher sources
of leadership and student engagement with school. Educational Administration
Quarterly, 35(supplemental):679-706.

125
Leithwood, K., & Jantzi, D. (2009). A review of empirical evidence about school size
effects: A policy perspective. Review of Educational Research 79(1):464-490.
Leithwood, K., Jantzi, D., & Haskell, P. (1997). Developing the organizational learning
capacity of school systems: A case study. Paper presented at the annual meeting
of the University Council for Educational Administration, Orlando, FL.
Levin, B (2000) Putting students at the centre in educational reform. Journal of
Educational Change, 1:155-172
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage.
Luthar, S. S., & Becker, B. E. (2003). Privileged but pressured? A study of affluent
youth. Child Development, 73:1593-1610.
Luthar, S. S., & Latendress, A. (2005). Children of the affluent: Challenges to well-
being. Current Directions in Psychological Science, 14 :49-53
Macbeath, J., Demetriou, H., Rudduck, J., & Myers, K. (2003). Consulting pupils-
A toolkit for teachers. Cambridge: Pearson Publishing.
Macbeath, J., Myers, K., & Demetriou, H. (2001). Supporting teachers in consulting
pupils about aspects of teaching and learning and evaluating impact. Forum,
43(2):78-82.
Marks, M. H. (2000). Student engagement in instructional activity: Patterns in the
elementary, middle, and high school years. American Educational Research
Journal, 37(1):153-184.
McIntyre, D., & Pedder, D. (2004). The impact of pupil consultation on classroom
practice. In M. Arnot, D. McIntyre, D. Pedder, & D. Reay (eds), Consultation
in the classroom (pp. 7-41). Cambridge: Pearson Publishing.
McIntyre, D., Pedder, D., & Rudduck, J. (2005). Pupil voice: Comfortable and
uncomfortable learnings for teachers. Research Papers in Education, 20(2):
149-168.
McLaughlin, M., & Talbert, J. (2001). Professional communities and the work of high

126
 school teaching. Chicago: University of Chicago Press.
McLinden, D., & Trochim, W. (1998). From puzzles to problems: Assessing the impact
of education in a business context with concept mapping and pattern matching.
Implementing Evaluation Systems and Processes, the American Society for
Training and Development, 18:285-304.
Mertens, D. M. (2009). Transformative research and evaluation . New York: Guilford
Press.
Michalski, G. V., & Cousins, J. B. (2000). Differences in stakeholder perceptions about
training evaluation: A concept mapping/pattern matching investigation.
Evaluation and Program Planning, 23(2):211-230.
Miles, M. B., & Huberman, M. (1994). Qualitative data analysis: An expanded
sourcebook. Beverly Hills: Sage.
Miller, R., Greene, B., Montalvo, G., Ravindran, B., & Nichols, J. (1996). Engagement
in academic work : The role of learning goals, future consequences, pleasing
others, and perceived ability. Contemporary Educational Psychology, 21(4):
388-422.
Milligan, J. (1979). Schema learning theory: An approach to perceptual learning.
Review of Educational Research, 49(2):197-207.

Mitra, D. L. (2003). Student voice in school reform: Reframing student-teacher

Relationships. McGill Journal of Education, 38(2):289-304.
National Research Council & Institute of Medicine. (2003) Engaging schools: Fostering
high school students’ motivation to learn. Washington, D.C.: The National
Academies Press.
Newmann, F. (1990). Student engagement in academic work: Expanding the perspective
on secondary school effectiveness. In J.R. Bliss & W.A. Firestone (eds),
Rethinking effective schools: Research and practice (pp.58-76). Englewood
Cliffs, NJ: Prentice-Hall.

127
Newmann, F (ed) (1992). Student engagement and achievement in American secondary
schools. New York: Teachers College Press.
Newmann, F. M., Wehlage, G. & Lamborn, S. D. (1992). The significance and sources
of student engagement. In F.M. Newman (ed), Student Engagement and
Achievement in American Secondary Schools (pp.11-39). New York: Teachers
College Press.
Novak, J. (1990). Concept mapping: A useful tool for science education. Journal of
Research in Science Teaching, 27(10):937-949.
Novak, J. D., & Gowin, D. B. (1984). Learning how to learn. Cambridge, England:
Cambridge University Press.
Nystrand, M. & Gamoran, A. (1991). Student engagement: When recitation becomes
conversation. In H.C. Waxman and H. Walberg (eds), Effective Teaching:
Current Research (pp.257-276). Berkeley, CA: McCutchan.
Organisation for Economic Co-operation and Development (OECD). (2002).
Understanding the brain: Towards a new learning science. Paris: Centre for
Educational Research and Innovation.
Osterman, K. (2000). Students‟ need for belonging in the school community.
Review of Educational Research, 70(3):323-367.
Patrick, B. C., Skinner, E. A., & Connell, J. P. (1993). What motivates children‟s
behavior and emotion? Joint effects of perceived control and autonomy in the
academic domain. Journal of Personality and Social Psychology 65:781-791.
Pekrul, S, Levin, B (2007) Building student voice for school improvement (pp. 711-
726). In D. Thiessen and A. Cook-Sather (eds), International handbook of
student experience in elementary and secondary school. Dordrecht, Netherlands:
Springer.
Penny, N. E. (2002). Creating selection criteria for a teen center intervention program:
A concept mapping/pattern matching study. Paper presented at the annual

128
 American Evaluation Association Conference, Washington, DC.
Peterson, P., Swing, S., Stark, K., & Wass, G. (1984). Students‟ cognition and time on
task during mathematics instruction. American Educational Research Journal,
21:487-515.
Pope, D. (2002). Doing school: How are we creating a generation of stressed-out,
Materialistic, and miseducated students? New Haven, CT: Yale University Press.
Rico, G. L. (1983). Writing the natural way: Using right-brain techniques to release
your expressive powers. Los Angeles, CA: J.P. Tarcher.
Rivière, D., Sotomayor, L., West-Burns, N., Kugler, J., & McCready, L. (2008).
Towards a multidimensional framework for student engagement. Toronto, ON:
Centre for Urban Schooling, Ontario Institute for Studies in Education at the
University of Toronto.
Rizzo-Michelin, L. L. (1989). Concept mapping in evaluation practice and theory: A
Synthesis of current empirical research. Unpublished Master‟s Thesis. Ottawa,
ON: University of Ottawa.
Rosas, S. R. (2005). Concept mapping as a technique for program theory development:
An illustration using family support programs. American Journal of Evaluation,
26(3):389-401.
Rudduck, J. (2002). The 2002 SERA lecture: The transformative potential of
Consulting young people about teaching, learning and schooling. Scottish
Educational Review, 34(2):133-137.
Rudduck, J. (2007). Student voice, engagement, and school reform. In D Thiessen &
A. Cook-Sather (eds), International Handbook of Student Experience in
Elementary and Secondary School. Dordrecht, Netherlands: Springer.
Rudduck, J., & Flutter, J. (2004). How to improve your school: Giving pupils a voice.
London, UK: Continuum David Fulton.
Rudduck, J., Chaplain, R., & Wallace, G. (eds) (1996). School improvement: What

129
 can pupils tell us? London: David Fulton.
Rumberger, R. W., & Palardy, G. J. (2002). Raising test scores and lowering dropout
rates: can schools do both? Paper presented at the annual meeting of the
American Educational Research Association, New Orleans, April.
Rutter, M. (1983). School effects on pupil progress: Research findings and policy
implications. Child Development 54(1):1-29.
Ryan, A. M. (2000). Peer groups as a context for the socialization of adolescents‟
motivation, engagement, and achievement in school. Educational Psychologist,
35:101-111.
Ryan, A. M., & Patrick, H. (2001). The classroom social environment and changes in
adolescents‟ motivation and engagement during middle school. American
Educational Research Journal, 28:437-460.
Sarason, S. (1971). The Culture of the School and the Problem of Change. Boston:
Allyn and Bacon.
Scardamalia, M. (2001). Getting real about 21st century education. The Journal of
Educational Change 2:171-176.
Scardamalia, M., & Bereiter, C. (2003). Knowledge building. In Encyclopedia of
Education (2nd ed, pp.1370-1373). New York: Macmillan Reference, USA.
Scheerens, J. (1992). Effective schooling: Research, theory, and practice. London:
Cassell.
Scholl, J. (1987). Cognitive maps as orienting schemata. Journal of Experimental
Psychology: Learning Memory and Cognition, 13(4):615-628.
Schultz, J., & Cook-Sather, A. (eds) (2001). In our own words: Students’ perspectives
on school. Lanham, MD: Rowman &Littlefield.
Schwartz, M., & Fischer, K. (2006). Useful metaphors for tackling problems in teaching
and learning. On Campus 11(1):2-9.
Senge, P. (2000) Schools that learn. New York: Doubleday.

130
Skinner, E., & Belmont, M. (1993). Motivation in the classroom: Reciprocal effects of
teachers behavior and student engagement across the school year. Journal of
Educational Psychology, 88:571-581.
Smilkstein, R. (1991). A natural teaching method based on learning theory. Gamut
36:12-15.
Smith, W., Butler-Kisber, L., LaRocque, L., Portelli, J., Shields, C., Sparkes, C., &
Vibert, A. (1998). Student engagement in learning and school life: National
project report. Montreal: McGill University Office of Research on Educational
Policy.
Solomon, D., Watson, M., Battistich, V., Schaps, E., & Delucchi, K. (1996).
Creating classrooms that students experience as communities. American Journal
of Community Psychology 24(6):719-748.
Soo Hoo, S. (1993). Students as partners in research and restructuring schools. The
Educational Forum(Summer):386-393.
Statistics Canada. (2003). Update on family and labour studies: Dropping out of high
school on the decline (89-001-XIE). Statistics Canada. Retrieved August 28,
2004, accessed: www.statcan.ca/english/freepub/89-001-XIE/2002001/drop.htm
Statistics Canada. (2004). In and out of high school: First results from the second
Cycle of the Youth in Transition Survey, 2002. Statistics Canada. Retrieved
August 28, 2006, accessed:
www.statca.ca:8096/bsolc/english/bsolc?catno=81-595-M2002014
Steinhouse, L. (1975). An introduction to curriculum research and development.
London: Heineman.
Sutherland, S., & Katz, S. (2005). Concept mapping methodology: A catalyst for
organizational learning. Evaluation and Program Planning, 28:257-269.
Swandt, T. (1994). Constructivist, interpretivist, approaches to human inquiry. In
N. Densin and Y. Lincoln (eds), Handbook of Qualitative Research (pp. 118-136).

131
 Thousand Oakes: Sage.
Taylor, A. F. (1989). Predictors of peer rejection in early elementary grades: Roles of
problem behavior, academic achievement, and teacher performance. Journal of
Clinical Child Psychology, 18(4):360-365.
Timperley, H., & Clarke, S. (eds) (2003). Unlocking formative assessment. Auckland,
NZ: Hodder-Moa.
Thompson, B. (2002). What future quantitative social science research could look like:
Confidence intervals for effect size. Educational Researcher, 31(3):25-32.
Trochim, W. M. K. (1989). An introduction to concept mapping for planning and
evaluation. Evaluation and Program Planning, 12:1-16.
Trochim, W. (1990). The evaluator as cartographer: Technology for mapping where
we’re going and where we’ve been. Paper presented at the Conference of the
Oregon Program Evaluators Network. Portland, OR: Accessed at:
http://www.socialresearchmethods.net/research/OPEN/The%20Evaluator%20as%
%20Cartographer.pdf.
Trochim, W. (1993). The reliability of concept mapping. Paper presented at the
annual meeting of the American Evaluation Association, Dallas, TX.
Trochim, W. M., Marcus, S. E., Masse, L. C., Moser, R. P., & Weld, P. C. (2008).
The evaluation of large research initiatives: A participatory integrative mixed-
method approach. American Journal of Evaluation, 29(1):8-28.
Trochim, W., & Linton, R. (1986). Conceptualization for evaluation and program
planning. Evaluation and Program Planning 9:289-308.
Twenge, J. M. (2009) Generational changes and their impact in the classroom: Teaching
Generation Me. Medical Education, 43: 398-405.
Tyack, D., & Cuban, L. (1995). Tinkering toward utopia. Cambridge, MA: Harvard
University Press.
Vibert, A. B., & Shields, C. M. (2003). Does ideology matter? Student engagement and

132
 critical practice. McGill Journal of Education, Spring Issue.
Voelkl, K. E. (1995). School warmth, student participation, and achievement. Journal
of Experimental Education 63(2):127-138.
Voelkl, K. E. (1996). Measuring students identification with school. American Journal
of Education 105(May):294-318.
Voelkl, K. E. (1997). Identification with school. American Journal of Education 105:
201-319.
Walter and Duncan Gordon Charitable Foundation. (1993). School improvement grant
program for secondary schools in Manitoba: Handbook for Applicants. Toronto:
Author.
Wandersee, J. (1990). Concept mapping and the cartography of cognition. Journal of
Research in Science Teaching, 27(10):923-936.
Wehlage, G., Rutter, R., Smith, G., Lesko, N., & Fernandez, R. (1989). Reducing the
risk: Schools as communities of support. New York: The Falmer Press.
Wentzel, K. R., & Asher, S. R. (1995). Academic lives of neglected, rejected, popular,
and controversial children. Child Development, 66:754-763.
Wentzel, K. R., & Caldwell, K. (1997). Friendships, peer acceptance, and group
membership in relation to academic achievement in middle school. Child
Development, 68:1198-1209.
Willms, J. D. (2000). Monitoring school performance for “standards-based reform.”
Evaluation and Research in Education 14(3&4):237-253.
Willms, J. D. (2003). Student engagement at school: A sense of belonging and
participation. Results from PISA 2000. Paris, France: Organization for
Economic Co-operation and Development.
Willms, J. D., Friesen, S. & Milton, P. (2009). What did you do in school today?
Transforming classrooms through social, academic, and intellectual engagement.
(First National Report). Toronto: Canadian Education Association.

133
Woods, P. (1996). Critical students: Breakthroughs in learning. International Studies in
Sociology of Education 4(2):123-146.
Yamamoto, K., Thomas, E. C., & Karns, E. A. (1969). School-related attitudes in
middle-school-age students. American Educational Research Journal 6: 191-206.
Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An
overview. Educational Psychologist 21:3-17.

134
 APPENDIX A

Letter of informed consent

To potential study participant:

I would like to invite your voluntary participation in a research study focusing on understanding
student engagement with high school. I will be speaking with administrators, teachers and
students to obtain their perceptions of student engagement.

I will ask individuals to participate in a face-to-face interview (administrators) or focus groups

(teachers and students) lasting approximately one hour in duration. During the interview you
will be asked questions about your understanding of student engagement, your views on your
school's efforts to promote student engagement and the impacts you think have been achieved
at your school. As the interview proceeds, I may ask questions for clarification or further
understanding, but my part will be to mainly listen to your speak about your views,
experiences, and opinions.

It is the intention that each interview will be audiotaped and later transcribed to paper; you
have the choice of declining to have the interview taped. You will be assigned a number that
will correspond to your interviews and transcript. The information obtained in the interview
will be kept in strict confidence and stored in a secure location. All information will be reported
in such a way that individual persons, schools, and school divisions cannot be identified.

You may at any time refuse to answer a question or withdraw from the interview process. You
may request that any information, whether in written form or on audiotape, be eliminated
from the project. Finally, I would encourage you to ask questions about the research and your
involvement with it. At your request, I will provide you with a summary of the findings of this
study.

135
Thanks you in advance for your participation,

Stephanie Sutherland

Ph.D. Candidate, Theory and Policy Studies in Education

Ontario Institute for Studies in Education of the University of Toronto

_________________________________________________________________________

By signing below, you are indicating that you are willing to participate in the study, you

have received a copy of this letter, and you are fully aware of the conditions above.

Name: _________________________ School: _________________________

Signed: ________________________ Date: ___________________________

136
 APPENDIX B

REPORT PREPARED FOR PARTICIPATING SCHOOLS

Student and Teacher Perceptions of Student Engagement:

A Concept Mapping Study

Participants: Students, Teachers & Administrators – Seven Oaks School

Division

Prepared by:

Stephanie Sutherland

OISE / University of Toronto

Department of Theory & Policy Studies (Floor 6)

252 Bloor Street West

Toronto, Ontario

M5S 1V6

ssutherland@oise.utoronto.ca

June 2002
_____________________________________________________________________________________________

Please do not cite or reproduce without permission of the author.

137
Background

In the fall of 2001, two schools within the Seven Oaks School Division were approached to

participate in a study focusing on increasing our understanding of student engagement. Each of

these schools had been involved with the Manitoba School Improvement Program (MSIP) for a

number of years. MSIP‟s philosophy has been, and continues to be, “to improve the learning

experiences and outcomes of secondary school students, particularly those at-risk, by building

schools‟ capacities to become transforming schools that engage students actively in their own

learning (Manitoba School Improvement Program Strategic Plan, 1995). Therefore, schools

working with MSIP offered an ideal context to investigate students‟ and teachers‟ understanding

of student engagement within an improvement oriented framework.

After an extensive review of the literature, I came to the conclusion that student engagement was

indeed a slippery concept. Depending on a person‟s role, student engagement seemed to take on

multiple meanings. That is, depending on one‟s position (i.e., student, teacher, administrator,

academic, politician, etc.) there exists many degrees of meaning. For the present study, I thought

it would be intriguing to have arguably the two most important groups in education - teachers

and students - define the concept themselves. In doing so, I needed a methodology that was

participatory in nature. Specifically, I needed a means to collect data that was authored by

participants, a type of “bottom-up” approach. Concept Mapping was chosen as a method that

138
would best facilitate group level analyses.4 There were 8 teachers and 8 students from each of

the two schools who volunteered to participate (32 participants in total).

The Concept Mapping Process

Brainstorming Sessions

I approached the administration in the participating schools and was received with enthusiasm
and with the conviction that a study of this nature was important and worthy of research. In
November 2001 I traveled to Winnipeg and met with groups of teachers and students from the
two schools. During these meetings, the groups (separately) engaged in brainstorming sessions
based on the following statement: “generate short statements or phrases of what student
engagement means to you”. The student and teacher groups had little difficulty generating
such statement lists. As per the Concept Mapping method, I took the statements (from
teachers and students) and combined them into one master list. Care was taken to ensure
equal representation of both student and teacher ideas. The final list consisted of sixty
statements (see Table 1).

Sorting & Rating Task

Next, I printed the statements on 3” by 5” cards. Again, I traveled to Winnipeg the following March
(2002) and met with the same teacher and student groups from each of the two schools. Each person
was given their own pile of the sixty cards and was asked to “sort the cards in a way that makes sense
to you”. There were only two rules, 1) you were not allowed to put all of the cards in one pile, and 2)
you were not allowed to have sixty separate piles (containing one statement each). In completing this
sorting task, everyone was really asked to think about how the different statements fit together – or
not! After the sorting task was complete everyone was asked to rate each individual statement on a
scale of one to five (five being really important to their understanding of student engagement, one
being not very important).

4 For a more detailed description of Concept Mapping the reader is directed to:

Trochim, W. (1989). An introduction to concept mapping for planning and evaluation. Evaluation and

Program Planning, 12 (1): 1-16.

139
Generation of the Concept Maps

The next step was to enter the teacher and student generated data in the Concept System
software. The computer program groups the data into clusters. These clusters represent a
“best fit” according to how teachers and students sorted the statements. Table 2 illustrates
how the statements were grouped into clusters. Figure 1 graphically displays the data
contained in Table 2. If you look to the right hand side of Figure 1, you’ll notice some smaller
clusters, this means that both students and teachers sorted these items in a similar fashion (see
especially, Diversity/Belonging, Students at the Centre, Engagement as a feeling and Student –
Teacher interactions). Conversely, the larger, or more elongated the cluster appears are
examples of groups sorted the statements differently (see especially, Beyond the classroom,
Aspects of pedagogy, Variety in school structure and Professional educators).

Pattern Match

Based on the rating task (rate statements in importance from one to five), the software
produces a pattern match map (see Figure 2). The two vertical lines represent the two groups
involved – teachers and students. The horizontal lines represent the degree to which the
students and teachers either converge or diverge in their perceptions of importance. From a
cursory look at Figure 2, it appears that both students and teachers rated the statements
contained under the heading “Diversity/Belonging” as most important. Whereas, both groups
rated items under “School policy/structure” as least important with respect to student
engagement. The areas that illustrate the most divergence in student and teacher rating were
the clusters “students at the centre” and “aspects of pedagogy” (see again, Table 2 for
statements within each of these clusters).

Why is this Important?

 Specifically, this study is an example of participatory research that can help to stimulate
discussion in schools;
 This study adds to the critically needed study voice perspective in education;
 This research can help drive school-level planning and reporting, and has the potential to
increase the effectiveness of school-based projects focusing on the engagement of secondary
students.

140
What’s Next?

 More detailed analyses;

 Change/edit cluster names
 Organize student and teacher discussion groups based on these findings;
 Utilize this data for ongoing school-based planning and reporting;
 Discuss how this data can impact division and school level policies.

Table 1: Combined Teacher & Student Items

Student Engagement Brainstorming Items (Teachers & Students)

1. Guidance Support 31. High teacher expectations of students

2. Parental involvement 32. School safety

3. Timely grading/assessment of student work 33. Need for student space

4. Opportunities for student leadership 34. Compromise

5. Active role in classroom discussions 35. School spirit

6. Teacher as friend 36. Doing work in groups

7. Teacher is passionate about teaching 37. Student ownership in learning

8. Lots of program/course options 38. Balance of teacher & student ideas

9. Student independence in learning 39. Community involvement

10. Variety in teaching style 40. Different types of school involvement

11. Relevance of class material 41. School policies

12. Incentives to promote student learning 42. Teacher job satisfaction

13. Extra-curricular activities 43. Student input into learning

14. Changes in regular timetable 44. Trust

15. Small classes 45. Having the same teacher/student

more

141
 than once

16. Involvement in student council 46. Competition

17. Hiring of younger teachers 47. Student attitudes/interest in classes

18. School reputation 48. Have fun in class

19. Feel comfortable at school 49. Experience success in learning

20. Teachers’ reputation 50. Different assessment methods

21. Importance of friends/socializing 51. Student motivation to learn

22. Changes in administration 52. Demanding curriculum

23. Student-teacher relationships 53. Opportunities for teacher-student

decision-making

24. Teacher professional development 54. Teacher is entertaining/interesting

25. School is accepting of different social groups 55. Mixed ability classes

26. School is multi-cultural 56. Challenging class projects

27. Support for student ideas 57. Respect for others

28. Effective communication 58. Supportive principal/vice-principal

29. Interesting classes 59. Smiling

30. Involvement in sports 60. Being recognized

Note: The above items were generated by teacher and student brainstorming sessions. Next, these
items were combined to create a master list. Care was taken to ensure equal representation from each
of the groups (teachers & students).

142
Table 2: Statements by Cluster (Student & Teacher Sorting Task)

Cluster: Professional educators

1) Guidance support
8) Lots of program/course options
17) Hiring of younger teachers
20) Teachers’ reputation
24) Teacher professional development
42) Teacher job satisfaction
58) Supportive principal/vice-principal

Cluster: Variety in school policy/structure

2) Parental involvement
14) Changes in regular timetable
22) Changes in administration
40) Different types of school involvement

Cluster: Aspects of pedagogy

3) Timely grading/assessment of student work

10) Variety in teaching style
15) Small classes
28) Effective communication
31) High teacher expectations of students
50) Different assessment methods
52) Demanding curriculum

Cluster: Beyond the classroom

4) Opportunities for student leadership

13) Extra-curricular activities
16) Involvement in student council
18) School reputation
30) Involvement in sports
35) School spirit
39) Community involvement
41) School policies

143
Cluster: Diversity/Belonging

21) Importance of friends/socializing

25) School is accepting of different social groups
26) School is multi-cultural
32) School safety
33) Need for student space

Cluster: Student – Teacher interactions

5) Active role in classroom discussions

7) Teacher is passionate about teaching

9) Student independence in learning
11) Relevance of class material

29) Interesting classes

37) Student ownership in learning

38) Balance of teacher & student ideas
54) Teacher is entertaining/interesting
56) Challenging class projects

Cluster: Students at the centre

12) Incentives to promote student learning

27) Support for student ideas
43) Student input into learning
45) Having the same teacher/student more than once
55) Mixed ability classes

Cluster: Engagement as a habit of mind

36) Doing work in groups

47) Student attitudes/interest in classes
48) Having fun in class
49) Experience success in learning
51) Student motivation to learn
53) Opportunities for teacher-student decision-making

144
Cluster: Emotions

6) Teacher as friend
19) Feel comfortable at school
23) Student-teacher relationships
34) Compromise
44) Trust
46) Competition
57) Respect for others
59) Smiling
60) Being recognized

145