25

CHAPTER TWO: LITERATURE REVIEW

Introduction

In his article Toward a finer description of the connection between arts education

and student achievement, Aprill (2001) argues that in order to increase standardized test

scores, students need to be provided with authentic intellectual work that requires them to

engage a wide range of representations, and to express themselves in a wide range of

forms. Empirical research has demonstrated that when the development of the types of

higher order thinking skills required for “authentic intellectual work” are emphasized in

an arts-integrated curriculum, and when the foundational affective elements of a school

are strong, students will achieve at significantly higher levels through an arts-integrated

curriculum than they would in a traditional, non-arts-integrated curriculum. Indeed, the

most successful arts-integrated programs are credited with moving entire districts from

mediocre to high-achieving status within their state’s hierarchy of student achievement on

standardized assessments (Housen, 2002) .

This chapter will begin with a definition of higher-order thinking, its importance

in math education today, and the strategies used to identify higher-order thinking in

mathematics learning. This section will also include a brief description of the Common

Core Standards for Mathematical Practice.

I then define arts integration, describe the historical background that has led to this

approach, including the socio-cultural issues to which it responds, and provide a literature

review about arts integration. The majority of this review has been divided into two parts.

The first part will describe three types of arts integration and the research related to
 26

program-focused studies; the second will explore studies that are meta-analyses of arts-

integrated programs.

In the final sections of this chapter, I will introduce Beautiful, Beautiful Math by

situating it among the art integration models to be discussed. I will then highlight the

research that has been most influential in my thinking, and refer to their findings to

demonstrate the ways in which the teaching strategies used in Beautiful, Beautiful Math

can help students to develop higher-order thinking skills.

Defining Higher-Order Thinking

Higher-order thinking takes place when an individual combines new information

with information already stored in memory, and interrelates, rearranges, and extends all of

this information to achieve a purpose or to find answers to perplexing problems (Lewis &

Smith, 1993). To use higher-order thinking skills, students must start with information in

their long-term memories (Hardiman, 2012). Hardiman identifies eight strategies teachers

can employ to ensure their students retain such information, including: rehearsal, the

repeated practice of information; elaboration, adding one’s own meaning to what is being

learned; generation, creating new information from individual pieces of learning;

enactment, physically acting out material; oral production, producing words out loud

rather than reading silently; effort after meaning, investing a large amount of effort to get

comprehension; emotional arousal; and, pictorial representation. While these strategies

are primarily used to help students retain information such as facts, definitions, and

routine procedures, Hardiman stresses that students without adequate foundational

 27

knowledge struggle, and sometimes are entirely unable to use higher-order thinking

strategies.

Additionally, in order to ensure student success with higher-order thinking,

schools and individual teachers must also work on foundational affective and non-

cognitive factors that have been shown to be important for student learning in general.

Camille Farrington’s research on schools’ affective environments and teachers’ attention

to the emotional landscape of their classrooms demonstrates that non-cognitive factors

significantly impact student learning (Farrington et al., 2012). These include what

Farrington calls the “academic mindsets,” the ways in which students feel about

themselves and their abilities within the school environment. These can be demonstrated

when a student feels like they “belong,” their efforts will lead to growth, and they can be

successful, and when they value their work. Additionally, students need to be able to

persevere within an academic environment, be engaged with school, motivated to learn,

and able to develop the ability to pay attention.

Higher-order thinking is an umbrella which includes several types of thought

processes, namely critical thinking, strategic thinking, creative thinking, and meta-

cognitive thinking. Lauren Resnick (1987) explained that higher-order thinking is often

difficult to define yet “we know it when we see it.” She did, however, provide a

definition of higher-order thinking as that which is nonalgorithmic, complex, often

yielding multiple solutions, relying on nuanced judgment, requiring the use of multiple

criteria, including uncertainty, necessitating self-regulation, imposing meaning (finding

structure in apparent disorder), and effortful. Norman Webb (1997, 1999) drew from

Resnick’s definition in his four-part taxonomy, Depth-of-Knowledge, to determine the

 28

alignment between the complexity of tasks described in state standards (including

Common Core) and those required by items on the state’s assessments. The complexity

of both the content and the cognitive demand are described in Webb’s Depth-of-

Knowledge (DOK) levels, which apply across disciplines. DOK -1, “Recall and

Reproduction,” includes recall of factual information and routine procedures. DOK-2,

“Basic Application of Skills/Concepts,” includes the use of appropriate information and

procedures within routine problems. DOK-3, “Strategic Thinking,” requires reasoning,

developing a plan or sequence of steps, some decision-making and justification, and is

abstract, complex, or non-routine. DOK-4, “Extended Thinking,” includes problems that

investigate or apply to the real world, require time to research and problem-solve, and

may have multiple conditions involved in the task. At this fourth level, these problems are

non-routine and cross disciplines and content areas.

The tasks Webb describes in his DOK Levels 3 and 4 require students to

“interrelate, rearrange, and extend information,” all of which require higher-order

thinking. Because our entire economic system, communication systems, and dependence

on global communities have evolved at unprecedented rates during the past two decades,

these requirements are particularly important for learning mathematics today. The

mathematical skills needed for today’s jobs in our “information society” are far more

complex than those required even twenty years ago. These skills allow people to make

sense of data, reason numerically, and use mathematics to make sense of their world.

They have become increasingly important to our economic stability and well being.

Lacking these skills, people are not qualified for many jobs, and they also have difficulty

with now routine quantitative tasks.

 29

Today’s mathematics curriculum has become more demanding as a result of

different standards movements; for example, the Common Core State Standards for math

now require students to address more tasks at DOK levels 3 and 4 than ever before. The

Common Core Standards for Mathematical Practice describe the different types of

mathematical “expertise” that students in all grades need to be developing in order to

solve more of these rigorous problems. The first five Practice Standards are based on the

NCTM Process Standards of problem solving, reasoning and proof, communication,

representation, and making connections; the final three Practice Standards are built upon

recommendations from the National Research Council’s report Adding It Up, which

describes the need for mathematical proficiency in adaptive reasoning, strategic

competence, conceptual understanding, procedural fluency, and productive dispositions

(Initiative, 2012). The eight Standards for Mathematical Practice are included in full in

Appendix E.

There continues to be a lingering fear that we as a country are not moving far

enough fast enough to ensure our world standing (Graham, Cuoco, & Zimmermann,

2010). Beyond the public debate about keeping up with Singapore, China, and other

rapidly advancing nations, however, is also the fear that our democratic ideals may erode

if our citizenry does not develop high levels of thinking with mathematics. As Lauren

Resnick stated about our shared democratic “ideal community” nearly a quarter of a

century ago, “…we envision a culture of reason, analysis, and reflection, based on certain

shared knowledge…Building such civic consciousness…may be the most important

challenge facing educational research and reform today (Resnick, 1987a).”

 30

Drawing on all the above, I will use a simple definition of higher-order thinking

based on Lewis and Smith’s (1993) definition: Higher-order thinking is combining new

information with prior knowledge to find answers to perplexing, non-routine problems. I

will also draw from Webb’s task descriptors in his DOK levels 3 and 4, because those

provide behaviors that are both observable and measurable. We must not forget, however,

that there are two prerequisites described above that must be in place to move students

into the realms of higher-order thinking. First, students must have adequate and accurate

mathematical information in their memories. Secondly, students need to be part of a

nurturing emotional environment in which they are encouraged to become part of a

dynamic academic community.

Defining Arts Integration

Arts integration is the use of the arts as a pedagogical tool for enhancing student

learning and helping information “stick” in students’ memories, by developing conceptual

understanding, and enhancing their ability to use higher-order thinking (Hardiman, 2012).

In some arts-integrated curricula, various arts forms are used to help students learn skills

such as motivation, perseverance, and other habits of mind that can be transferred from

the arts to other subjects. For example, a visiting artist who provides lessons on a variety

of instruments may want students to play Haydn’s Surprise Symphony (#94) for a concert.

During their lessons, the visiting artists identify that the kettledrums are not coming in at

precisely the right moment after the soft piano opening. Therefore, they show their

students precisely how to determine the correct moment to play, and they help them
 31

practice several times before having the entire orchestra start again from the beginning.

Thus, the visiting artist is teaching more than Haydn’s symphony. They are also

providing lessons on how perseverance and careful revision lead to a much stronger

result. Although learned in music, students can apply these same habits in any learning

experience.

At other times, actual content from non-arts classes is taught during an arts class.

Usually this includes factual information, such as historical background, rather than skills

that will transfer more broadly. An example of this type of arts integration is when the art

teacher has students working with collage and shows them the work created by Romare

Bearden in the 1960’s. In order for her students to more fully understand the power of

these works, the art teacher provides a brief synopsis of the historical events that took

place when Bearden created his work, and his formation of the Spiral Group of artists who

sought to make a contribution to the civil rights movement. Thus, students learn about

both the civil rights movement and the art of collage at the same time.

Occasionally, the arts and another subject are more deeply intertwined. This can

happen when the art teacher or a visiting artist plans with another teacher; at those times,

students learn art skills and concepts along with the content and big ideas of the other

discipline. An example of this is having students examine the anti-war and anti-

mechanization paintings produced after World War I, such as those by Otto Dix, while

reading Remarque’s All Quiet on the Western Front. As a culminating product, students

might create their own wood block prints in order to illustrate the ways in which machines

have both helped and hurt humanity. Thus, students are learning about World War I, the
 32

history of art, a major work of literature, and the creation of wood block prints, all within

the context of a single unit.

Historical Background

To understand how the different forms of arts integration evolved, I will first

briefly explore the history of arts integration within three school movements that rose out

of the destruction caused by World War I, including Rudolf Steiner’s Waldorf schools,

Kurt Hahn’s Outward Bound schools, and John Dewey’s educational philosophy of

“learning by doing.” Each of these movements helped establish a path toward the various

arts integration models that sprang up in the late twentieth century and whose research I

will later explore in more detail.

The socio-cultural milieu of the two decades between the World Wars was a time

of great social and political change throughout both Europe and the United States. In

1919, the Bolsheviks took over Russia, the same year that the Treaty of Versailles

imposed humiliating reparations on Germany and redistributed all its territories and

annexed lands. With the exception of Russia, democracy was introduced in every country

formerly ruled by a monarchy. The League of Nations met for the first time in Geneva,

Switzerland in 1920. In the United States, the Great Migration of African-Americans

began in 1910, and its intensity increased between the wars; ultimately nearly 6 million

people moved north. The Harlem Renaissance flourished, and from 1919 through most of

1929, the economy of the United States grew at an unprecedented rate. Indeed, until the

stock market crashed in October, 1929, the 1920’s in the United States could be

characterized as a time of high optimism.

 33

It was during the immediate post-war period that people in both Europe and the

United States were seeking a new, peaceful and prosperous way of living, and their hopes

were pinned on new forms of schooling as well as well as democracy. Because many

people were looking for new and much more humanistic forms of schooling, three men

were able to design new schools with curricula that relied heavily on arts integration.

These three were Rudolf Steiner and Kurt Hahn in Germany and John Dewey in the

United States. Their ideas and ideals, coupled with the work from schools based upon

their philosophies, established a foundation which led to the creation of arts integration

programs during the late twentieth and early twenty-first centuries.

The first arts integration in schools took place after the destruction Europe faced in

the aftermath of World War I, primarily as an alternative to the Prussian volksschules.

Established and partially funded by Frederick William I in the early 18th century,

volksschules were the first form of compulsory education in the world (Barkin, 1983).

Although the volksschules had been successful in creating literate population, by 1918

their rote teaching methods were seen as antiquated and repressive (Flavin, 1996).

Educational philosophers believed that students needed to discover their individual talents

and become capable of thinking for themselves.

Similar ideas emerged across the Atlantic, as the United States began freeing

schools from rigid confines as well. American common schools, which were based on the

Prussian model (Reavis, 1945), also helped establish a literate population but the rigidity

of their methods were considered outdated in the aftermath of World War I.

Rudolf Steiner opened his first school in 1919 in Stuttgart, Germany, as a place

where the employees of the Waldorf-Astoria Cigar Factory could send their children
 34

instead of to the restrictive volksschules (Dahlin, 2007). Although based upon many of

the principles of anthroposophy, the religion Steiner founded, Waldorf schools were not

themselves religious institutions. Among Steiner’s most deeply held beliefs were that

children from an early age need to be surrounded by beauty and actively engaged in play

and opportunities to build up memory. Much of the play and memory work in Waldorf

Schools became centered around the arts. Students sing while transitioning in or between

classes, they declaim poetry, they dance for exercise, and they draw to learn mathematics.

Waldorf educators believe that using the arts to teach surrounds children with beauty and

allows them to remember more of what they learn.

In 1920, Kurt Hahn also opened his first school in southern Germany, Schule

Schloss Salem where he served as its headmaster until 1933. Again in a break with

Prussian tradition, students at Salem were active physically and mentally. They learned to

sail, established goals for themselves, and were graded on their character as well as in

imagination, art work, and practical work (handicrafts) (Miner, 2000). Students took part

in “expeditions,” in which they were challenged both physically and mentally. Initially

only the physical side of Hahn’s educational model took root in the United States; in the

1950’s Outward Bound programs were founded with the mission of taking students and

adults into the wilderness to push their physical and emotional limits. In 1992,

Expeditionary Learning was founded to replicate Hahn’s full school experiences in this

country. A natural next step for Expeditionary Learning was to integrate the arts more

thoroughly into what became “learning expeditions” (Cousins, 2000;

ExpeditionaryLearning, 2011).
 35

John Dewey’s educational philosophy, especially his ideas about students

“learning by doing” (Dewey, 2007), took hold throughout the United States in the 1920’s

and 30’s. Because he did not have a specific school model, his beliefs and philosophy

were open to a wide range of interpretations, some of which led to solid student learning

while other interpretations kept students busy without much academic attainment. Often

Dewey’s work was garbled in the translation between philosophy and practice, as in this

description of Dewey’s method from To Kill a Mockingbird (Lee, 2010):

“Our teacher says Miss Caroline’s introducing a new way of teaching. She
learned about it in college. It’ll be in all the grades soon. You don’t have to learn
much out of books that way—it’s like if you wanta learn about cows, you go milk
one, see?”
“Yeah Jem, but I don’t wanta study cows, I-”
“Sure you do. You hafta know about cows, they’re a big part of life in Maycomb
County.”
I contented myself with asking Jem if he’d lost his mind.
“I’m just trying to tell you the new way they’re teachin‘ the first grade, stubborn.
It’s the Dewey Decimal System.”

Later Scout adds:

The remainder of my school days were no more auspicious than the first. Indeed,
they were an endless Project that slowly evolved into a Unit, in which miles of
construction paper and wax crayon were expended by the State of Alabama in its
well-meaning but fruitless efforts to teach me Group Dynamics.

Despite the difficulties of translating Dewey’s philosophy into pedagogy, over

time schools in the United States were expected to provide students with opportunities to

make and do things, and often this was translated into making art. Rarely, however, did

making art follow Dewey’s ideals about experiencing art (Dewey, 1980). Except in a

handful of schools, Dewey’s beliefs ultimately gave way to art that was primarily

contained within the realm of art classes and music classes, physical education classes,

and shop.
 36

Models of Integrated Arts and Related Program-Based Research

By the 1960’s and 70’s, many schools in the United States were teaching art in

separate classrooms with different teachers handling the visual arts and music classes. As

described by Efland, the function of visual art and music classes in elementary schools

was to provide classroom teachers a break from their students or even a chance to catch

up a student who had fallen behind. Efland describes the “hidden functions” of the school

art style as including a minimum of “cognitive strain” (Efland, 1976). By the 1970’s, and

even in many schools today, this remains the model for elementary arts teaching.

Although in this dissertation proposal I define arts-integration as the use of the arts

as a pedagogical method for enhancing student learning, there is not yet a widely accepted

definition of arts-integration. Art educators and researchers who study the subject often

assume an understanding of its meaning. Winslow was the first person to write about an

integrated arts program, explaining that “whenever broader aspects of any school subject

are considered, it will be realized that the integration of subject matter and of school

experience is inevitable” and that “in such an educational program, art must be made to

function as an integral part…” (Winslow, 1939).

More recently researchers have narrowed the definition of arts integration to

include only those classrooms in which the arts and the other content area teachers plan

their lessons collaboratively (DeMoss & Morris, 2002). Classrooms in which the arts and

regular classroom teacher do not tightly collaborate would instead be classified as “arts

enhancement” or “arts enrichment” classrooms. Because it is very difficult to determine

from most research studies the extent to which the teacher and artist (or art teacher)
 37

actually planned and taught together, I have chosen to use arts integration more broadly

and to include everything except the “school arts style,” to be arts integration of one type

or another. Thus, arts integration happens whenever the arts are taught alone or together

with another subject in a way that encourages students to use the skills and conceptual

understandings from one subject to build stronger skills and conceptual understandings in

the other subject. Sometimes this is one-sided, usually with art serving the “core” subject;

at other times, as I described earlier, teachers of the two subjects team up and students

gain insights and deep understandings in both subjects.

Below, I will use three models to describe different forms of the arts integration

used in schools: the Arts Enrichment Model; the Non-Substantive Arts Integration

Model; and the Substantive Arts Integration Model. These models differ from each other

in the ways in which students are taught (either in secluded arts classes or throughout the

various subjects), and in the ways in which the arts are used (either as a vehicle for deeper

understanding of both arts and another discipline or primarily as a reward at the end of a

unit of study).

Research on Arts Enrichment Programs

Arts enrichment programs are those in which the arts instruction exists apart from

other courses, whether in an after school program or in courses during the school day.

There is usually no expectation for the arts learning to automatically transfer to the

learning of other subjects; indeed, as Hetland and Winner (2004) complain, the specific

learning taking place in arts courses is often undocumented and, therefore, often unknown

to researchers.
 38

The Arts Enrichment Model is when the arts are taught separately but with much

more emphasis on student learning. Infused with constructivist learning philosophy, Arts

Enrichment Model programs encourage students to reflect upon their experiences and to

make meaning manifest through the visual arts, drama, music, dance, etc. Some of these

programs live within schools, and students can participate in them during the school day.

This model also includes many secondary classes in the visual arts, music, and drama as

well as Visiting Artists programs.

Other Arts Enrichment Model programs, however, take place after school hours,

and sometimes in off-site locations. Arts Enrichment Model Programs in Rochester, New

York include music classes at Hochstein School of Music, visual arts classes at the

Memorial Art Gallery, and the numerous dance classes at studios throughout the city and

suburbs. More nationally recognized programs include the Lincoln Center Institute,

classes for children at the Joffrey Ballet in Chicago and the New York City Ballet’s

children workshops, along with the New York Philharmonic’s Young People’s Concerts.

These programs may include themes and key ideas from other disciplines and their work

may be rich and rewarding, but the overall school curriculum remains aligned to

“reinforce the separate and discrete character of academic disciplines” (Clark, 1997).

One of the most commonly repeated impacts of arts enrichment programs is the

“Mozart Effect,” the idea that studying music will increase a child’s reading skills. This

idea first arose from a 2000 meta-analysis of 24 correlational studies that attempted to

measure the impact of music instruction on reading. Ten of the studies included data

provided by the College Board, allowing for comparisons between the number of music

courses students took in high school with their SAT scores (Butzlaff, 2000). The two
 39

measures, number of music courses taken during high school and SAT scores, were very

strongly correlated; correlation does not necessarily imply causation, however. To test the

hypothesis that the study of music caused higher SAT scores, Butzlaff included as part of

this research an additional meta-analysis with six experimental studies; this analysis

yielded no reliable effect (Butzlaff, 2000). Winner and Hetland re-examined Butzlaff’s

data for the REAP study and determined that there is not yet enough evidence to say that

the study of music enhances students’ achievement in literacy (Winner & Hetland, 2001).

Hetland’s meta-analysis of experimental research did, however, find positive impacts of

music study on spatial-temporal skills (Hetland, 2000a).

James Catterall mined four national longitudinal databases from the Department of

Labor and the Bureau of Labor Statistics, with more than 8.900 students ages 12 to 18

from 1998 to 2008, to determine whether increased participation in that arts correlates to

stronger academic achievement in students from families with lower socio-economic

status (Catterall, 2012). These databases included survey data in addition to annual

interviews, examinations of school transcripts, and parent interviews. He designed an arts

engagement scale to find students most intensely engaged in the arts, via extracurricular

activities or coursework. He sorted the data by socioeconomic status and then followed

those students’ academic achievement, determined by their grades, standardized

assessment scores, and whether they graduated from high school and/or college. He also

followed their civic engagement scores, established via their levels of voting,

volunteering, and engagement with local or school politics.

While Catterall cautioned against assuming that correlation between arts

participation and academic achievement is causal, the results are impressive. Low SES
 40

students who had high levels of arts engagement showed more positive outcomes than

their low-arts-engaged peers. They did better academically and were more civic minded

throughout middle and high school; they showed achievement levels close to or even

surpassing those of the population as a whole. They were more likely to complete a

calculus course, to graduate from high school, and to attend and then graduate from a

four-year college. They were also much more civic-minded, volunteering and voting in

much larger numbers. Additionally, their GPA’s and standardized test scores were

higher. Despite these impressive statistics, however, the question remains whether

stronger students are simply more inclined to participate more in the arts than their lower

achieving peers.

Although the research was on a much smaller scale, studies of The Song Room

(TSR), explores experimentally controlled data which highlights similar results and

provides more insights into the specifics of what students learn in music classes (Caldwell

& Vaughan, 2012). The Song Room is an Australian non-profit organization providing

free music and arts-based programs for low SES children. Students received music

instruction for an hour a week. Research examined data from 10 TSR elementary schools

and then compared student achievement and learning dispositions with 4 non-TSR

schools, matched for similar SES. Results demonstrated that children who participated in

TSR received higher grades and higher standardized achievement tests across all subjects,

even though earlier baseline comparison data showed no significant differences. Students

also showed significant positive differences in their Social Emotional Wellbeing scores

and in resilience. Students also gained approximately one year in reading and
 41

approximately a half-year in science and technology compared to students in matched

schools.

A final arts enrichment program that showed strong student achievement growth is

The Byron Study, a five-year partnership between the Byron (Minnesota) School District,

the Minneapolis Institute of Arts, and the Visual Understanding in Education (VUE), a

program that brings the Visual Thinking Skills (VTS) curriculum from the Museum of

Modern Art to schools around the country (Housen, 2002). Byron is a rural community

located 85 miles from Minneapolis. The study used a controlled experimental design that

observed students in each of two age groups, beginning with 2nd and 4th graders.

The VTS curriculum has special class periods for teachers to use works of visual

art and have students grapple with discovering their meaning through such questions as,

“What’s going on here?” and “What more can you determine?” It is both open-ended in

its questioning and discussion formats yet highly structured in the work it explores and the

way the demand for critical thinking skills increases across the five-year trajectory of this

program.

The results of this study showed that the VTS curriculum accelerated aesthetic

growth. Researchers also found that VTS caused the growth of critical thinking and

enabled its transfer to other contexts and content, especially to reading. By the end of five

years, Bryon placed in the top 8 percent of Minnesota schools on the basis of its students’

academic achievement on standardized tests.

Based on the descriptions of these programs, we can get at least some insights into

the cognitive and non-cognitive factors that students are taught in strong arts enrichment

programs. Although Catterall’s research shows correlation, it is difficult to make the

 42

jump to causation with these data. The Song Room (TSR) research and the Bryon Study,

however, provide more insights into what was taught in their programs and what was

learned. In the TSR program, students’ scores on measures of social emotional wellbeing

increased dramatically; therefore, everything else being the same between the test and

control schools, we can make the assumption that TSR may well impact the non-cognitive

factors such as academic mindsets and engagement in school. Lacking a description of

the curriculum, however, there is little more that we can assume.

The Visual Thinking Skills (VTS) curriculum, on the other hand, is quite well

documented. Its primary focus is on helping students develop the critical thinking skills

that allow them to carefully analyze visual works of art through individual work, small

group discussions, and whole class discussions. Students in the Bryon Study, therefore,

were clearly taught many of the cognitive strategies, such as elaboration, generation, oral

production, effort after meaning, and pictorial representation. Whether teachers used

these same cognitive strategies in other subject areas, we simply do not know. It is more

likely, however, that may have happened in Byron, where the classroom teachers were the

primary teachers of VTS.

Research on Non-Substantive Arts Integration Programs

The second model is the non-substantive arts integration model. With this

approach, the arts are used throughout the curriculum, but only in the service of another

discipline, and often as a reward. The arts may be used to demonstrate a student’s

learning in the final product of a project-based learning unit or even as a way to learn. For

example, a student may be asked to draw ten apples, circle two apples at a time, and then

figure out how may groups they have. The arts may also be pulled out at the end of class
 43

as a reward, like, for example, giving students permission to draw on the back of their

tests if they finish early. While this type of arts integration may sometimes increase

student motivation for learning, rarely does that learning include learning more about the

arts themselves.

Non-substantive arts integration programs include those programs which bring the

arts into classrooms in a role subservient to another discipline. The arts integration is

intended to “spice up” the curriculum, helping to engage and motivate students. Many of

these programs, however, are of high quality; the enriched units do provide additional

ways for students to learn the content, and often students’ standardized test scores

increase. One example is the Dallas’ Arts Partners program (Reardon, 2005). Begun in

1998, the Dallas program served nearly all 157 elementary schools by providing training

and funds for work with museums, theaters, and other arts groups. Teachers included

more field trips to cultural sites and invited artists in residence into their classrooms.

Researchers compared ArtsPartner students’ statewide reading test scores, given between

grades 3 and 4, with those of a control group. ArtsPartner students showed a 10-point

gain while the control group gained just 3 points. Similar increases in writing sample

scores were reported for fourth graders. Little more is known about the results from this

intense period of effort and funding. We can only say that it seems the ArtsPartners

program may have some positive impact on literacy skills in the elementary grades.

A second non-substantive arts integration study focuses on a program called

Escher’s World, a 12-hour spring and summer math enrichment program held in a math

studio format at MIT’s Media Lab (Shaffer, 1997). Twelve students explored the

mathematics concepts of mirror and rotational symmetry through short problems and then
 44

long explorations individually, with a partner, and in small groups. Specific attention was

given to helping students form a community of learners through protocols and games.

Although Escher’s name is used for the title of this program, there is no evidence that

students actually studied Escher’s art work during their 12-hour sessions. Instead, they

focused on small projects that incorporated the concepts of mirror and rotational

symmetry using Geometer’s Sketchpad.

Researchers conducted pre- and post-interviews of students, including additional

interviews several months after the completion of the workshop, to determine their

knowledge of mirror and rotational symmetry as well as their affect toward mathematics,

feelings of self-empowerment, control, expression, and their ability to use visual imagery

in solving math problems. Interview data and problem sets were coded for each of these

categories. Based on pre- and post- measures and determining the statistical significance

of the differences, researchers found that students retained their knowledge of symmetry

several months later, that they liked mathematics more and felt more empowered to take

ownership of their learning. They were also more expressive in describing their learning

and problem solving.

Minneapolis’s Arts for Academic Achievement (AAA) program is a much larger

program, serving the entire Minneapolis School District. Its purpose was to transform

teaching and learning through partnerships between schools and arts organizations.

Although the project itself served grades K-12, the achievement study focused only on

students in grades three through five, specifically examining reading and mathematics

achievement (Ingram & Riedel, 2003; Rabkin & Redmond, 2006).

 45

Three sets of multiple regression models estimated the effect of arts integration on

student learning. The first set of models examined the impact of arts integration,

controlling for gender, race/ethnicity, socio-economic status (SES), special education and

English language learners (ELL) status. The second set of models examined the

relationship between arts integration and student achievement for specific subgroups,

including ELL and low SES students. A third set of models examined the interaction

effects. Researchers used gain scores for the experimental group only, a quasi-

experimental design with no control group. Key findings include:

 Third, fourth, and fifth grade reading gain scores were higher for students

whose teachers integrated the arts into ELA lessons.

 For each unit increase in the use of arts integration, students’ gain scores

increased by 1.08 points, 1.32 points, and .71 points respectively.

 This relationship was strongest for low SES students and ELL students.

 The more arts integration, the higher the stronger the impact on test scores.

These findings parallel those of Catterall, who showed the impact arts courses had

on low SES students (Catterall, 2012). As with Catterall’s research, however, correlation

is not necessarily causation; we need to use caution when interpreting these results despite

their potentially positive impacts.

Non-substantive arts integration programs can impact achievement, possibly

through the use of many of the cognitive and non-cognitive factors identified previously,

especially in the areas of engagement and motivation. The non-substantive arts

integration programs with the most clearly identified outcomes, i.e. Escher’s World, are

designed explicitly to impact as many of these factors as possible.

 46

Research on Substantive Arts Integration Programs

Julia Marshall introduced a third model, the Substantive Arts Integration Model,

as a way to make conceptual connections between art and other disciplines (Marshall,

2005). In this model, divisions between disciplines begin to blur as students learn about

both art and another discipline at the same time. For example an English teacher may

have students cite evidence about the qualities of light in one of the Water Lilies paintings

by Monet as they discuss Mary TallMountain’s collection of poems, The Light on the Tent

Wall. Marshall’s belief is that by connecting the arts with disciplines across their

underlying concepts, students both “weave” knowledge, creating nets of cognitive

information, and also “spin” - or generate - new knowledge. Ultimately this leads to

“runaway learning,” in which ideas, knowledge and insights are generated by first

establishing connections and then projecting new concepts (Lakoff & Johnson, 1999).

Interestingly it is this third model, the Substantive Integration Model, that aligns best with

current arts integration descriptions as an approach to teaching in which “students engage

in a creative process which connects an art form and another subject area, and meets

evolving objectives in both” (Silverstein & Layne, 2010). Substantive arts integration

programs are those for which meaningful learning objectives exist for both the arts and for

the other content areas. When possible, the two are taught in tandem, with the arts

enhancing content and the content helping students better understand the arts.

Project CREATES, in Tulsa, Oklahoma, one example of a substantive arts

integration program, began in the summer of 2000 with dual purposes: to integrate the

arts into content areas and to also develop the musical talent of students in low SES

elementary schools (Montgomery, Otto, & Hull, 2007a, 2007b). Teachers integrated the
 47

arts into their content lessons by relying on the community’s resources for artists and field

trips in addition to learning better methods for incorporating the arts into other content

through professional development offerings from local universities and arts organizations.

Research was conducted over a three-year period with a controlled experimental

design at the school and classroom level, matching participating schools and classes with

those who were not participating. They also assessed students’ learning in the arts. While

they found increases in nearly all achievement test results, students who participated in

the talent development portion of the program, that is individual and group music training

in strings or drums, consistently outperformed all other students on all measures of

achievement. At first it might appear that the talent development portion produced this

boost in achievement. However, in addition to these quantitative findings, the researchers

used qualitative observational and student interview findings and coded for such

dispositions as ability to concentrate, feeling like an important member of the school

community, collaboration, sense of accomplishment, etc. They found that all CREATES

students improved markedly in these measures. No significant differences were found

between those who did and did not participate in the talent development part of the

program, meaning that the classroom arts-integration activities appear to have increased

the “academic mindsets” of the students, which are foundational for deep learning to take

place (Farrington et al., 2012).

In a random, experimental controlled study, 1,140 fourth and fifth grade students

were assigned to either a classroom with drama-based instruction in social studies and

language arts or to a regular text-based classroom (Walker, McFadden, Tabone, &

Finkelstein, 2011). Those in with drama-based instruction used the Theatre Infusion
 48

project curriculum, forty lesson plans that were compiled into a handbook called The

Magic Circle of Drama (now out of print). Five of the nine units dealt with historical

topics; the others were intended for languages arts classes. At the beginning and again at

the completion of the experiment, students were measured for pro-social and pre-

cognitive outcomes and attitudes toward the arts in addition to academic achievement.

Statistical analysis was conducted in two stages. The first stage included linear

regression models for academic achievement, multiple regression models for predicting

prosocial and precognitive growth, and an analysis of covariance (ANCOVA) for the

attitudes on the arts. In the second stage, the models were examined separately for each

grade level.

The results showed that the drama-based classrooms consistently and significantly

outperformed the control classrooms on every measure. Students in fifth grade who had

been involved in the drama-based program for two years had the strongest results of all.

The researchers concluded that such arts integrated programs should be considered an

important part of classroom pedagogical strategies, rather than frills. In addition to

strengthening students academically, drama-based programs allow students to “try on”

different perspectives and to work out different ways of approaching and solving issues,

which provide a foundation for prosocial and precognitive behaviors.

The Canadian program Learning through the Arts (LTTA) began in the mid-

1990’s as an artistic outreach program of the Royal Conservatory of Music. It was in part

developed to address the needs of immigrants in learning the languages of their new

country, and help to them assimilate into Canadian culture and their local communities.

Studies of the Toronto pilot determined whether the arts program also impacted academic
 49

achievement and used comparative statistical measures of student achievement in LTTA

schools vs. overall Canadian measures. Researchers also examined attitudes toward

school (Elster, 2001). Later research throughout Canadian LTTA schools examined

academic achievement data as well as student and parent interview data (Smithrim &

Upitis, 2005; Upitis, 2011; Upitis & Smithrim, 2003). The results of a three-year

longitudinal study consistently showed that students did no worse academically, and

interestingly one significant difference was an increase in mathematical computation

achievement. Researchers theorize that students are so much more engaged in their

school work as a result of the LTTA program, that areas where achievement is extremely

dependent on simply paying attention are most likely to show increases (Upitis &

Smithrim, 2003). Parent, student, and teacher interviews were extremely favorable

towards the LTTA program, with families often expressing “a spiritual feeling” towards

the program.

Perhaps the most tightly-knit arts-integrated program is the Waldorf School

curriculum. Students learn to draw in math classes, they sing and dance frequently to

learn reading and writing, and they participate in drama throughout all grade levels of

literacy (Easton, 1997). In order to justify its public funding of Waldorf Schools,

Sweden’s government undertook a research study which used survey data in addition to

student achievement data on national standardized assessments to determine its impact

(Dahlin, 2007). The difficulty with comparing Waldorf students with those in other

publically funded Swedish schools is that the Waldorf program is very different from

Sweden’s national curriculum. In Waldorf schools, students do not begin to learn to read

or do mathematics until they are seven years old. Despite these differences, however,
 50

researchers found that students in Waldorf Schools ultimately caught up with their peers

and did just as well in terms of attending and graduating from universities. Waldorf

students were found to have much more positive attitudes toward each of their school

subjects, however, than their Swedish peers in other schools.

Similar results were found in a study of schools in California and all other U. S.

public Waldorf schools, many of which are charter schools (Larrison, 2013; Larrison,

Daly, & VanVooren, 2012). Researchers compared standardized test data with district

averages across five years, from 2005 through 2011. They found that students’ scores

were higher across most measures in Waldorf schools, but often not statistically higher,

since Waldorf students lag in the early years on standardized measures of achievement

and later catch up. Aggregating data across all grade levels and schools may limit

researchers’ abilities to determine nuances of individual Waldorf school programs that

would help us better understand these differences.

The Milwaukee School District has had a Waldorf School for over twenty years.

While the Milwaukee Waldorf School relies on the standard Waldorf curriculum, it has

modified that curriculum to align better with Wisconsin’s learning standards. Therefore,

it is easier to make comparisons of achievement between this arts-integrated program than

it is with the Swedish Waldorf Schools. Research studies used a matched comparison

model and found that after four years of Waldorf schooling, 63% of students were reading

above grade level, compared with 25% their first year; during the same time period that

these data were gathered, however, scores for matched SES schools in Milwaukee

declined (Byers, 1996; McDermott et al., 1996; Wood, 1996).

 51

Because the substantive arts integration programs are designed to impact both arts

learning and content learning, curriculum designers have taken many cognitive and non-

cognitive factors into account when designing these programs. Additionally, because

their curricula and even videos of classrooms are often available online, there are

opportunities for outside researchers to examine their structures, their learning objectives,

and even sometimes the ways in which the lessons are implemented in real classrooms.

Summary of the Research on Different Arts-Integration Models

It doesn’t appear to matter which category an arts integration program fits into;

instead what does matter for student learning – and transfer to other disciplines – is

whether high-order thinking is emphasized. When it is, as in the Visual Thinking

Strategies program, students are more likely to use higher-order thinking strategies in

subjects beyond the arts. Table 2.1 on the following page summarizes the structures, both

affective and cognitive, shown earlier to be foundational to higher-order thinking.

 52

TABLE 2.1: Affective and Cognitive Factors Foundational to Higher-Order

Thinking

PRESENCE IN AN PRESENCE IN PRESENCE IN A

ARTS A NON- SUBSTANTIVE
ENRICHMENT SUBSTANTIVE ARTS
PROGRAM: ARTS INTEGRATION
The Byron Study INTEGRATION PROGRAM:
PROGRAM: Milwaukee’s
FACTORS IMPACTING STUDENT Escher’s World Public Waldorf
LEARNING: School
NON- Academic Mindsets:
COGNITIVE I belong.
FACTORS:  My effort leads to growth.
+ +
 I can be successful.
 I value my work.
Academic Perseverance √ √
Engagement with School √ √
Motivation + √ √
Paying Attention + √ √
COGNITIVE Rehearsal +
FACTORS: (repeated rehearsal of
information)
Elaboration (semantic
elaboration – adding meaning) + √ √
Generation (generating
information, new or previously + √
known)
Enactment (physically acting +
out material)
Oral Production (producing a
word rather than reading it +
silently)
Effort after Meaning (amount
of effort exerted to get + √ √
comprehension)
Emotional Arousal + √
Pictorial Representation + √ +

Note: Checkmarks indicate areas addressed as in the particular program;

those marked with a + are especially emphasized.
 53

The next section highlights the difficulties inherent in using meta-analysis as a

method of examining arts integration programs. While this research often fails to separate

different types of programs and the many different art forms used in arts-integrated

programs, when carefully examined it does demonstrate that these programs can have

clear, positive impact on student learning.

Meta-Analyses and Compendia of Research Studies

Because the arts have been used in schools for widely different purposes, from

rewards to pre-professional training to inspiration for cross-disciplinary studies, the major

meta-analysis studies on the arts in schools can prove more confusing than illuminating.

These studies do, however, provide an important research base from which we can draw

initial conclusions and begin to formulate questions. Additionally, these studies highlight

the ways in which the focus of research on arts integration has shifted over time to the

point where we now have the beginnings of a much richer and more nuanced

understanding of what works with arts integration programs, and why they increase

student learning in content other than the arts.

Because it is an easy-to-use and popular reference book, Hattie’s Visible Learning

(Hattie, 2009), a synthesis of meta-analyses, provides a good starting place to examine

evidence of the effectiveness of arts-integration programs. Except for drama-rich

language arts programs, which were linked with increases in students’ reading abilities,

and intense music studies, which were correlated to increases in students’ spatial

reasoning abilities, Hattie describes the arts as having only a very small impact on student

learning in other content areas.

 54

Several limitations exist in Hattie’s analysis of the research literature prior to

2009, including the absence of two major meta-analyses, Critical Links (Deasy, 2002) and

Champions of Change (Fiske, 1999), and the lack of the follow-up explorations and

critiques to these research studies, in which the authors began to frame potential

understandings of why some forms of arts integration work extremely well while other

forms produce very small gains in student achievement (Aprill, 2001; Smithrim & Upitis,

2005; Winner & Hetland, 2003, 2008). By failing to include the full range of studies and

later critiques, analyses, and suggestions, Hattie’s simpler analysis suggests that arts

integration programs are nice but not necessary for student achievement. The fuller, more

complete analysis that follows suggests that exactly the opposite is true.

The first and only true meta-analysis of arts integration programs is Conard’s 1992

doctoral dissertation (Conard, 1992), which examines only creative dramatic arts

programs, defined as “an improvisational, non-exhibitional, process-centered form of

drama in which participants are guided by a leader to imagine, enact, and reflect upon

human experience” (McCaslin, 1990) quoted in Conard, 1992). Conard explains that at

the time of her research there were few experimental or quasi-experimental studies of

other types of arts integration, which limited the scope of her meta-analysis.

After selecting creative drama treatment studies based on experimental or quasi-

experimental methods, quantifiable variables, and sufficient data, twenty studies were

included in the meta-analysis, which found a good to excellent effect size on students’

reading scores (.24), vocabulary (.29), oral language (.50), mathematics (.29), writing

(.77), and I.Q. (.86). The impact appeared stronger in pre-school and the elementary

grades than secondary; there were no significant differences between boys and girls,
 55

between regular classes and remedial classes, or between urban, suburban, and rural

schools. The overall effect size for all twenty studies was 0.48.

The characteristics of the creative drama programs clearly associated with student

achievement included who provided the treatment. Effect sizes were larger when the

classroom teacher taught the dramatic arts program than they were when either the

researcher or a drama specialist did. Results were inconclusive about the length of the

treatment program and the type of dramatic arts program provided.

The next major examination of multiple arts programs appeared in 1999 from the

President’s Committee on the Arts and Humanities, Champions of Change (Fiske, 1999).

This report synthesized 143 arts research studies through an eclectic set of categories.

The most useful for this review of literature is the summative research on the Chicago

Arts Partnerships in Education, known by the acronym CAPE, which was implemented

throughout the Chicago Public School District in the fall of 1994 (Catterall & Waldorf,

1999). The CAPE program included visiting artists who worked as partners with regular

classroom teachers to design and implement lessons and units where the arts joined in the

instruction with the academic subjects. Small clusters of schools were invited to apply for

grants to pay for the visiting artists and assist with the support of school-based arts

coordinators.

This is the first published research on a whole district arts integration initiative

with outside evaluators, in this case from the North Central Research Educational

Laboratory (NCREL). It includes a variety of comparisons (including matched

comparisons) between high poverty CAPE and non-CAPE schools, based on the Iowa

Test of Basic Skills (ITBS) and the Illinois Goals of Assessment Program (IGAP) test, a
 56

set of exams reflecting state standards in several subjects and grade levels. Evaluators

found no significant differences between CAPE and non-CAPE schools in student

motivation measures that included engagement, liking school, self-efficacy, and press for

academic achievement. Although CAPE school students were higher in all these

measures, those differences were not statistically significant. Differences were much

stronger between CAPE and non-CAPE schools on standardized tests, however. Very

strong differences emerged by 1998 for both math and reading scores at the 6th grade level

along with more moderate differences in grade 3. High school score differences were not

as large nor as significant, although researchers state that may be because so few high

schools participated in CAPE.

Additional analysis included an examination into specific schools and classrooms

to determine the nature of the arts integrated curriculum when it was succeeding. The

following characteristics emerged:

1. Both the art and academic standards are covered in an integrated lesson.

2. The arts play a true partner role, not a subservient role.

3. Students walk away with big ideas.

4. Students take their work seriously.

5. The experience has a planned assessment with a rubric or scoring guide.

CAPE researchers further noted that schools in which arts integration succeeded

included principals who were supportive, highly skilled artists, and adventuresome, risk-

taking teachers. These differences between successful and not as successful CAPE

schools begin to help determine parameters for effective arts integration in other
 57

programs. Interestingly, although CAPE schools’ standardized tests improved in

mathematics, that was one subject cited as not being totally teachable through the arts.

In 2000 Project REAP (Reviewing Education and the Arts Project) was completed

by Project Zero at Harvard University, and in 2001 a conference on the REAP research

was held at The Getty Center in Los Angeles (Hetland & Winner, 2001; Winner &

Cooper, 2000; Winner & Hetland, 2001). Three meta-analyses were conducted on the

approximately 200 controlled research studies the authors could find, both published and

unpublished, since 1950. Each meta-analysis synthesized a different academic outcome

(composite math and verbal outcomes summed; verbal outcomes; math outcomes). Two

additional meta-analyses were included in the REAP Conference, one which reviewed the

correlation between music and spatial reasoning (Hetland, 2000a, 2000b)and a second that

reviewed dramatic programs’ correlation with achievement in reading (Podlozny, 2000).

The overall meta-analysis found clear causal links between the study of an art

form and learning in other forms, including:

1. the relationship between listening to music and spatial-temporal reasoning,

2. learning to play music and spatial reasoning, and

3. classroom drama and verbal skills.

Areas which showed a slight causal link included:

1. learning to play music and mathematics, and

2. dance and nonverbal reasoning.

Areas where they found no reliable causal relationship were:

1. arts-rich education and improvement on verbal or mathematical scores,

 58

2. arts-rich education and creative thinking scores on assessments of critical

thinking,

3. learning to play music and reading scores on standardized assessments,

4. visual arts and reading, and

5. dance and reading.

The nuances of these findings were then described more completely in the other two

research papers. All other potential causal connections between the arts and student

achievement are described as not (yet) verified.

Hetland’s analysis of the connection between listening to music and spatial

reasoning is described as a highly significant, moderately sized effect. These results

suggested that training in music could possibly result in spatial learning. The second

meta-analysis in this paper confirmed that hypothesis, yet Hetland cautions her readers

against assuming that this might impact students’ success in school; not enough is yet

known to stretch the findings that far. Indeed, Hetland stresses that the dispositions and

concepts that are at the core of music instruction, including understanding melody,

harmony, how to combine timbres or rhythmic variations, etc., are worthy of study in and

of themselves, no matter whether that learning transfers to other disciplines as well.

Polzony’s meta-analysis of REAP’s data on the dramatic arts replicates Conard’s

findings and extends and deepens those findings by sifting the studies through a finer

sieve. She was able to find eighty usable experimental controlled studies, four times more

than Conard, which she then sorted into seven different outcomes: oral measures of story

understanding; written measures of story understanding; reading achievement; reading

readiness; oral language development; vocabulary; and writing. Polzony was also able to
 59

sort the studies more finely by: enactment type (how children enacted the stories, i.e.

through stage presentations or pantomime); the degree of structure in the enactment; types

of populations of students; whether students were tested on the material they studied or on

new material (to determine whether their understandings transferred).

Overall, the effect sizes ranged from modest (.15 for vocabulary) to robust (.475

for written story understanding). The most impressive finding of this study was that

drama not only helps students master the texts they enact, but also helps them become

capable of mastering new material that is not enacted; transfer of skills learned in

dramatic programs was clear and strong. In his critical examination of the REAP studies,

Aprill states that while there is little evidence of “magic transfer” between learning in one

content area and achievement in another, there are important reasons why student

achievement in language arts and mathematics improves when arts instruction increases

(Aprill, 2001).

The problem, however, is not whether the content areas are actually connected or

not (they are), but rather the assumption that that connection is linear – that is, that the arts

is a little lever you press, and out pops out a little knowledge pellet in science or math or

social studies. It is the interaction of and the translation between the arts and language

and mathematics as symbol systems, the mediating between the different domains of

knowledge which generates the learning as authentic intellectual work (Aprill 2001).

Aprill further describes what has a positive impact on test scores: “…not test

prep, but rather authentic intellectual work – requiring students to engage a wide range of

representations and to express themselves in a wide range of forms – that makes the

difference” (Aprill 2001). It would appear that the most successful programs documented
 60

in the REAP research were closer to Aprill’s descriptions than those less successful in

raising students’ standardized assessment scores. Furthermore, Aprill’s explanation of the

interaction between the arts and other disciplines helps explain why test scores in some

disciplines, such as mathematics, improve even when there is little direct interaction with

the arts during math lessons, highlighted in the earlier discussion of CAPE schools.

In a later analysis of REAP’s results (Hetland & Winner, 2004), the authors

describe important shifts that need to happen in both the focus of arts integration research

and its methods. These include focusing on student dispositions such as motivation and

attendance, discovering cognitive bridges between the arts and other disciplines, and

explicitly teaching for transfer. They also recommend that “any study of arts learning,

whether learning in the arts or learning transferred from the arts, should report clear

descriptions of teaching methods, because the characteristics and quality of teaching

certainly affect how well students can use what they learn flexibly and appropriately”

(Hetland and Winner 2004, p. 167).

In 2002, the Arts Education Partnership of Washington, D. C. released Critical

Links: Learning in the Arts and Student Achievement and Social Development (Deasy,

2002). The purposes of this compendium were to bring together research studies that

showed promising inquiry into the academic and social impacts of arts programs and also

to provide those who design arts programs with promising strategies. The compendium

includes approximately 125 summaries of both empirical and qualitative research,

organized by the artistic medium, i.e. dance, visual arts, music, etc. At the end of each

section, an essay by another researcher summarizes the state of that particular medium’s

research.
 61

Although severely criticized for drawing conclusions based on inconclusive

statistics (Winner & Hetland, 2003), Critical Links achieved its stated purpose of

highlighting promising strategies for arts programs, even though many of these strategies

were not yet based on a firm empirical footing. Every research study in Critical Links

includes a summary about the importance of how individual teachers implement each

program. In his concluding essay, Deasy highlights the need for a better understanding of

both the neurological foundation of what happens when students participate in arts

programs, and a broader view of what transfer is and does, pushing for transfer to include

habits of mind and dispositions that may ultimately lead to stronger academic outcomes

for students. This aligns with Winner and Hetland’s statement:

It is important to understand just what kinds of cognitive, affective, and social

skills are taught and learned in arts classes. Are these the same kinds of skills
learned in academic courses? Are some unique to the arts? If we can identify the
cognitive and social skills taught in arts classes, especially those taught uniquely
or especially well, we will have built a strong argument for the importance of arts
education that does not treat the arts as handmaidens to reading, writing, and
arithematic (Winner & Hetland, 2003).

By 2007 arts education theory had shifted more completely toward Deasy’s

request for better understanding of how teachers implement arts programs. Burnaford’s

literature review of arts integration brought together many of the same studies as the

previous meta-analyses and compendia listed above, but also includes numerous

qualitative studies and UNESCO’s survey/case study analysis research on global arts

integration practices (Bamford, 2006) that attempt to address teachers’ implementation of

arts programs. Findings concluded that arts-rich programs were seen primarily in high-

quality programs with teachers who are well trained and working in schools with

administrative support.
 62

In 2007 arts education researchers also began more definitively to examine how

the delivery of the arts varied from program to program (Gullatt, 2007) and emphasized

the importance of determining habits of mind and dispositions toward learning that may

be enhanced through the arts but that can be elusive to quantitative measures (Russell &

Zembylas, 2007). In this model, arts programs influence students’ dispositions and/or the

methods they use to learn, which then in turn impact their ability to learn more

successfully in another, non-art field of study. During this same year, REAP’s authors

described the need for arts educators to truly understand the specific types of thinking

developed through the study of the arts (Hetland, Winner, Veenema, & Sheridan, 2007).

As they state, “Only when we have determined and can document levels of what students

actually learn when they study an art form does it make sense to look for transfer of that

learning to other subjects” (Hetland, Winner, et. al. 2007 p. 4).

Identifying specific neurological links was the purpose behind the Dana

Foundation’s 2004 consortium on the arts and cognition as well as its 2008 report of

research findings (Gazzaniga, 2008). The difference between this and previous research

is that the Dana Foundation brought together neuroscientists rather than arts educators.

Important conclusions from these studies include:

1. Interest in performing arts leads to a high state of motivation which in turn

produces the sustained attention necessary to improve in other areas of

cognition.

2. Specific neural links exist between high levels of music training and the ability

to manipulate information in both working and long-term memory; these links

extend beyond the domain of music training.

 63

3. Correlations exist between music training and both reading acquisition and

sequencing

4. Phonological awareness, a central predictor of early literacy, is correlated with

both music training and the development of a specific brain pathway.

5. Learning to dance leads to more effective observational learning which may

transfer to other cognitive skills.

While research in the area of arts and cognition continues, the author emphasizes

that a “life affirming dimension is opening up in neuroscience: to discover how the

performance and appreciation of the arts enlarge cognitive capacities will be a long step

forward in learning how better to learn and more enjoyably and productively to live”

(Gazzaniga, 2008).

Mariale Hardiman’s work forms a bridge between cognitive science and

pedagogy. Her research focuses on how the arts teach students to learn strategies that

help them remember information and processes in any domain (Hardiman, 2003, 2010;

Rinne et al., 2011). The Brain-Targeted Teaching (BTT) Model assumes that students

need content and procedural skills, often as prerequisites of conceptual understanding.

Hardiman stresses that integrating artistic activities into instruction is likely to enhance

long-term memory for content, because students are actively using seven cognitive

strategies that lead to long-term memory. These include: repeated rehearsal (practice);

elaboration (elaborating on information rather than just repeating it); generation (creating

new information); enactment (physically acting out something); production (words are

produced aloud rather than silently read); effort after meaning (grappling or struggling to

make meaning of something); and pictorial representation (Hardiman, 2012). Because the
 64

arts naturally require students to use these cognitive strategies, students involved in arts

programs use the same strategies in learning the skills and processes of other disciplines

as well.

Camille Farrington’s research focuses on the non-cognitive factors that establish a

foundation for academic learning (Farrington et al., 2012). Factors she has found

contribute most substantially include: academic mindsets (“I belong”; “My effort leads to

growth”; “I can be successful”; “I value my work”); academic perseverance; engagement

with school; motivation; and paying attention. As we saw earlier, many of these non-

cognitive factors (“habits of work,” or “dispositions”) have also been identified as what

students learn from the arts.

Summary of the Research on Arts Integration

It is clear from the research on arts integration that the arts have a positive impact

on student learning, as measured by standardized assessments, yet it has been very

difficult for researchers to determine precisely what it is about the way the arts are

integrated within these more successful programs that creates their positive impact on

student learning. During the past two decades, scientists have made tremendous strides

in determining what happens physically when people learn. We now know that the

amygdala, a walnut-sized part of the brain located near the spinal cord, makes “fight or

flight” decisions from all sensory input, including making the decision to “stay and

struggle to learn” or to “opt out” (Zull, 2002). Thus, Farrington’s work on developing

warm, welcoming school communities that help students develop strong socio-emotional

bonds with each other and with their teachers, is critical for significant learning to take
 65

place. The arts are often emotionally rewarding and motivating, especially in conjunction

with other disciplines; thus, the arts provide multiple opportunities to build up students’

connections to school (Upitis, 2011).

Brain cells grow and build connections every time students work hard to learn.

Mariale Hardiman’s research highlights numerous ways the arts allow students’ brains to

hold on to knowledge, including facts and procedures that become critically important

when deeper thinking is required (Rinne et al., 2011). Thus, the arts can be used to assist

students to retain what they’ve learned.

Finally, despite the fact that deep learning can be a struggle, humans appear to be

hardwired to do precisely that kind of struggling. Our brains actually progress through a

“scientific inquiry cycle” when integrating new knowledge with previous knowledge

(Zull, 2002): We must first become engaged with a problem or question; we then proceed

to concrete experience and reflective observation; next we attempt to explain what we’ve

discovered through a hypothesis; we then test the hypothesis in order to evaluate whether

it holds up, and we begin the cycle again. Given a warm socio-emotional environment

along with adequate background knowledge, the arts are uniquely situated to both

challenge and inspire students to do this kind of deeper thinking.

As summarized in Table 2.1 (see p. 61) those arts-integrated programs that moved

students to stronger academic learning included many of Farrington’s and Hardiman’s

elements for socio-emotional well being and the retention of knowledge. Once these

elements are in place, students can then be challenged to think more deeply.

Unfortunately, the only arts-integration program documented thoroughly enough to see

that push toward higher-order thinking is The Museum of Modern Art’s Visual Thinking
 66

Skills program (Housen, 2002), which uses a very open-ended questioning method

coupled with an intense focus on using evidence to support hypotheses; both these

methods are recommended for building students’ higher-order thinking skills (Resnick,

1987b).

Thus, an arts-integrated mathematics program that helps create the foundational

socio-cultural elements, uses a variety of arts-integrated methods to help students retain

their knowledge, and which also presents students with challenging, perplexing and non-

routine problems would be ideal for developing higher-order thinking skills in

mathematics. This is exactly what the Beautiful, Beautiful Math program attempts to do.

Implications of the Literature on My Design of Beautiful, Beautiful Math Lessons

The research summarized in this chapter has helped to inform my thinking on the

relationship between Beautiful, Beautiful Math (BBM) and the development of higher-

order thinking skills. To better understand the cognitive underpinnings of higher-order

thinking, I have drawn on Hardiman’s (2012) Brain-Targeted Teaching (BTT) Model and

Webb’s Four Depth of Knowledge (DOK) levels (2002). I have, however, also drawn

from Farrington’s (2012) research, which focuses on those affective factors that contribute

most substantially to helping students to establish a foundation for academic learning and

the use of higher-order thinking skills, and which, incidentally have also been identified

as the “habits of work” or “dispositions” that students learn when they are engaged in the

arts (Smithrim & Upitis, 2005). These are also listed in Table 2.1 on p. 61.

Hardiman assumes that students need content and procedural skills, often as

prerequisites of conceptual understanding and the development of higher-order thinking

 67

skills. Her research suggests that integrating artistic activities into instruction is likely to

enhance long-term memory of content because students are actively using the seven

cognitive strategies that lead to long-term memory. These strategies are listed in Table

2.1 as well. Tasks that require students to use more of these cognitive strategies and that

have built in more of the foundational, non-cognitive factors appear to lead to stronger

student learning outcomes, as Table 2.1 (p. 51) highlights for three such programs

(Dahlin, 2007; Housen, 2002; Shaffer, 1997).

The Visual Thinking Strategies (VTS), described in the Byron Study, are designed

to elicit higher-order thinking about works of art through the open-ended questioning

cycle of: “What’s going on here?” “How do you know that?” and “What else can we find

out?” These three simple questions are used at the beginning of each BBM lesson, at the

point when students are first examining a work of art. They will also be used to set the

stage for an inquiry-based math lesson.

Expeditionary Learning’s Workshop 2.0 lesson format, used in the sample BBM

lesson in Chapter One, will provide a structure to support the inquiry-based lesson. In a

Workshop 2.0 lesson, the teacher follows these steps:

1. 2-minute Grapple – during this time the students work independently to solve a

problem that is beyond anything they’ve learned up to that point; the teacher

circulates but must remain completely silent.

2. 2- minutes Share – students share their ideas with a partner; the teacher circulates

but continues to remain silent.

3. 8-minute Focus – the teacher now presents a mini-lesson, often relying on

students’ work and discussions.

 68

4. 25 – minutes Application – students now work independently or with a partner or

small group to apply what they’ve just learned to one or more problems; this is a

time for practicing what they’ve learned. The teacher can now confer with

students or even pull the class together in a “catch and release” style mini-lesson.

5. 5 – minutes Synthesis – the teacher, often with much assistance from students,

recaps the learning for the day, reminding everyone of its purpose and importance.

Because BBM tasks are designed to have students use most of Hardiman’s

cognitive strategies, to work collaboratively to develop Farrington’s non-cognitive

factors, and to demonstrate observable behaviors reflecting Webb’s Depth of Knowledge

at levels 3 and 4, teachers who effectively incorporate BBM into their teaching should

increase their capacity to develop higher-order thinking skills in their students. My study

will investigate this proposition, and use coaching and classroom observations to

determine the kinds of supports teachers need to be able to implement BBM with integrity,

in order to achieve the desired outcomes.

 69

CHAPTER THREE: METHODOLOGY

Introduction

This chapter will articulate in detail the design of the study, which has been

informed by an action research paradigm. It begins with an explanation of why action

research is the chosen methodology for the study and why the proposed research reflects

the tenets of action research. It then provides a brief overview of the study and a

description of its context. Next I provide the details of the research plan, including

participants’ recruitment, the proposed intervention, and the plans for data collection and

data analysis. Finally, the chapter ends by circling back to the tenets of action research

and how those tenets were exemplified in my study.

Action Research

Maas (1997) explains that action research is self-directed, is about building

professional relationships, is systematic, and is ongoing (Maas, 1997). Action research is

a living, evolving process rooted in everyday experience, and is less defined in terms of

hard and fast methods. “It is a verb rather than a noun” (Lyotard, 1979; Reason &

Bradbury, 2008). As the researcher develops the skills of such inquiry, action research

emerges over time in an evolutionary process (Reason & Bradbury, 2008).

Although action research is often described metaphorically and somewhat

differently depending on the field and specific focus of the researcher using it, the

underlying process remains largely the same. First, action research is cyclical in nature,

something that sets it apart from other methodologies. The simplest form of this cycle is
 70

“look, think, act” (Stringer, 2007). Kemmis & McTaggart describe a spiral of Plan-Act

& Observe-Reflect (Kemmis & McTaggart, 1988). Herr and Anderson describe the cycle

this way:

1. develop a plan of action to improve what is already happening,

2. act to implement the plan,

3. observe the effects of action in the context in which it occurs;

4. reflect on these effects as a basis for further planning, and then begin the cycle

again (Herr & Anderson, 2005).

Action research stands apart from other research methods, because it is through

the ongoing process of data gathering and analysis that the researcher has the flexibility to

revise the research plan throughout the study. Another important difference is that in

action research, the researcher’s orientation toward the problem is vitally important:

“Action research does not start from a desire of changing others ‘out there,’ …rather it

starts from an orientation of change with others” (Reason & Bradbury, 2008). The

researcher is a participant, too, and is expected to learn, to grow, and, ultimately, to

change his or her actions, ideas, and opinions.

When action research analysis is written, it may take the form of a narrative, more

of a story than a report of research findings. A narrative format provides space for the

researcher to reflect on the research process as well as the findings (Herr & Anderson,

2005). Nel Noddings (1991) explains, “Stories have the power to direct and change our

lives” (Noddings, 1991). The purpose of action research is to change the lives of both

researcher and participants; narrative allows these changes to be documented and shared.

Narrative provides a way of representing research results that enhances their usability
 71

(Argyris & Schon, 1989). Herr and Anderson describe how having a vicarious experience

through story enables readers to develop a naturalistic form of generalization; that

generalization then leads to their personal understanding and an internal conviction that

culminates in action (Herr & Anderson, 2005). The ultimate test of an action research

study is whether its results are actually used. The most important question is: “Have we

authored our work in such a way that lives have changed for the better, most importantly,

the lives of children…?” (Carter, 1993).

My research study adhered to the tenets of action research described above by:

being participatory, using multiple sources of evidence, being values-oriented while

seeking to address a significant issue, and becoming a living, emergent process. One of

the major concerns I had grappled with in my doctoral work was how my research would

impact the work I do, specifically the work I do with teachers trying to adopt new

pedagogical practices. Action research is particularly useful when the researcher’s

primary intentions are to add to the body of practical, craft knowledge, often specific to

the researcher’s own profession, rather than to the body of research defined by the

academy. Because the data are frequently probed to help determine next steps, it can

provide immediate information that can be utilized right away; a more traditional or

formal type of research may not have an impact on practice for many months or even

years (Richardson, 1994).

The intervention in this study asked teachers to do something radically different in

terms of their mathematics teaching, namely to use an object of art to introduce new

mathematics skills and concepts. Therefore, to expand the use of this approach it was

vitally necessary to show clearly how it increases students’ use of higher-order thinking,
 72

which in turn has been shown to directly influence student achievement outcomes. It was

especially important to me that my research serves some purpose in helping to improve

the educational practices of the teachers with whom I work, as each of these teachers may

well impact the education of between 500 (for elementary) and 4,000 (for secondary)

students over the course of their teaching careers.

One of the strengths of the action research methodology is that its cycles of

planning, action, and reflection integrate action and knowledge, thus avoiding the ‘gap’

between knowing and doing that befuddles so many change efforts (Reason & Bradbury,

2008). Action research also has the advantage of allowing the researcher to use the

workplace rather than examine a new place just for study. Because action research is

localized, it can be nimble in altering the study as it is undertaken. The purpose of action

research is to produce practical knowledge, knowledge that is helpful to people in their

everyday lives (Reason & Bradbury, 2008).

Additionally, positivist claims of “generalizability” are usually not the point of

action research. We need to remember that objectivity itself is a bias, and that every

method comes with its own set of biases (Remen, 2007). As today’s plethora of positivist

research clearly show, objectivity often creates a bias called “the illusion of causality,” the

belief that correlation necessarily implies causation (Crites, 1986). Instead of

generalizability, action researchers strive for authenticity, meaning that the findings of a

research study not only ring true for others in the field, but also allow people to act

creatively when faced with practical issues in their lives (Reason & Bradbury, 2008).

My research focused on examining the key design features needed for the BBM

approach to elicit higher-order thinking from students, including the decisions made about
 73

lessons with teachers, and the outcomes of teachers’ actions as they were manifested in

new lessons. By deliberately and systematically collecting and analyzing my data and

frequently reflecting on my findings with the assistance of the participating teachers, I

believe I have become the kind of research professional Schon calls a “reflective

practitioner,” one who “learns to learn” about my practice (Schon, 1983).

Research Context

In developing and implementing the BBM lessons studied in this dissertation, I

worked with two schools: Brick School, a small independent school, and Engineering

High School, a public school within a large urban school district and a part of the

Expeditionary Learning network. Both schools are located within the boundaries of the

same large urban public school district. The following sections describe each of these

schools, the reasons why I chose them, my previous relationships with each school, and

how each context influenced my study.

Brick School

Brick was a small school; its overall population in grades K – 6 was just 46

students at the time of this study. It was founded in 1984 as an alternative to schools with

rigid curricula and testing requirements. Located in the city’s arts district, tuition was

approximately $12,000 per pupil per year; only about 10% of its students are from

minority populations. Its history of involvement in the arts and project-based learning

made it an ideal location to pilot different ways of teaching; additionally, because it did

not administer any standardized tests, teachers were not pressured to “cover the

curriculum” or to spend class time preparing for tests and then administering them.
 74

When I moved to the city in which Brick is located eighteen years ago, I substitute

taught at Brick for a couple of months and found it to be a warm, welcoming place with a

dedicated teaching staff who focused closely on the needs of each individual child. When

I returned eighteen months ago to do an audit of its math program, the lead math teacher

expressed a strong interest in Beautiful, Beautiful Math. We continued our conversations,

and he piloted a BBM lesson a year ago.

I believe Brick School provided an environment away from pressures of

standardized testing and curricula that helped me determine the key design components

for BBM in a nearly ideal elementary school environment. Because students were used to

using art throughout all their studies, bringing objects of art into math lessons did not feel

strange to them. Because I had worked with the lead math teacher over the past year and

a half, we had already developed a good working relationship by the time this study

began.

Engineering High School

Engineering High School completed its grow in the fall of 2015 and now has

students in grades 7 – 12. It is located in one of the poorest neighborhoods of the city;

nearly 90% of its students are eligible for free or reduced lunch. Of its population, 55% is

Hispanic, primarily Puerto Rican and Dominican, 35% percent African American, and

10% white. Approximately 20% of its students have identified special needs.

I was instrumental in establishing Engineering High School with a former District

Superintendent and its Board of Education. The Bill and Melinda Gates Foundation

provided a grant to start the school as an Expeditionary Learning School. It also received
 75

a very large grant from New York State, which was used to purchase technology,

including tablet computers for every student.

I have worked with many of the mathematics teachers at Engineering in past years

and have established a solid working relationship with them. They and some of the newer

teachers, whom I have not worked with, have been part of an Expeditionary Learning

Western New York Math Cluster during the months of January – March, 2015. Because

of their involvement in this cluster, I’ve been able to work closely in designing lessons

with both the new and the veteran Engineering High School teachers. I believe that

having already established a solid working relationship with these teachers proved helpful

in recruiting them and working with them in this research study.

Researcher’s Positionality

I had two roles in this action research study. I was both the researcher and a

participant, as the co-designer of the BBM lessons. Although I observed and videotaped

these lessons, I did not directly teach students. Instead, I tried to discern, sometimes by

myself and sometimes in partnership with the teachers, what the key design elements are

that make BBM lessons successful in having students use higher-order thinking. As the

co-designer of these lessons, however, I am an insider in the study. Herr and Anderson

(2005) explain that this is important because, “ …the degree to which researchers position

themselves as insiders and outsiders will determine how they frame epistemological,

methodological, and ethical issues in the dissertation.” They also caution that it is not

easy to define one’s precise position. Throughout this study, I worked on carefully

determining what my position within the study was, and just as importantly, how my
 76

position as an insider was influencing my research. Additionally, as a co-designer, I was

able to use directly what we learned from the action research to improve the design of

later BBM lessons.

Additionally, two months prior to my dissertation research, both of the high school

teacher participants in this study were enrolled in my three-day professional development

workshop on Beautiful, Beautiful Math at the Memorial Art Gallery. During that

workshop, they had ample time to examine works of art in person; this helped them

develop their ability to both examine a work of art and find potential mathematical

connections within it and also to find a work of art that embodies a particular mathematics

content standard.

Research Design

As mentioned in Chapter One, my overarching question was: “How can objects of

art be used as effective catalysts for higher-order thinking in mathematics lessons?” To

answer this question, I articulated three research sub-questions:

1. What do exemplary Beautiful, Beautiful Math (BBM) lessons look like?

2. To what extent do BBM lessons result in higher-order thinking in

mathematics?

3. What are the key design features and other implementation factors that need to

be in place to increase the potential of BBM lessons to have the desired

outcome?

This research study was divided into three stages. First I determined the baseline

higher-order thinking that was apparent in each teacher’s classroom in a typical, non-BBM
 77

math lesson, through the use of the observation tool (PAQT, see Appendix A) combined

with videotaping. Secondly, I initiated a first action research cycle with each teacher,

collaborating with the teacher to design a BBM lesson and then observing the

implementation of that lesson, using the PAQT observation tool along with videotaping

the lesson as the teacher implemented it. This first action research cycle concluded with a

post-lesson interview where I reviewed the PAQT scores with the teacher, to begin the

process of identifying key elements of BBM lessons leading to higher-order thinking, and

to determine our next steps. Finally, I repeated the same action research cycle, starting

with the design of a second BBM lesson. I completed this cycle once with each of the

three teachers. Finally, I conducted a final post-study interview with each teacher to elicit

their insights about my preliminary findings.

Participant Recruitment

My recruitment plan was to personally invite teachers I had worked with in the

past to consider becoming participants in my research. I asked a few teachers who had

shown an interest in adopting innovative teaching practices and whose classrooms were

relatively free of student disruptions. Through this process, I was able to recruit three

teacher participants for my research study. All potential participants were sent an

information letter explaining the study and inviting their participation; this letter informed

them that their participation was voluntary and that they were free to cease participation at

any time or to not participate in the study altogether, thus ensuring that all Human

Subjects regulations were met.

Precautions were put in place to safeguard the identities of all subjects, including

the removal of text related to teacher identities from all data sources; I used pseudonyms
 78

in place of actual identities, and did not publish any identifying information. I was the

only person with access to any identifying information and did not disclose names, e-mail

addresses, telephone numbers or any other identifying information. All identifying

information was stored on a password-protected computer which only I could access.

Brief Descriptions of the Three Teacher Participants

George is a white male in his mid-thirties who has been teaching elementary

school for about ten years; he had been at Brick School for eight years at the time he

worked with me. George taught mathematics to all students except those in kindergarten.

Yvette, a white woman in her early thirties, has taught math for ten years, first in

middle school and now in high school. I met Yvette several years before this study, when

she became one of the founding teachers of Engineering High School. She applied to

become part of that effort because she is fascinated with educational technology and very

much appreciates having access to a wide array of technological teaching tools along with

the administrative support to learn about them and use them extensively and effectively in

her teaching. Yvette is an immigrant from Russia. Her family moved first to Turkey,

where she spent a few years of her early childhood, and later to the United States.

Because she struggled with reading and writing English, Yvette found mathematics to be

a refuge. Yvette’s background has given her keen insight into what it means to be an

outsider; additionally, being an immigrant herself, she has proven to be exceptionally

helpful with the many immigrant and English-as-a-second-language students in her

school.
 79

Karl is a twelve year veteran mathematics teacher, white and in his late thirties. I

first met Karl two months before the start of this research project. He was enrolled in a 3-

part workshop on Beautiful, Beautiful Math, which another school designer and I held for

Expeditionary Learning at Rochester’s Memorial Art Gallery. Karl taught in the same

school as Yvette, just across the hall. Yvette recommended the BBM workshop to Karl,

since she had participated during the previous year.

Description of the Intervention

As explained in the Research Design section, the implementation of my study took

place in three stages with individual teachers:

1. Determining the baseline of higher-order thinking in each classroom

during a typical lesson;

2. Running at least two action research cycles of Beautiful, Beautiful Math

lesson planning, implementation, and reflection;

3. Reviewing the preliminary results of exemplary BBM lesson design

elements that appear to lead to higher-order thinking through the

interviews with each teacher.

For the first stage, I used the Protocol for the Assessment of Quality Teaching, or

PAQT, observation tool (Gallagher, 2013), which tracks student engagement, the level of

cognitive rigor in the tasks students are working on, and the use of academic vocabulary

(see Appendix A). This provided me with a baseline measure of the higher-order thinking

taking place during each teacher’s typical lessons. In addition to using the PAQT

instrument, I videotaped these classes and kept copies of the lesson plans. The video
 80

recordings, lesson plans, and comments section of the PAQT observation tool provided

me with a record that I returned to when I had questions about PAQT results; they also

provided a resource for discussions with teachers.

I shared the PAQT scores of their baseline lesson with each teacher as we moved

into planning the Beautiful, Beautiful Math lessons. This allowed them to see where their

typical lessons were strong and where there were challenges in having students engaged

in higher-order thinking. We then collaborated to design a first BBM lesson. I audio

recorded all our meetings to ensure a record of what transpired.

The teacher then implemented the first BBM lesson while I again used the PAQT

tool and video recording to create a record of the implementation. The teachers and I then

met and reflected on the implementation of the lesson, specifically looking for the key

design features that seemed lead to higher-order thinking. I used an interview protocol

(see Appendix B.1) to begin this reflection process, initially relying on and then

ultimately building out from the teacher’s own observations of what did and did not work

to increase students’ higher-order thinking.

The planning of the second BBM lesson with each teacher built on what we

learned through this process. During the implementation of the second lessons, I once

again used the PAQT tool and video recording to create a record of the implementation.

Afterwards, the teacher and I again reflected to identify those key design elements that led

to higher-order thinking, using the same post-implementation protocol.

A Student Survey, to determine what the students thought about the Beautiful,

Beautiful Math lessons, was co-created with the two high school teachers at Engineering

High School and then designed in a Google form by two of Karl’s students. George, the
 81

elementary teacher at the Brick School, also co-created a survey with me and ran it off on

paper for students to take during class. Both surveys were administered about a month

after the completion of the final Beautiful, Beautiful Math lesson.

Finally, I again scheduled an interview with each individual teacher to share the

findings from the overall study and to elicit each teacher’s impressions about these

findings along with other insights they may have (see Appendix B.2 for the Final Teacher

Interview Protocol).

Table 3.1 on p. 80 includes the full timeline for conducting the research and lists

by teacher the dates for every planning session, implementation of the lessons, and lesson

debriefing meeting, as well as the date we met for the final interview.
 82

TABLE 3.1: Research Study Timeline

GEORGE
DATE LESSON DESCRIPTION
3-27-15 Baseline Focused on lines of symmetry; students cut their own
symmetrical image from a folded sheet of paper, grade 4
4-14-15 Planning for grades 5-6 – Focus on volume, ratios, & modeling
4-27-15 Planning – Focus on volume, ratios, & modeling
5-7-15 Pre-Lesson – Background on Sculpture; not counted since this
BBM #1 lesson focused solely on art
5-8-15 Lesson at the MAG: Creation Myth Sculpture
5-11-15 Follow-Up to the MAG; not observed or recorded
5-14-15 Follow-Up to the MAG; data from field work used to calculate
volume & weight of sculptures using density tables
5-14-15 Post-Lesson Interview
5-14-15 BBM #2 Planning for fractions lesson with grades 1 & 2
5-27-15 Lesson at the MAG; students used “fraction lenses” to view art
5-29-15 Post-Lesson Interview
8-21-15 Final Interview; reviewed preliminary analysis

KARL
3-26-15 Baseline Focused on absolute value functions & inequalities with Algebra
2 class
4-27-15 BBM #1 Planning; focus on arithmetic sequences
4-29-15 Lesson in classroom with Algebra 2 students, both non-regents
and honors classes
4-29-15 Post-Lesson Interview
4-29-15 BBM #2 Planning; focus on binomial probabilities with Algebra 2 students
– both non-regents and honors.
5-8-15 Lesson at the MAG
5-13-15 Post-Lesson Interview
8-3-15 Final Interview

YVETTE
3-26-15 Baseline Focused on trigonometric identities
4-16-15 BBM #1 Planning; used Crested Swans to focus on transformations of
functions with pre-calculus class
5-5-15 Lesson in classroom
5-6-15 Lesson, part 2
5-6-15 Post-Lesson Interview
5-6-15 BBM #2 Planning for golden ratio lesson using Till Freiwald’s Untitled
Portraits with pre-calculus class
5-8-15 Lesson at the MAG
5-12-15 Post-Lesson Interview
7-31-15 Final Interview; reviewed preliminary analysis
 83

Data Collection

To address the research questions, I collected and analyzed seven data sources:

lesson plans; audio recordings of my BBM planning sessions with teachers; video

recordings of lessons; PAQT Observation records; audio-recordings of post-lesson

implementation interviews with teachers; audio-recordings of final teacher interviews;

and my researcher’s journal. Data collection was ongoing throughout my collaboration

process with each teacher, as was the analysis of that data (see Appendix D for a table

summarizing my data collection and analysis plan).

Each of the data sources was deliberately chosen to provide information about

specific aspects of my three research questions. Following is additional information about

each data source.

Lesson plans were the written product of my collaborative lesson planning work

with each teacher. As such, each lesson plan provided the blueprint for the BBM lesson as

well as a resource to return to as the teacher and I searched for the key design features that

led to higher-order thinking. Therefore, lesson plans were especially useful in answering

research sub-question 1, describing what exemplary BBM lessons look like, and 3,

identifying the key design features that led to higher-order thinking. The lesson plans also

served in a more limited way to address research sub-question 2, in that they helped

identify sections of BBM lessons most likely to result in higher-order thinking.

Audio recordings of my planning sessions with teachers provided a record of

our interactions and thoughts as we collaborated to design a BBM lesson. I referred back

to these recordings to determine the decisions made about each part of the lesson and how

we felt those decisions would lead to students’ higher-order thinking. These audio
 84

recordings were especially useful in identifying the key design features in research sub-

question 3.

Video recordings of lessons provided a record of the implementation of each

BBM lesson. They were useful in showing what exemplary BBM lessons look like,

thereby answering research sub-question 1. They also provided a record which the

teachers and I could refer back to as we analyzed their PAQT scores during our post-

lesson interviews; I also referred back to the video recordings to help verify and elaborate

on the PAQT scores needed to answer sub-question 2. Finally, I used the video

recordings to help determine the key design features necessary to ensure higher-order

thinking, thus helping to answer sub-question 3.

PAQT observation records were the critical component in determining the level

of higher-order thinking. Gallagher’s PAQT observation tool (see Appendix A) has

correlated three observable measures - student engagement, Webb’s depth of knowledge

of the task (cognitive rigor), and academic vocabulary - with student achievement

outcomes. Each of these three measures is a proxy for teacher actions and learning that

have taken place over a long period of time prior to the observation:

 Student engagement is a proxy for how well the teacher has established a strong

culture of learning and caring;

 Webb’s depth of knowledge measure is a proxy for teacher content knowledge;

 Academic vocabulary is a proxy for the teacher’s pedagogical practices.

Gallagher found the correlation between PAQT scores and standardized test scores

was especially strong in mathematics; one standard deviation in the PAQT scores resulted

in an 11-point gain on the California Standards Test (Gallagher, 2013). These three
 85

observable measures combined to determine how strongly students were using higher-

order thinking during a particular lesson. Thus, a class using higher-order thinking will

be actively engaged in a rigorous task, and they will be sharing their insights and ideas

with each other as well as their teacher.

The PAQT observation scores were instrumental in determining which lessons to

highlight as exemplary for research sub-question 1. Additionally, the PAQT records

provided the evidence that determined the extent to which implementing BBM lessons

actually impacted students’ higher-order thinking, which is at the heart of research

question 2. Finally, because the PAQT tool breaks each lesson up into ten equal parts for

scoring and note taking, these records were useful in determining which lesson segments

were especially strong in fostering higher-order thinking, thereby helping the teacher and

I in identifying the key design features that led to higher-order thinking, sub-question 3.

Audio-recordings of post-lesson implementation interviews with teachers also

assisted in answering each of the three research questions. The teacher’s perspective

about whether and where higher-order thinking occurred in his/her lesson was important

for understanding how exemplary lessons were likely to be viewed by other teachers, and

how well the teachers themselves could identify the extent to which higher-order thinking

changes through the implementation of BBM lessons. These interviews were especially

critical in my search for key design features and the other implementation factors needed

to ensure that BBM lessons are successful, sub-question 3.

Audio-recordings of final teacher interviews, conducted after all lessons were

completed and after my initial data analysis, provided the teachers with a final say and the

opportunity to voice ideas they developed after my last interaction with them. These
 86

interviews also helped to validate, expand, and articulate my preliminary findings,

especially on the topic of key design features (research sub-question 3). During these

interviews, I also shared my preliminary findings about the key design features; having

this feedback was essential in supporting and refining my claims.

Students’ Survey Responses captured student perceptions about how effective

the BBM lessons had been for them, thus helping to answer each of the three research sub-

questions. The survey questions are included in Appendix C.

My researcher’s journal was the last of my seven data sources. My journal

included research memos, field notes, e-mail, notes about other correspondence, and notes

on conversations. It documented my own transformations in understanding, and it served

as a place to capture my hunches, preliminary findings and hypotheses. It also helped me

document emerging concepts and themes, which assisted in subsequent data analysis

(Charmaz, 2006). Finally, my researcher’s journal served as a source of verification for

the authenticity of my data, helping me insure the validity and trustworthiness of my

findings, an area discussed in more detail later in this chapter.

During my action research process, I not only studied my participants as I

collaborated with and observed them, I also included them in checking my emerging

ideas, patterns, and findings, most notably in the post-study interview.

 87

Data Analysis

To answer my first research sub-question, What do exemplary Beautiful, Beautiful

Math lessons look like?, I first created a narrative that described each of the BBM lessons,

along with each teacher’s experiences, as a way to provide more contextual details than

could be included in a lesson plan. An important intermediate step in creating this

narrative was merging the lesson plan and PAQT observation records of each BBM lesson

into one integrated document – the annotated lesson plan document reported in Appendix

F. This document provided me the ability to examine each aspect of the lesson, as it was

originally planned, against its actual implementation with students. With this insight, I

then determined the commonalities of those lessons with the highest PAQT scores, in

order to develop a richer understanding of what exemplary Beautiful, Beautiful Math

lessons look like.

Research sub-question 2 was: To what extent do Beautiful, Beautiful Math lessons

result in higher-order thinking in mathematics? To answer this question, I again relied on

the PAQT scores to determine whether there was a shift in higher-order thinking from the

PAQT observation records from the baseline lessons to the BBM ones for each teacher.

Additionally, I used the lesson plans and my researcher’s journal to supplement the PAQT

observation records; this allowed me to be certain that I had not mis-recorded any parts of

the PAQT form.

I also relied on the teachers’ post-lesson implementation interviews to elicit

teachers’ impressions about the extent to which their students engaged in higher-order

thinking during each BBM lesson; again, these impressions along with the annotated

lesson plans and my researchers’ journal allowed me to identify specific lesson segments
 88

that best illustrated students’ engagement in higher-order thinking and to assess the extent

and nature of that engagement. I then reviewed the video segments that matched those

areas of higher-order thinking to complete an in-depth analysis of those specific segments,

to identify the learning opportunities offered students and how students took on those

opportunities. Finally, I reviewed my entire analysis of the lesson as soon as possible

with the teacher for validation.

Answering research sub-question 3, What are key design features and other

implementation factors that need to be in place to increase the potential of BBM lessons

have the desired outcomes?, required me to identify the key design features and other

implementation factors that need to be in place to increase the potential that BBM lessons

will increase higher-order thinking. To begin answering this sub-question, I first recorded

the key decision points from the lesson planning sessions and the rationale for the

decisions we made. These decisions included our reasons for using a specific work of art

or why we structured the protocols and questions the way we did. Additionally, I used my

researcher’s journal to make note of any concerns the teacher expressed about our

decisions.

My next step in answering research sub-question 3 was to identify and record any

insights and/or preliminary hypotheses about design features and/or implementation

factors that arose from my annotated lesson plans and post-lesson implementation

interviews, again using my researcher’s journal to collect this information. I used the

PAQT observation records and researcher’s journal to identify any particularly

compelling lesson segments that I should return to for further analysis.

 89

My final step was to use all my data sources, as well as the annotated lesson plans,

to identify key design elements and other factors that seem critical to the success of BBM

lessons. In the spirit of action research, significant insights were shared with the teachers

at the beginning of each new planning session so that these ideas could inform the design

of the following BBM lesson.

After all the BBM implementations, I examined all the data again to determine

similarities and differences, and to articulate some preliminary findings regarding key

design features and implementation factors. These preliminary findings were checked as

part of the final interview process with each teacher.

Validity and Trustworthiness

It was important to have safeguards in place to assure validity and trustworthiness,

with validity meaning that I measured what I intended to measure in this study, and

trustworthiness meaning that the conclusions I have drawn from the data make sense.

With respect to validity, first of all the PAQT observation tool was directly aligned

to measure students’ higher-order thinking in real time throughout my lesson

observations. Thus, I analyzed the PAQT data with each participant during our

reflections after each BBM lesson. Member checking continued after the lessons had been

completed, when I brought the data analysis, interpretations, and conclusions back to the

participants for verification and input. They were able to help me determine how closely

my preliminary findings aligned with their own experiences of creating and teaching BBM

lessons. This provided me with invaluable insights as I worked to finalize my own

understandings.
 90

Trustworthiness is key to action research, and I built that through multiple data

sources and multiple layers of analysis. First, I analyzed the data with each participant

during our reflections after each BBM lesson. This continued after the lessons were

completed when I brought the preliminary data analysis, interpretations and conclusions

back to the participants for verification and input. My researcher’s journal proved useful

for examining my own thinking, reorienting and refocusing my thoughts as the study

evolved. As ideas, questions or concerns surfaced throughout this study, I noted them in

my researcher’s journal.

Triangulation was another method of assuring trustworthiness and validity. I used

a wide variety of data sources, including interview responses from three different

teachers, student survey responses, lesson plan data, and the PAQT scores for each lesson.

I worked to ensure that every potential hypothesis arose from the analysis of at least three

different data sources. Finally, I used a critical friend, someone who understands research

practices but is outside this particular research study, to offer alternative explanations or

analyses of the data and provide some perspectives on any bias or assumptions that may

have arisen (Herr & Anderson, 2005).