Sarah Villegas Action Research Paper Final Draft

1
Running Head: ASSESSING CRITICAL THINKING IN ELEMENTARY STUDENTS
Assessing the Critical Thinking Skills of Fourth and Fifth Grade Students
A Field Project Presented to the Faculty of the Graduate School of Education TOURO UNIVERSITY - CALIFORNIA In Partial Fulfillment of the Requirements of the Degree of MASTER OF ARTS in EDUCATION With Emphasis in Innovative Learning
by Sarah Villegas January 2013
2
ASSESSING THE CRITICAL THINKING SKILLS OF ELEMENTARY STUDENTS USING RUBRICS
Abstract The use of rubrics as assessment tools to evaluate critical thinking skills was examined in the context of eight fourth and fifth grade classrooms at an elementary school. Eight fourth and fifth grade teachers who have all been trained in and have experience integrating project-based learning were invited to participate in both a survey as well as a brief interview to share their reactions to a draft critical thinking skills rubric for grades three through five, which was developed by a district-level blended assessment team consisting of both educators and administrators. Nearly all of the participating expressed doubt over whether would use this tool in their own classroom, and all believed that the rubric needed to be revised in order to shorten it and make it more student-friendly. Keywords: critical thinking, assessment, rubrics
3
Introduction and Statement of the Problem Background and Need for Study What was considered a good education fifty years ago is no longer enough for success in college, career, and citizenship in the 21st century. Historically speaking, education has always been shaped by the needs of the societies in which it is set (P21, 2007). John Dewey, responding to the changes brought on by the industrial revolution, said that It is radical conditions which have changed, and only a radical change in education suffices Knowledge is no longer an immobile solid; it has been liquefied. In spite of the growing movement to turn schools into factories churning out future laborers, Dewey advocated for an education system that enlightened both the individual and society by developing a childs social power and insight (Dewey in P21, 2007, p. 1). These reflections seem eerily prescient given todays educational landscape. The hard truth recognized by Dewey a century ago that radical change is necessary is one almost universally shared by university and business leaders, educators, and policymakers alike. Both the 1983 Nation at Risk report and the Labor Secretarys 1992 Commission on Achieving Necessary Skills (SCANS) denounced the education system as antiquated, stuck in the industrialmodel past, and contributing to the failing economy of the United States. Many today would argue that nothing has changed, except for increased testing and accountability. It is indisputable that there is an ever-widening gap between what students learn in school and the skills they need in 21st century workplaces. In addition to the historical background of the 21st century movement in education, it is also important to note its origins within behavioral science. The pioneering work of Lev Vygotsky and Jean Piaget in the 1920s collectively influenced Bloom in his creation of a taxonomy, which ordered cognitive skills by level of complexity. Blooms taxonomy proposed
4
that, in order to learn at the higher levels (analysis, synthesis, evaluation) a student must have learned prerequisite knowledge and skills at lower levels (knowledge, comprehension, application) (P21, 2007). Howard Gardner later furthered understanding of how we learn and whom we deem smart with his theory of multiple intelligences, including inter- and intrapersonal, spatial, musical and kinesthetic. Despite the implications of this body of scientific research around the ways we learn and how we are intelligent, our education system has remained steadfastly stuck in the past, resistant to change, and focused almost exclusively on accountability and equity. In addition to its virtual dismissal of the behavioral science of learning, the education system has both lagged behind technological advances and been misguided in its application, with new digital devices (being) employed without reference to new approaches to learning (P21, 2007, p. 7). In an era of buzz phrases such as bring your own device and one to one computing, school districts are feverishly shelling out funds for 21st century devices like laptops, iPads, interactive whiteboards and responders and putting them in classrooms structured for 20th century learning. The fallout is widespread, with exasperated students, discouraged teachers and irate school boards. Often the harbingers of whats new and notable in technology, students continually bemoan their teachers dearth of techie skill and their perceived squandered opportunities for blended learning. Without adequate training and professional development in integrating technology into all facets of instruction, teachers inevitably default to what they know and are comfortable using (overhead projectors, television), despite the tens of thousands of dollars of state of the art equipment gathering dust in their classroom. Nothing inflames school board members, parents, administrations and the tax-paying community more than wasted funding.
5
The collective implications of these societal needs, background knowledge of the science of learning, and the need for better implementation of new technologies for learning led to the call over ten years ago for an educational framework designed around 21st century skills and an emphasis on real world relevance (P21, 2007, p. 7). Both President Barack Obama and Education Secretary Arne Duncan have been consistent in their message that assessments that merely measure a students ability to bubble in an answer on a test are insufficient and have called for standards and sophisticated assessments that measure 21st Century skills like problem-solvingand entrepreneurship as well as individual student growth, creativity, and critical thinking (Duncan, 2011; Obama, 2009). The 21st Century skills movement is more than a decade old, yet important questions remain about implementation and the lack of momentum around the movement (NEA, 2012). The National Educators Association (NEA) helped establish the Partnership for 21st Century Skills (P21) in 2000 and in 2002 developed a Framework for 21st Century Learning, focusing on eighteen 21st century skills. However, over the past decade, concern arose that the framework was too convoluted (NEA, 2012). In response, P21 surveyed leaders to prioritize the eighteen 21st century skills. The survey results clearly indicated that four specific skills were the most important. These became known as the Four Cs: critical thinking, effective communication, collaboration on teams, and expanding upon students natural creativity (NEA, 2012). The Partnership for 21st Century Skills believes that in order for students to be successful in higher education coursework and in a global job market, U.S. schools must teach both the 3Rs and the 4Cs, thereby fostering classrooms that mimic the real world environments found in those settings.
6
P21 defines the 4Cs in its white paper, The Intellectual and Policy Foundations of the 21st Century Skills Framework, describing them as higher order cognitive competencies students need to be effective and self-reliant lifelong learners (P21, 2007, p. 12). Critical thinking is explained as the capacity of active investigative thinkingthat is purposeful, reasoned, and goal-directed (Halpern, in P21, 2007, p. 12). In addition, the NEAs Educators Guide to the 4Cs identifies specific criteria and outcomes for critical thinking, including reasoning effectively, using systems thinking, making judgments and decisions, and solving problems (NEA, 2012, p. 9). On a local level, the Napa Valley Unified School District has responded to the changing educational landscape by revising its mission, which is now to transform students lives by instilling 21st Century skills and to inspire lifelong learning (NVUSD, 2012). The district believes it essential to make learning relevant to the real world so that students succeed in their careers, add value to their communities and become contributing members of society. In addition to mastering basic educational requirements, the district places equal importance on students mastering 21st Century skills, specifically, the four Cs. The priorities for the 2012-2013 school year are: Ready All Students for College and Careers Provide Opportunities for All Instill 21st Century Skills
With the movement toward national Common Core State Standards (CCSS) for mathematics and English language arts, the education field has made it clear that the priorities for K-12 education have shifted (CCSS Initiative, 2010). Educational researchers, alongside teachers, have developed goals and benchmarks for what students need to know and be able to
7
demonstrate in order to be successful in the 21st Century world of college and careers. Nevertheless, practitioners have been resistant to implement 21st Century learning or the Common Core State Standards. Many members of the education field feel that the 4 Cs are the key ingredient to usher our students into the 21st Century world of CCSS and next generation assessments. Critical thinking is a non-negotiable skill for success in the 21st Century milieu. Purpose of the Study Currently, teachers primarily use activity-driven rubrics to assess project-based learning assignments. These rubrics include ways to evaluate student work products and processes, but not necessarily the soft skills inherent in project-based learning. These soft skills include the ability to communicate effectively, interpretation and analysis, reasoning, problem solving, and reflection, among others. These skills are notoriously difficult to assess. As a result, teachers often neglect to evaluate skills such as collaboration, or opt to assign unsubstantiated grades to things such as teamwork or creativity without providing both themselves and their students with the parameters, learning objectives or outcomes for such skills. Assessing the 4 Cs and other soft skills will take on increasing importance with the burgeoning importance of projectbased learning, 21st Century skills, the Common Core standards, and next generation assessments like those being developed by SmarterBalanced (SmarterBalanced, 2010). The purpose of this study is to develop and test a set of district-wide Common Core-aligned learning outcomes and vertically aligned assessment rubrics to measure mastery of those learning outcomes. These learning statements are then to be used by teachers and students to teach and assess the 4 Cs. The outcomes as related to the district implementation of the aforementioned rubrics are: 1) a set of district-wide Common Core aligned learning outcomes that prepare students to be active and successful participants in the 21st Century; 2) assessment tools that align vertically from
8
elementary through high school to measure mastery of those learning outcomes; and 3) increased implementation of project-based learning that both engages students in relevant, rigorous inquiries and holds them accountable for both their work products and processes as well as for the soft skills they are required to demonstrate. Research Question What are the challenges to implementing assessment tools to evaluate critical thinking? Hypothesis: PBL implementation is a complex process requiring educators, students and their families, policy makers, and community members to develop a new set of standards and assessments around the "soft skills" required for success in the 21st century. Strategies and Activities The researcher used a mixed-methods research approach to examine how the implementation of district-created 4 Cs rubrics, particularly critical thinking, is sustained at the school and district levels, and how these rubrics support the assessment phase of the PBL implementation process at the classroom level and within their schools. Review of Related Literature Critical thinking has always been recognized as important by teachers and researchers. Virtually every textbook and curriculum available references critical thinking, with some including call-out boxes with critical thinking activities for students to complete if they have extra time. Roland Case (2005) of the Critical Thinking Consortium states that there is universal agreement (p. 1) about the need to make thoughtful judgments in virtually every aspect of our lives. Case (2005) also asserts that critical thinking is not a different type of thought. Rather, it is normal thinking done thoughtfully and reflectively, with the goal being a balanced, researched conclusion. For critical thinking to be achieved by students, students have to be provided with
9
models of what critical thinking looks like. They also need to be guided through the processes of thinking critically, assessed on the depth and complexity of their own critical thinking and given feedback as they practice thinking critically in various situations, both independently and with their classmates. Formative assessment and feedback are essential ingredients to teaching students to be critical thinkers. John Mergendoller (2011), Executive Director of the Buck Institute for Education, agrees with Case (2005) and further asserts on his 21st Century Skills blog on the Partnership for 21st Century Learning website that students need to know what is expected of them and how they are progressing towards those learning outcomes. Are they thinking carefully or carelessly? Are they being thoughtful or thoughtless? Students can learn to evaluate their own thinking and they can learn to evaluate arguments and reasoning of their peers. This ability to think about the quality of their own and of others thinking is encouraged by timely, relevant, actionable feedback from the teacher, from their peers, and from their own self-assessments. A majority of the research conducted around the issue of critical thinking has been done by Peter Facione (2011), who has studied critical care nurses ability to think critically, along with a panel developed consensus on the role of critical thinking (CT) in educational assessment and instruction, and developed the California Critical Thinking Disposition Inventory as a measure of clinical judgment. One of the interesting findings of Facione and his panel is the idea that the absence of critical thinking and a lack of fostering it in our students and society results in catastrophic consequences that are far-reaching and long-lasting (2011, p. 1). Examples provided by Facione and the panel of what could and does happen when people make important decisions without thinking things through include patient deaths, job loss, war casualties, divorce, suicide attempts,
10
academic failure, upside down mortgages, among other what were they thinking? fallouts (2011, p. 3). This can be taken by those seeking to further understand critical thinking and the ways to integrate it into educational practice as both a call to action and a prescient warning, particularly profound given the recent economic collapse and divisive politics paralyzing the nation. The experts Facione convened are clear that critical thinking is fundamental to, if not essential for, a rational and democratic society (2011, p. 23). If a society does not teach its citizens both the value of thinking for themselves and how to think for themselves, Facione believes it is madness to advocate democratic forms of government (2011, p. 23). The idea of critical thinking, or good thinking, as an essential skill not merely for students, but for all members of society is not novel. Facione asserts that you teach people to make good decisions and you equip them to improve their own fortunes and become contributing members of society, rather than burdens on society (2011, p. 1-2). Faciones definition of critical thinking delves deeper than the laypersons understanding in that he provides a working consensus of experts in the field, which is understood to be purposeful, reflective judgment which manifests itself in reasoned consideration of evidence, context, methods, standards, and conceptualizations in deciding what to believe or what to do (Facione, 2011, p. 26). The expert consensus goes further to define the ideal critical thinker as: habitually inquisitive, well-informed, trustful of reason, open-minded, flexible, fairminded in evaluation, honest in facing personal biases, prudent in making judgments, willing to reconsider, clear about issues, orderly in complex matters, diligent in seeking relevant information, reasonable in the selection of criteria, focused on the inquiry, and persistent in seeking results which are as precise as the subject and the circumstances of inquiry permit (p. 26).
11
While few researchers have delved into the complicated issue of assessing or evaluating something as complex as critical thinking, there is a burgeoning body of research related to critical thinking as it relates to education and how educators have fared in their attempts to qualify or quantify the degree to which their students exhibit the characteristics of effective critical thinkers. Roland Case (2005) argues that when it comes to teaching critical thinking in schools, the implementation is disheartening and he feels strongly that the rhetoric greatly outstrips practice (p. 1). In other words, everyone is talking about it, but no one is actually doing it. It is something that we greatly value, yet insufficiently teach. Perhaps one of the reasons teachers neglect integrating such an essential educational and life skill as critical thinking is that it is notoriously challenging to quantify, qualify and evaluate (Case, 2005). Case (2005) contends that there are three main reasons that critical thinking is seldom the focus in learning environments and educational practice. The first is what he terms a proliferation of thinking skills (p. 1). That is to say educators are attempting to impart too many skills to their students. If critical thinking is simply one of myriad processes a student must undertake, or if it is just one other skill a teacher must cover, it will never be given the amount of attention necessary for students to become masterful critical thinkers. The second contributing factor to the neglect of critical thinking instruction is what Case (2005) defines as the ranking of thinking skills (p. 1). When teacher training programs, educators and professional development gurus relegate critical thinking to one of many higherorder thinking skills, the presumption is that there are subordinate and prerequisite thinking skills that have to be mastered before critical thinking can be tackled. Case (2005) asserts that critical thinking appears at the end of the list(of higher-order thinking skills), often reserved solely for the best students (p. 1).
12
Finally, Case (2005) discusses a separation of skills from content as the third cause of the dearth of critical thinking in schools (p. 2). Most teachers are wholly concerned with covering content and standards; some are even focused solely with covering material which will be tested on state or local exams. As such, soft skills like critical thinking, which will not directly be assessed on standardized tests, are addressed only as time permits and after subject matter content has been covered. In essence, critical thinking is treated as an afterthought or extension activity, rather than integrated into the content or given a weight equal to the content itself. Browne and Freeman (2000), in their study of the distinguishing features of critical thinking classrooms, explain that critical thinking comes in many different forms but that those classrooms that foster critical thinking possess features such as asking and allowing frequent questions, active learning, a fascination with the contingency of conclusions and being comfortable with being uncertain (p. 6). Browne and Freemans (2000) conclusion is that critical thinking can be taught with the active assistance ofstudents, which is to say that learning is always a two-way street whereby the unnatural process of questioning ones own thinking, checking assumptions and being reflective is made to feel safe and is understood by all of the participants in the learning environment to be the way they conduct business (p. 8). In addition to the research supporting the vaunted importance of critical thinking, there were a few studies specifically investigating the issue of assessing critical thinking. Peach, Mukherjee, Hornyak (2007), reflected on the daunting process of evaluating critical thinking and concluded that assessment in general is a challenging issue and that assessment of critical thinking is one of the most difficult (p. 313). In particular, the Peach et al. (2007) study on attempts to quantify and assess critical thinking yields the most beneficial and practical information around the issue of this study.
13
Because the business college of the University of West Florida determined that critical thinking would be one of their five overarching business college learning outcomes, it became necessary for them to both define critical thinking and develop an assessment of it (p. 314). Members of the business college began by developing learning outcomes around business critical thinking. They then designated a single capstone course (Business Policy) within which they would assess student critical thinking, with the rationale being that all students were required to take the class and that it came at the conclusion of their business college experience, in the final semester before graduation. The capstone faculty then developed a rubric, tested it during the spring assessment cycle, met to evaluate the process and results, and revised the rubric accordingly (pp. 314-125). In their reflection on the assessment process using the critical thinking rubric, the authors noted several conclusions, including the fact that using the rubric took a significant amount of time to grade, and there was uneven application of the rubric by instructors (p. 316). One emergent theme from the research on assessment is that evaluation without intervention is meaningless. Another relevant finding was that authentic assessment of something as complex as critical thinking requires commitment from all stakeholders within the educational community (Peach et al., 2007). The researchers in the Peach et al. (2007) study discuss already over-worked faculty members attending what seemed like an endless series of meetings in order to develop, review and revise these assessments (p. 316). They also caution that those who embark on developing an effective assessment process should be clearly cognizant of the added demands it will require, and have sufficient commitment to accept the findings (p. 316). They also mention the importance of allocating sufficient resources, developing a network of knowledgeable people to share experiences throughout the process, and reading academic
14
literature around the topics of assessment and critical thinking before beginning assessment (pp. 316-317). Most notably, the authors stressed that rubrics for assessing critical thinking must be developed by the educators who will put them into practice. They mention that the creation of rubrics is an ongoing process and one that is continually revisited and revised. They also found that the initial rubric comprehensively assessed everything the faculty thought students should be doing and was not practical when it came to applying the rubric to student work (pp. 316317). They note that the research on rubrics reveals them to be difficult, situation-specific and (able to) improve with practice (p. 316). The authors interestingly noted that while rubrics are difficult, assessing critical thinking with a rubric is more difficult than assessing other soft skills such as communication or collaboration (p. 317). Other findings of note include the need to establish and monitor the consistency of educator implementation of the rubrics, the importance of seeking out and involving champions of the process, or those administrators and educators who will be invested in the success of the difficult work. Finally, they caution those engaging in similar undertakings to avoid common pitfalls when creating assessments for something like critical thinking. The shortcuts they mention include rushing through the process and making the rubrics look good to external constituents rather than focusing on their usefulness and application to improve student learning (Peach et al., 2007, p. 317). Practical application of the various studies on critical thinking, assessment, rubrics and student learning outcomes provides data to analyze and theories to revisit. One common theme related to assessment is that it is a continuous process as opposed to a one and done affair. The key is the get it down, then get it right (Mager, 1973, cited in Peach et al., 2007, p. 316). Along
15
the same lines as the Peach et al. study, Stein, Hayes, and Unterstein (2003) also conducted a study related to the assessment of critical thinking skills. Over three and a half years, they developed and continuously revisited assessment tools related to critical thinking for students at Tennessee Technical University. Their major finding was that a canned or pre-produced assessment or rubric would never meet the needs of their own faculty and was not something that would be embraced by the educational community as relevant (Stein et al., 2003). It was essential that the assessment be authentic and, to echo the sentiments of the Peach (2007) study, be created by those educators who would put it into practice (Stein et al., 2003, p. 1). A notable feature of the Stein study is the steps the researchers took to ensure content validity through the comparison to other student performance measures of critical thinking such as the California Critical Thinking Skills Test (CCTST) and the ACT (Stein et al., 2003). In conclusion, much of the research related to the issue at hand is either focused solely on critical thinking or focused solely on using rubrics as assessment tools. The few research studies which examined using assessment tools to evaluate critical thinking were from university-level studies of the critical thinking of collegiate and post-collegiate students. Hence, there is need for more study of the teaching and assessing of critical thinking as well as the other three critical Cs: communication, collaboration and creativity. Research Methods Problem & Research Question The implementation of project-based learning is a complex process requiring educators, students and their families, policy makers, and community members to develop a new set of standards and assessments around the "soft skills" required for success in the 21st century. The
16
research question at hand here is: what are the most effective methods for implementing assessment tools to evaluate critical thinking? Materials The impetus for this study was the convening of an expert blended team within the Napa Valley Unified School District (a small group of practitioners made up of a mix of elementary, middle and high school administrators and teacher-leaders, and different content areas) who were charged with developing student-friendly versions of the grades 4-5, 7-8, and 11-12 EdLeader21 rubrics (Appendix A) and supporting materials that encourage the use of those rubrics. The initial draft form of the critical thinking rubric developed by this 4 Cs Rollout team was used in this research study (see Appendix B). In addition, I developed a short two page survey (see Appendix C), which I provided to teachers both electronically and in hard copy form. I also provided teachers with an IRB consent form. Participants Eight fourth and fifth grade teachers who have all been trained in and have experience integrating project-based learning were invited to participate in both a survey as well as a brief interview to share their reactions to a draft critical thinking skills rubric for grades three through five, which was developed by a district-level blended assessment team consisting of both educators and administrators. Procedures To begin the study, I first needed to participate in two of the Napa Valley Unified School Districts 4 Cs Rollout Team collaboration days in the fall. The task of this expert team for the first two days of collaborative work was to answer the question: What are we measuring? Our assignment was to create user-friendly 4Cs measurement tools (refined from the Ed Leader21
17
rubrics) using matching I can targets, so that all our schools would have a similar understanding, of what each C looks like at each level of proficiency and the system to implement and account for mastery of these skills at each grade level. The expert team worked first as whole team aligning the work purpose and product goals, then we divided into four separate teams (one for each C). Finally, we reconvened as a whole group and engaged in a SWOT analysis (Strengths, Weaknesses, Opportunities, Threats) of each teams work. The expert team used Google Docs, Edmodo, and structured templates and tools, sharing online to increase productivity and model effective uses of web 2.0 tools. Several considerations and stipulations were addressed during these two collaborative days, including: 1) the tool must be both formative (show growth within a year) and summative (show progress from year to year), 2) the tool must allow for students to show evidence of their growth towards learning targets, 3) the tool must support educators ability to provide targeted feedback and intervention as needed, 4) the tool must provide opportunities for student selfreflection and goal-setting, 5) the tool must be able to be used interactively inside a technologybased platform, and 6) the tool needed to include performance indicators on a scale of 4 to Not Yet, which would simply imply that the student at the Not Yet performance level was striving to demonstrate that particular skill, but had not yet demonstrated it. The sample rubric is shown in Appendix B. Future supporting products developed by this team when collaborative work resumes in January 2013 will seek to answer the questions: How can students use this tool? How do teachers use this tool to drive instruction? Specifically, the projected work products are as follows:
18
samples of the rubrics as student progress charts embedded in Echo (an online grading platform used in the Napa Valley Unified School District that is specifically designed to support the grading of project-based learning assignments).
samples of formative performance tasks and feedback using the rubrics examples of rubric use and evidence/artifacts within content area learning experiences idea list/tip sheet for rollout at individual sites and conversations in PLCs around essential learning outcomes and assessments
After collaborating with the expert team, I then took the draft form of the rubric crafted by the critical thinking team and limited it to just the section related to grades three through five. I prepared my survey, focusing on the reactions teachers had to the rubric itself: format, scope, accessibility. The survey is shown in Appendix B. I provided the eight fourth and fifth grade teachers at my site the rubric, along with the survey and asked them to record their responses. During the final week of the study, I asked two teachers, one from each grade level, if they would allow me to interview them to gain further insight into the feedback teachers had regarding the rubric. I used the same questions as those on the survey (Appendix B) and asked the teachers to elaborate on their responses while I took notes. Analyzing the Data After collecting all of the surveys (Appendix B) from the participating educators, I created tables for survey questions 1, 2 and 4 (Tables 1-3) in order to visually represent the responses and examine trends. For question 6, I made a list of the narrative responses and highlighted repeated words and phrases. I also created a table for question 7 (Table 4), which related to strengths evident in the rubric, because several of the teachers had the same or similar
19
responses. I was able to reduce all of their responses down to four categories: 1) clarity, 2) well thought out, 3) standards-aligned, and 4) thorough. I followed the same process when analyzing question 8 (Table 5). For this question, which related to areas of concern about the rubric, I once again narrowed the participant responses down to four overarching categories: 1) confusing, 2) not kid-friendly, 3) length, 4) wordy. Finally, I decided not to create a table for question 9, which asked participants if they would use the proposed rubric as an assessment tool in an upcoming PBL unit. The response was a unanimous yes from participants, so instead, I chose to synthesize the justifications they gave for their yes responses (question 10), which I then created into a table (Table 6), representing the four overarching explanations given: 1) general guideline, 2) targeted skills, 3) during presentations, 4) student outcomes. I also elected to interview two of the participants as representatives of each grade level to probe further into the feedback the grade level teams had regarding the survey. I asked each to elaborate specifically on questions 6, 7, 8 and 10. After reviewing my notes from the two interview sessions, I processed the information by reducing the narrative responses down to a few consistent themes by creating a simple coding system: 1) the rubric would be helpful during the planning process of PBL = PLAN, 2) the rubric needs to be revised for length and word choice = LONG, 3) teachers should be able to use parts of the rubric as opposed to only the whole = PART, 4) the rubric is an improved assessment tool for evaluating critical thinking = GOOD, 5) the rubric is still a very subjective method of evaluating student learning = SUBJ. I then reviewed the transcripts from the interviews and noted the codes next to responses.
20
Findings The response to the rubric as an authentic assessment tool of critical thinking skills was mixed. Teachers found both positive and negative attributes in the tool and offered constructive feedback useful for eventual revision of the rubric. One important finding was that teachers recognized the need for the rubric as both an instructional tool as well as measure of student skills. At this point, however, the participants felt that they would use the tool, in its current rendition, for its parts rather than as a whole assessment and, even then, primarily as a guide for student outcomes. All teachers who participated in the survey indicated the number of years they have been teaching as well as the number of years they have been integrating 21st Century skills and project-based learning (PBL) into their teaching. As you can see from Tables 1 and 2 below, the teachers participating in this study are experienced in both their teaching and in their implementation of PBL. As such, they should be qualified to evaluate the effectiveness or usefulness of the critical thinking rubric at issue in this study. Table 1. Years of Teaching
30 25 20 15 10 5 0 Length of Years Teaching Teacher 1 Teacher 2 Teacher 3 Teacher 4 Teacher 5 Teacher 6 Teacher 7 Teacher 8
21
Table 2. Years Implementing PBL 3 2.5 2 1.5 1 0.5 0 Number of Years Implementing PBL Teacher 1
Teacher 2
Teacher 3 Teacher 4 Teacher 5 Teacher 6 Teacher 7 Teacher 8
In addition, all teachers indicated on their surveys that they had received training in project-based learning at the Buck Institute for Education-sponsored PBL World conference either this past summer (2012) or the previous summer (2011). In the past, the participating educators mentioned using several varying types of assessment tools as part of the evaluation process within project-based learning units. Examples of these include self-created rubrics, teacher observation, grade-level team created rubrics, the adopted California state standards for English language arts, feedback from colleagues, and other informal assessments (see Table 3).
22
Table 3. Previously Used PBL Assessment Tools
100% 80% 60% 40% 20% Standards 0% Critical Thinking Assessment Tools Previously Used Unanimously, teachers mentioned that the rubrics they had been favoring most in the past (the self-created rubrics) were only somewhat effective at capturing or fully measuring the extent to which a students had demonstrated critical thinking during a project or assignment. This appraisal indicates a need for better assessment tools when evaluating something as complex as critical thinking. That said, the teachers also unanimously expressed concern over the practicality of the proposed NVUSD expert team-created critical thinking skills rubric for grades three through five, noting that it was not kid-friendly and that students would have difficulty using it to selfreflect on their progress (see Table 5). Participants also noted that the word choice of the descriptors was too complex for the intended age group (and perhaps for educators as well) and that the performance level criteria would still engender subjectivity when used to evaluate student progress toward the expected outcomes. Teachers were flummoxed by what they termed vague descriptors in the rubric, such as Systems Thinking. Other challenges presented by the Self-Created Rubrics Grade-Level Created Rubrics Observations
23
rubric included its length as well as some confusion over the performance levels. Participants were not able to readily identify which performance level (4, 3, 2, Not Yet) corresponded to an above, on, or below grade level ranking. Table 5. Areas of Concern with Rubric
90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Rubric Areas of Concern Some of the strengths of the rubric indicated by participants include the fact that it seemed well thought out and aligned to the standards (see Table 4). In addition, participants appreciated its thoroughness and that it provided an organized, clear way to assess what critical thinking looks like in the classroom. Confusing Not Kid Friendly Lengthy Wordy
24
Table 4. Rubric Strengths
80 70 60 50 40 30
Clear
Well Though Out Standards-Aligned Thorough
20
10 0 Strengths of Proposed Rubric
The teacher participants unanimously agreed that the draft rubric could indeed be useful within their PBL implementation; however, not necessarily as an end-of-unit assessment tool (see Table 6). The consensus was that in order for them to be able to use the rubric in that fashion, it would need to be taken back to the design team to be further revised for word choice, length and student-friendliness. Participants indicated that the tool could be useful in its current incarnation as a reference during a PBL unit for teachers to set expectations for the critical thinking their students should be able to demonstrate as well as a monitoring tool teachers could use to gauge their classs overall critical thinking through observation. Finally, the participating teachers noted that they could see themselves utilizing parts or individual sections of the rubric at a time as opposed to trying to address the entire two-page rubric and all of its subcategories for each student in their class.
25
Table 6. Potential Rubric Uses
90% 80% General Guideline
70%
60% 50% 40% 30% 20% 10% 0% Potential Uses for the Rubric During a PBL Unit Student Outcomes During Presentation Phase Target Skills
Discussion and Implications Overall, both the literature and the findings of this study indicate a need for clearer definitions of what it means to be a critical thinker (Kuhn, 1999). Definitions of critical thinking vary widely; however, there is agreement that metacognition and reflection are essential components in order for students to internalize and replicate higher order thinking skills outside of the classroom (Kuhn, 1999). There is also consensus about the need for a coalition or network when developing outcomes related to skills such as critical thinking. Moreover, studies have found that authentic assessment tools for critical thinking are challenging for educators to design that are developed must be carefully vetted, monitored, and revised in order to remain effective at measuring critical thinking. With the increased interest in project-based learning (PBL) as a means of developing skills such as collaboration, critical thinking and problem solving, educators need both tools and support in order to plan and implement PBL effectively and
26
measure student content and 4 Cs learning within that context (Thomas, 2003). Particularly, studies on PBL have found that teachers struggle when trying to design assessments that require students to demonstrate their understanding of skills such as critical thinking (Thomas, 2000). Despite these challenges, both the research and the conclusions of this study indicate that problemscan be effectively facilitated by a supportive school environment that allows teachers to reflect on their practices and to attempt changes in these practices through enactment linked with collaboration and feedback (Thomas, 2000, p. 25). Despite the challenges presented by trying to assess something as cerebral as critical thinking, Yeh argues that we embrace the notion that tests drive curriculum and design better assessments that emphasize critical thinking in the form of careful argumentation as a way to improve the quality of instruction (2001, p. 12). With the struggles educators have faced in implementing critical thinking instruction and assessment, one might question whether it is a worthy enough cause for inquiry and effort. Kuhn offers: to be competent and motivated to know how you know puts one in charge of ones own knowing, of deciding what to believe and why and of updating and revising those beliefs as one deems warranted. To achieve this control of their own thinking is arguably the most important way in which people both individually and collectively take control of their lives (1999, p. 23). As a result, educators must support the learning of critical thinking and other soft skills through the cultivation of knowledge around what it means to think critically and be metacognitive and the gift of time for repeated practice of skills such as analysis, making judgments, reasoning, problem solving, constructing arguments and reflecting.
27
Moving forward, it is critical for the Napa Valley Unified School District 4 Cs Rollout Team and other instructional teams looking to integrate critical thinking into the curriculum to collaborate with district-level, site-level and classroom-level stakeholders to 1) develop a consensus around a clear definition of critical thinking, 2) develop clearly written student learning outcomes and performance indicators that are both teacher- and student-friendly, 3) draft a rubric that is manageable in length but thorough in scope, 4) test the rubric in classrooms throughout the district, gathering evidence of validity and accuracy as well as working towards equitable calibration of the scoring of critical thinking among all teachers.
28
References Andrade, H. (2000). Using rubrics to promote thinking and learning, Educational Leadership, 57.5, pp. 13-18. Andrade, H. (1999). Student self-assessment: At the intersection of metacognition and authentic assessment. Paper presented at the Annual Meeting of the American Educational Research Association (Montreal, Quebec, Canada, April 19-23, 1999). Bailin, S., Case, R., Coombs, J. R., & Daniels, L. B. Conceptualizing critical thinking. Journal of Curriculum Studies, 31(3), 285-302. Browne, M. Neil & Freeman, K. (2000). Distinguishing Features of Critical Thinking Classrooms, Teaching in Higher Education, 5.3, pp. 301-309. Case, R. (2005). Moving critical thinking to the main stage. Education Canada, 45(2), 45-49. Common Core State Standards Initiative. (2010). Common Core State Standards for English Language Arts & Literacy in History. Social Studies, Science, and Technical Subjects, 2. Dean, D., & Kuhn, D. (2003). Metacognition and Critical Thinking. EdLeader21 (2012). Critical Thinking Rubric, Grades 3-4. Facione, P. A. (1998). Critical thinking: What it is and why it counts. Millbrae, CA: California Academic Press. Retrieved April, 1, 2004. Heyman, G. D. (2008). Children's critical thinking when learning from others.Current directions in psychological science, 17(5), 344-347. Kuhn, D. (1999). A developmental model of critical thinking. Educational researcher, 28(2), 1646. Marx, R. W., Blumenfeld, P. C., Krajcik, J. S., & Soloway, E. (1997). Enacting project-based science. The elementary school journal, 341-358.
29
Mergendoller, J. (2011). Teaching critical thinking skills through project based learning. Retrieved from: http://www.p21.org/tools-and-resources/p21blog/1097-teaching-criticalthinking-skills-through-project-based-learning. Napa Valley Unified School District (2012). Mission Statement. Retrieved from http://www.nvusd.k12.ca.us/superintendent. National Education Association. (2012). An Educators Guide to the 4 Cs. Retrieved from the NEA website: http://www.nea.org/assets/docs/A-Guide-to-Four-Cs.pdf. Partnership for 21st Century Skills (2007). The Intellectual and Policy Foundations of the 21st Century Skills Framework (white paper). Retrieved from the Route 21 website: http://route21.p21.org/images/stories/epapers/skills_foundations_final.pdf Peach, B. E., Mukherjee, A., & Hornyak, M. (2007). Assessing critical thinking: A college's journey and lessons learned. The Journal of Education for Business, 82(6), 313-320. SMARTER Balanced Assessment Consortium. (2010). Race to the Top Assessment Program application for new grants. Stein, B., Haynes, A., & Unterstein, J. (2003, December). Assessing critical thinking skills. In SACS/COC Annual Meeting, Nashville TN. Thomas, J. W. (2000). A review of research on project-based learning. San Rafael, CA: Autodesk Foundation. Retrieved April, 13, 2003. U.S Department of Labor (1992). Learning a Living: A Blueprint for High Performance (A SCANS Report for America 2000). The Secretarys Commission on Achieving Necessary Skills. Washington, D.C. Yeh, S. S. (2001). Tests worth teaching to: Constructing state-mandated tests that emphasize critical thinking. Educational Researcher, 30(9), 12-17.
30
Appendix A
31
32
33
Appendix B
34
35
Appendix C
Evaluating Critical Thinking with Rubrics Teacher Survey 1. How long have you been teaching? 2. How many years have you been integrating 21st century skills and/or Project-based Learning (PBL) into your curriculum? 3. What kind of training have you received on 21st century/PBL instruction? 4. What have you used in the past to assess the critical thinking aspect of learning? 5. How effective were those assessment tools in evaluating student learning? 6. Having reviewed the draft NVUSD-created rubric for Critical Thinking, what are your initial thoughts? 7. What are the strengths of this assessment tool? 8. What are the areas of concern/need? 9. Would you use this rubric to assess critical thinking as part of an upcoming PBL unit? 10. If you answered yes to question 9, please explain your response. 11. If you answered no to question 9, please explain your response.

Sarah Villegas Action Research Paper Final Draft

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Sarah Villegas Action Research Paper Final Draft

Uploaded by

Copyright:

Available Formats

1

Running Head: ASSESSING CRITICAL THINKING IN ELEMENTARY STUDENTS

by Sarah Villegas January 2013

Table 3. Previously Used PBL Assessment Tools

Table 4. Rubric Strengths

Table 6. Potential Rubric Uses

90% 80% General Guideline

You might also like