A MIXED METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGH SCHOOLS
ARTHUR L. PETTERWAY
Submitted to the Graduate School Prairie View A&M University in partial fulfillment of the requirements for the degree of
DOCTOR OF PHILOSOPHY
A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGH SCHOOLS IN TEXAS
A Dissertation by ARTHUR L. PETTERWAY
Approved as to style and content by:
L. M. Paul Mehta, Chair
HO Aid- ie. . Dou las Hermond
Dr. David Herrington Dr. Camille ibson
Dr. . Paul M Dean, The Whitlowe R. Green
e of Education
Dr. William Parker Dean, Graduate School
ABSTRACT A Mixed - Method Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas April, 2007 Arthur L. Petterway: B.A. – Dillard University M.Ed., Prairie View A&M University Dissertation Chair: Dr. M. Paul Mehta Ample research has been conducted on the intrinsic validity of standardized assessments, and on the factors affecting the assimilation and integration of English language learners (ELLs). The reliability of these assessments as a universal tool to measure student learning, and as a basis for determining school performance needed closer examination. The purpose of this study was to determine the impact of highstakes testing on ELLs. This was shown in both the quantitative and qualitative dimensions of the study. Data obtained from Texas Education Agency (TEA) were used to determine whether there was a relationship between the percentage of ELLs enrolled in a school and the percentage of all students passing the 10th grade Texas Assessment of Knowledge and Skills (TAKS) tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005, and 2006. The qualitative aspect of this study explored what certified English as a Second Language( ESL) teachers, non-certified ESL teachers who teach ELLs, iii
administrators, and district ESL personnel viewed as the impact that high stakes standardized assessments had on ELLs, ESL curriculum, and instruction in ESL classrooms. This study determined the impact of high-stakes testing on ELLs using the explanatory design of mixed method analysis. Data of 173 major urban high schools obtained from the Texas Education Agency (TEA). It was determined through the Pearson correlation computations using the Statistical Package for the Social Sciences (SPSS) that there was a significant relationship between the percent of ELLs enrolled in a school and the percent of all students passing the 10 th Grade TAKS tests in English Language Arts and Mathematics. In the qualitative portion of the study, the views and opinions of district ESL personnel were gathered. Principals, assistant principals, ESL and non-ESL teachers took part in an on-line, open-ended questionnaire; one-on-one interviews; and focus groups. The focus groups addressd the purposes of statewide testing; its intended consequences; problems and changes created by TAKS, and the recommendations to improve ESL curriculum and instruction. The results of the study affirmed the expected outcome that a significant relationship existed between the percentage of ELLs enrolled in a school and the percentage of all students passing the 10 th grade TAKS tests in both core areas of English Language Arts and Mathematics. The regression analysis predicted that as the percentage of
ELLs in a school increased, the performance on the statewide, highstakes testing in terms of all students passing the 10th grade TAKS tests decreased. Respondents of the study considered TAKS as a tool to gauge knowledge in the different core areas. English language learners were expected to have at least average scores on TAKS. There was a difference in the expected and actual results; respondents observed dismal or failing performance of ELLS in the actual results in TAKS. This was evident by the high failure rate of ELLs in their respective schools. Higher dropout rate and lower graduation rate of ELLs were problems encountered due to TAKS. Respondents favored a different test for ELLs, possibly given at a later date after ELLs had studied in the country for at least several years. Respondents believed that interventions were needed to help ELLs perform better. Both the school and the home, together with the community, have to be involved in preparing ELLs for their present and future roles in the American society. Results of this study may provide valuable data to district and school administrators to develop strategies that will improve the performance of ELLs on the statewide, high-stakes testing and to develop assessments that truly measure learning without the nullifying effect of linguistic and cultural bias. The study may also help to enhance the reliability of standardized assessments as a tool to determine accountability for student performance.
DEDICATION This dissertation is dedicated in humble gratitude to my Lord and Savior, Jesus Christ, in whom I move, trust, and have my being. Through HIS divine wisdom and purpose, HE gave me my parents: Bob Stevenson Petterway November 23, 1923-September 28, 1992 and Myrtice Lee Petterway February 10, 1927-February 15, 1959 They are now in Glory with HIM sharing this divine blessing that HE has bestowed upon me.
ACKNOWLEDGEMENTS I would like to acknowledge those without whom this work would not have been possible. First and foremost I wish to acknowledge the blessings bestowed upon me by my Lord and Savior Jesus Christ. Among these blessings are the kind souls I mention here. First, I would like to acknowledge my dissertation committee, Dr. M. Paul Mehta, Dr. William A. Kritsonis, Dr. Douglas Hermond, Dr. David Herrington, and Dr.Camille Gibson. They have been thorough, fair, understanding, demanding, and, most of all, dedicated to academic excellence in all phases of this work. I would especially like to thank Dr. M. Paul Mehta for taking time, along with his duties as the Dean of Education, and carrying the baton of being my committee chairman and Dr. Robert Marshall, my former committee chairman (who would not let Me fall off of my bicycle and taught me how to eat an elephant) for the long hours they have spent and the endless patience they have shown as they have guided me through this endeavor. I also wish to thank Dr. Kritsonis for his passion for making sure that I get published before I receive my PhD and taking on the duties of head cheerleader for this project. Thanks goes to Dr. Hermond for serving as lead statistician and for not allowing me to attempt to eat the whole pie and limiting me to a small slice. Many thanks also to Dr. Herrington who guided me in tying up several loose ends. I am also grateful to Dr. Gibson for taking time out of her very busy schedule to offer her support and encouragement.
Additional thanks go to Dr. William Parker, Dean of Graduate Studies, for believing in me and giving me the opportunity to prove myself beginning with my pursuit of my Masters Degree. I also gratefully acknowledge all of my professors and thank them for the wisdom and knowledge they have so generously shared. I would further like to thank all of the faculty and staff of Prairie View A&M University who have contributed to my achievement in countless ways. I also wish to acknowledge the unwavering encouragement of my student cohort as we shared the joy and pain of this incredibly challenging pursuit. I would like to thank my principal, Mrs. Linda Llorente, for her understanding and support as I pursued this dream. She has been abundantly patient and understanding of the demands this work has placed on my time and energy. I also wish to acknowledge the contributions of my peers and colleagues at Austin High School. Although it is virtually impossible to name all who have contributed to the completion of this work, I feel that there are several who must be thanked by name. I would like to thank Andy Lamboso and Rhodora Maligad who helped with proofreading and typing along with Kathy Koch, Betty Shaw, Debbie Kubiak, and Raul Asoy who helped with proofreading and editing. Finally, I wish to acknowledge the precious prayers offered by the righteous to strengthen and uphold me through this challenging time,
with a special thanks to my most avid prayer warrior, my aunt, Mrs. Sina Gunnels, and her chicken with a snuff cup under its wing.
TABLE OF CONTENTS Page ABSTRACT ..................................................................................... iii DEDICATION ................................................................................. vi ACKNOWLEDGEMENT .................................................................. vii TABLE OF CONTENTS ................................................................... ix LIST OF TABLES .......................................................................... xiv CHAPTER I. INTRODUCTION .......................................................... 1 Statement of the Problem....................................................... 3 Purpose of the Study ............................................................. 6 Research Questions ............................................................... 7 Quantitative .................................................................. 7 Null Hypothesis One ...................................................... 7 Null Hypothesis Two ...................................................... 7 Qualitative .................................................................... 7 Description of the Research Design ........................................ 8 Assumptions ......................................................................... 9 Limitations of the Study ........................................................ 9 Delimitations of the Study .................................................... 10 Definition of Terms ............................................................... 10 Significance of the Study ...................................................... 12 Organization of Study ........................................................... 13 CHAPTER II. REVIEW OF LITERATURE ......................................... 15
Page No Child Left Behind (NCLB) ................................................. 15 Historical Perspective .................................................. 15 Description of the Key Factors ..................................... 18 Expectations for Parents .............................................. 20 Response to NCLB ....................................................... 21 Adequate Yearly Progress (AYP) ............................................. 22 Purpose and Support to NCLB ..................................... 22 Changes and Updates.................................................. 24 AYP and Limited English Proficient (LEP) Students ................ 30 Definition of English Language Learners (ELLs) ........... 30 Issues and Other Considerations of LEP ....................... 34 High Stakes/Statewide Testing ............................................. 43 Principles of Testing Programs ..................................... 44 Accountability in Testing ............................................. 50 Effects of High Stakes Testing on Student Motivation .... 52 Other Considerations of Assessment on Testing ............ 56 Related Studies .................................................................... 59 Summary ............................................................................. 61 CHAPTER III.METHODOLOGY. ...................................................... 63 Introduction .................................................................................. 63 Research Questions .............................................................. 64 Quantitative ................................................................ 64
Page Null Hypothesis One .................................................... 64 Null Hypotheses Two ................................................... 65 Qualitative .................................................................. 65 Research Methods ................................................................ 66 Research Design................................................................... 67 Quantitative ................................................................ 68 Qualitative .................................................................. 68 Pilot Study ........................................................................... 68 Population and Sample ......................................................... 70 Quantitative ................................................................ 70 Qualitative ................................................................. 71 Instrumentation ................................................................... 72 Instruments ................................................................ 72 Validity ....................................................................... 74 Reliability ................................................................... 74 Research Procedures ............................................................ 75 Quantitative ................................................................ 75 Qualitative ................................................................. 75 Data Collection and Recording .............................................. 76 Quantitative ................................................................ 76 Qualitative ................................................................. 76 Data Analysis ....................................................................... 78
Page Quantitative................................................................ 78 Qualitative .................................................................. 79 Summary ............................................................................. 80 CHAPTER IV ANALYSIS OF DATA .................................................. 83 Findings .............................................................................. 85 Quantitative Research Question .................................... 85 Null Hypothesis One ..................................................... 86 Null Hypothesis Two ..................................................... 86 Qualitative Research Question ..................................... 100 Discussion ........................................................................ 141 Summary .......................................................................... 145 CHAPTER V SUMMARY, CONCLUSION, IMPLICATIONS AND RECOMMENDATIONS ................................................................. 147 Summary .......................................................................... 147 Conclusions ...................................................................... 150 Implications ...................................................................... 151 Recommendations for Further Study .................................. 154 REFERENCES ............................................................................. 157 APPENDICES .............................................................................. 177 Appendix A IRB .................................................................. 178 Appendix B Consent Form .................................................. 183 Appendix C Interview Questions .......................................... 186
Page Appendix D On-Line Questionnaire ..................................... 191 Appendix E Letter to Participants ........................................ 196 Appendix F Request for Extant Data from T.E.A... ................ 198 VITA ........................................................................................... 201
LIST OF TABLES Table 4.1.1 Means and Standard Deviations of ELLs Enrolled in School and All Students Passing the 2003 10th Grade TAKS for English Language Arts and Mathematics .............................. 87 4.1.2 Means and Standard Deviations of ELLs Enrolled in School and All Students Passing the 2004 10th Grade TAKS for English Language Arts and Mathematics ............................... 87 4.1.3 Means and Standard Deviations of ELLs Enrolled in School and All Students Passing the 2005 10th Grade TAKS for English Language Arts and Mathematics ............................... 88 4.1.4 Means and Standard Deviations of ELLs Enrolled in School and All Students Passing the 2006 10th Grade TAKS for English Language Arts and Mathematics ............................... 88 18.104.22.168 Comparison of Results in 10 th Grade English Language Arts TAKS ......................................................................... 89 22.214.171.124 Comparison of Results in 10 th Grade Mathematics TAKS...... 90 4.2.1 Pearson Correlation: 2003 10 th Grade TAKS for English Language Arts and Mathematics............................................ 91 4.2.2 Pearson Correlation: 2004 10 th Grade TAKS for English Language Arts and Mathematics............................................ 91 4.2.3 Pearson Correlation: 2005 10 th Grade TAKS for English Language Arts and Mathematics............................................ 92 xiv Page
Table 4.2.4 Pearson Correlation: 2006 10 th Grade TAKS for English
Language Arts and Mathematics............................................ 92 4.2.5 Coefficients for Percentage of All Students Passing the 2003 10th Grade English Language Arts TAKS ........................ 93 4.2.6 Coefficients for Percentage of All Students Passing the 2003 10th Grade Mathematics TAKS ...................................... 94 4.2.7 Coefficients for Percentage of All Students Passing the 2004 10th Grade English Language Arts TAKS ........................ 95 4.2.8 Coefficients for Percentage of All Students Passing the 2004 10th Grade Mathematics TAKS ...................................... 96 4.2.9 Coefficients for Percentage of All Students Passing the 2005 10th Grade English Language Arts TAKS ........................ 97 4.2.10 Coefficients for Percentage of All Students Passing the 2005 10th Grade Mathematics TAKS ...................................... 98 4.2.11 Coefficients for Percentage of All Students Passing the 2006 10th Grade English Language Arts TAKS ........................ 99 4.2.12 Coefficients for Percentage of All Students Passing the 2006 10th Grade Mathematics TAKS .................................... 100 4.3 Distribution of Respondents by Gender .................................. 102 4.4 Distribution of Respondents by Age ........................................ 102 4.5 Distribution of Respondents by Professional Position ............... 103 4.6 Distribution of Respondents by Highest Degree Earned ........... 103
Table 4.7 Distribution of Respondents by Years of Experience in
Education ............................................................................. 104 4.8 Distribution of Respondents by Certifications Held .................. 105 4.9 Why is TAKS Given as a Statewide Test to ELLs? .................... 107 4.10 What are the Anticipated Results of Statewide Testing for ELLs? ............................................................................ 111 4.11 What are the Actual Results of Statewide Testing for ELLs? ............................................................................ 115 4.12 What are the Intended Consequences of TAKS for ELLs? ....... 118 4.13 What Has Happened to ELLs Because of TAKS? .................. 122
4.14 What Problems Have Occurred for ELLs Due to TAKS? ......... 126 4.15 What Changes Have Occurred for ELLs Due to TAKS? .......... 129 4.16 What Recommendations are suggested for Improvement of ELLs Performance on TAKS?............................................. 132 4.17 What are the Recommendations, with Greatest Value, are offered for ELLs Success on TAKS? ................................. 136
CHAPTER I INTRODUCTION For years, the English language learners (ELLs) have been subjected to educational systems that did not expect them to rise to the same standards as their native English-speaking peers (Winograd, 2002). Although that it can take several years to acquire the second language skills needed to be successful in school (Collier, 1989), too often English language learners born in the U.S. are still in English as a second language (ESL) classes and far behind their grade level peers in the content areas by the time they reach high school (Freeman & Freeman, 2002). One factor that should be considered in this failure to reach grade level requirements is that language may constitute an element of selfidentity. It is possible that minority groups are insistent on retaining their ethnic language as their “first.” English proficiency then would be a mere elective instead of an indispensable learning tool. If this is the case, schools are being held accountable for the consequences of a sociocultural phenomenon that is beyond their limited powers to address. Public schools are under close scrutiny. Since they are supported by public funds, there is an increasing demand for accountability. The No Child Left Behind Act (NCLB) now requires all students to be accounted for in any state‟s assessment system, although that has not always been the case (Abedi, 2004). School districts are now required to 1
2 clearly demonstrate they deserve, and effectively utilize public funding. In itself, this is not a disturbing trend. Institutions that are wholly or partly supported by public funds should be accountable. This is essentially a consequence of democracy. A government that is created by, and for the people, is so unlike an aristocracy that is not required to serve a constituency beyond the guarantee of protection from marauders or invading armies. The U.S. system of government empowers the state to undertake measures that guarantee the common good. This goes beyond the guarantee of physical safety, since the term “common good” has a wider application, and implies a calculated sensitivity to every citizen‟s pursuit of happiness. While education is not categorized as a fundamental right, it is perceived as primary among a bundle of values essential for every person‟s quest for self-fulfillment and happiness. This explains why there is little argument about whether the government should be involved in education at all, and whether this is an endeavor better left to the private sector (Abedi, 2004). The government‟s involvement in education opens a wide avenue for the analysis and evaluation of results. In today‟s world, it is not enough that public schools have adequate facilities, although this constitutes one level of analysis. It is important that schools are safe and teachers are qualified, although in the hierarchy of priorities considered for evaluating schools, these outcomes are not standard. Schools are judged principally based on the amount of learning that takes place in
3 their classrooms. As an internal act, the evidence of learning is analyzed from scores students obtain on standardized assessments. Institutions are now facing an ever-increasing demand for accountability. There is pressure from every conceivable corner to make public schools accountable to their stakeholders. This means that it is not enough for students to learn in school; it is equally important that learning should occur in ways that are measurable. If students are unable to demonstrate what they have learned, it is presumed that no learning took place at all. The time when public schools are allowed to operate without proven success is over. It is appropriate to inquire about the valid manifestations of success and learning, and how they may actually be measured. Cultural construct renders school rankings flawed to a certain extent since they become less accurate as a measure of the faculty and administration‟s performance. Instead, they become unintended indicators of the ethnicity of the students to which schools cater (Abedi, 2004). Statement of the Problem High stakes assessment systems are meant to bring attention to the needs of ELLs, who are most at risk of not reaching the educational goals set for them (Anderson, 2004). But what results do statewide accountability tests really produce for ELLs (Anderson, 2004)? Assessment systems usually produce both positive and negative consequences (Anderson, 2004). The positive and negative consequences
4 of assessments are what is called „washback‟ (Alderson & Wall, 1993), or how the results of an assessment affect the stakeholders taking the test (Anderson, 2004). While quantifiable washback effects such as increased dropout rates or increased referral to Special Education have been researched, assessment washback is more complicated than numbers alone can tell (Anderson, 2004). Students who qualify for Special Education may be allowed to take alternative assessments in lieu of the state assessments such as the Texas Assessment of Knowledge and Skills (TAKS). It is interesting to note that while the numbers of African-American and Hispanic students are over-represented in Special Education, about eight to nine percent of ELLs are identified as receiving Special Education services in the United States (D‟Emilio, 2003; Zehler, Fleischman, Hopstock, Pendzick, & Stepherson, 2003). While these assessments are not on grade level, schools are expected to demonstrate that, based on students‟ scores on alternative assessments, improvement in academic performance is taking place. Data are needed that tell us more about the full range of intended and unintended consequences occurring in schools today (Anderson, 2004). Since school rankings affect student and faculty morale, they serve more as a force for the preservation of the status quo than a force for improvement in student performance. A school that works hard to ensure that learning occurs, and that its students progress academically,
5 but which has a large proportion of ELLs, will risk being ranked as underperforming because the measure used to evaluate its performance is blind to this important demographic reality. One way to get at these data is by talking with the stakeholders at the schools. Educators are the ones who deal directly with the impact of high stakes assessments, but are overlooked in research. While teachers‟ opinions are often cited as anecdotal evidence that a problem exists, their expert observations often go unrecorded in any systematic way (Anderson, 2004). Standardized assessments are a measure for holding schools accountable for student learning. At the present time, schools in Texas are ranked Exemplary, Recognized, Acceptable or Underperforming, depending on the performance of their students in the Texas Assessment of Knowledge Skills (TAKS). This produces a vicious cycle since exemplary schools attract the best students who may leave underperforming schools to seek what is perceived to be a higher quality of instruction in higher ranked schools. These labels tend to have a selffulfilling effect, or at least they make it difficult for underperforming schools to achieve higher performance scores on standardized tests, since they face the additional burden of surmounting language barriers and a history of low performance. Related to this concern is the prevailing system of voluntary segregation in most zones and districts. Some schools have either a
6 predominant population of White, Hispanic, or African-American students. Each of these student groups is given the same tests, and yet they have varying degrees of proficiency in the language in which the assessments are given. It begs to be asked whether these assessments, in fact, measure learning and whether they are linguistically and culturally neutral. The implication is that these students will be able to answer the test questions even if they do not have equal exposure to cultural references that may frame some of the test questions. This study is intended to explore what educators perceive as the consequences of statewide assessment for ELLs and what they observe as actually occurring (Anderson, 2004). Purpose of the Study The purpose of this study was to determine the impact of highstakes testing on ELLs. This was shown in both the quantitative and qualitative dimensions of the study. Data obtained from TEA were used to determine whether there is a relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th Grade TAKS tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005 and 2006. To support the quantitative aspect, this study explored what certified ESL teachers, non-certified ESL teachers who teach ELLs, administrators, and district ESL personnel viewed as the impact that high stakes
7 standardized assessments have on ELLs, ESL curriculum and instruction, and what they observed as actually occurring. Research Questions Quantitative Is there a relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005, and 2006? Hypotheses H01: There is no statistically significant relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in English Language Arts given in 2003, 2004, 2005, and 2006? H02: There is no statistically significant relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in Mathematics given in 2003, 2004, 2005, and 2006? Qualitative The major question addressed by this study was: What are the anticipated and observed consequences of the statewide testing, specifically TAKS, on ESL curriculum and instruction as viewed by certified ESL teachers, non-certified ESL teachers who teach ELLs, school administrators, and district ESL personnel?
8 This major question was explored using the following probes: 1. Why is TAKS given as a statewide test? 2. What are the intended consequences of this statewide testing? (Or what has happened because of TAKS?) 3. What problems have occurred related to or because of TAKS? 4. What changes were caused by this statewide testing? 5. What are your recommendations to improve this statewide testing? 6. What needs to be done for the ESL students to improve their performance in general and specifically for this statewide test? Description of the Research Design The study analyzed the issues and challenges faced by ELLs and the public schools that serve them. Quantitative data for this research were gathered from the Texas Education Agency (TEA) regarding the percentage of ELLs and the performance of 10th grade students from the major urban high schools in Texas on the Texas Assessment of Knowledge and Skills (TAKS) tests in English Language Arts and Mathematics for 2003, 2004, 2005, and 2006. Qualitative data were derived from one-on-one and focus group interviews and an on-line questionnaire focusing on respondents‟ views and opinions about the various ways that standardized assessments impact ELLs.
9 Assumptions Fraenkel and Wallen (2003) stated that an assumption is anything taken for granted rather than tested or checked. This study is no different and the following assumptions were made: (a) that the first language of the ELLs is Spanish and they have varying degrees of fluency in the English language; (b) that the ESL curriculum is appropriate for the mastery of the TAKS test for the ELLs; (c) that the on-line open-ended qualitative questionnaire will be completed by the respondents on time; and (d) that the respondents in the focus groups will truthfully express their views and opinions regarding issues or concerns brought to the group. Limitations of the Study Limitations of the study included several factors: mainly the qualitative questionnaire and the manner in which respondents gave their responses. The questionnaire may have vague questions open to more than one interpretation. The pilot study helped in streamlining the questionnaire to remove or modify such vague issues or concerns. Another limitation may have been the manner in which the respondents answered the question. For one reason or another, they may not have truthfully answered some of the questions. The respondents may or may not have completed the questionnaire due to no ready access to a computer or they just did not want to complete the questionnaire. These non-respondents became part of the mortality factor involved in the
10 study. Responses to the open-ended questions became difficult to classify under a certain category. This was facilitated through the NonNumerical, Unstructured Data, Indexing Searching & Theorizing Vivo“Nudist Alive” (NVivo) software system (Version 7.0) and by the focus group interviews where the respondents helped determine the category of such responses. A factor that may have been encountered in the quantitative dimension of the study was the lack of intended data for the study. Diligent efforts were made to gather data from available sources. Delimitations of the Study The questions for the on-line qualitative questionnaire may have been a delimitation of the study. The pilot study contributed to the improvement of the qualitative tool. Another delimitation may have been the choice of participants, especially in the focus groups. The “snowball technique” addressed this issue. Better interaction happened with added „quality‟ members to the focus groups. Qualitative data are available and the inclusion of the quantitative aspect of the study provided a challenge and an opportunity to determine if certain factors of the study have any impact on the ELLs. Definition of Terms Content Standards are broad descriptions of the knowledge, skills, and understandings that schools should teach and students should acquire in a particular subject area (McLaughlin & Shepard, 1995).
11 English Language Learners (ELLs) is the preferred term to describe a student whose native language is other than English (Chamot & O‟Malley, 1994). These students require instructional modifications, and eventually take the TAKS after two years of enrollment in the school districts. High Stakes Assessment is an assessment in which student promotion (i.e., high school graduation) can be denied if the scores do not reflect competence (NCBE, 1997). Limited English Proficient (LEP) refers to a student with a language background other than English, and whose proficiency in English is such that the probability of academic success in an English-only classroom is below that of an academically successful peer with an English-language background (CCSSO, 1992). No Child Left Behind Act (NCLB) of 2001 (PL – 107 – 110). It is the reauthorization of the Elementary and Secondary Education Act (ESEA). Opportunity-to-learn (OTL) Standard defines the level and availability of programs, staff and other resources sufficient to enable all students to meet challenging content and performance standards (McLaughlin & Shepard, 1995). Performance Standards are concrete examples and explicit definitions of what students have to know and be able to do to demonstrate that such students are proficient in the skills and knowledge framed by the content standards (McLaughlin & Shepard, 1995).
12 Standardized Assessments include the Texas Assessment of Knowledge and Skills (TAKS) and the State and Locally-Developed Alternative Assessment (SLDAA) for students who are exempted from the TAKS. A standardized assessment is a measurement of what students know and can do (McLaughlin & Shepard, 1995). Standards-based Reform requires setting standards of performance in academic subject areas as a means of improving the substance of school curricula and increasing the motivation and effort of students, teachers, and school systems and thereby improving student achievement (McLaughlin & Shepard, 1995). Washback shows how the results of an assessment affect the stakeholders taking the test (Alderson & Wall, 1993). Significance of the Study Expected outcome of this study may possibly provide additional valuable data for writers or researchers in regard to biases in standardized assessments that may encourage school districts to develop assessments that truly measure learning without the nullifying effect of linguistic and cultural bias. Additionally, this study enhances the reliability of standardized assessments as a tool in determining accountability where the performance of English language learners is concerned.
13 Organization of the Study Chapter I identifies the problem this study addresses: the impact of high stakes assessments on the curriculum and instruction of English language learners. It includes the hypotheses and research questions of the present study. Included are the definitions of terms valuable to the study. Chapter II includes the review of literature about the essential conditions and factors regarding the NCLB Act, the AYP implications for concerned schools, high-stakes, statewide assessments and the implications and challenges they present to the preparation and education of ELLs. The information reveals the difficulties that English language learners face when taking these high stakes assessments, the possible positive and negative consequences and possible “washback” related to the assessments. A mixed methods study is identified and expounded in Chapter III. Quantitative data for this research were gathered from the Texas Education Agency regarding the percentage of ELLs and the performance of major urban high schools in Texas in the statewide test (Texas Assessment of Knowledge and Skills) for 2003, 2004, 2005, and 2006. Qualitative data were derived from an on-line, open-ended questionnaire and interviews that focused on the respondents‟ views and opinions about the varied ways standardized assessments impact English language learners.
14 Results of the study are presented in detail in Chapter IV. Quantitative results include the available data collected from Texas Education Agency. Results of computations employing the Statistical Package for the Social Sciences (SPSS) statistical package, (Version 14.0) are shown in tabular presentations and explanations regarding the relationship among the variables are included. Qualitative results include the participants‟ views and opinions on the impact of high stakes testing on English language learners and the information collected from the online, open-ended questionnaire, individual and focus group interviews. Major findings of the study are discussed in Chapter V. Impact of high stakes standardized assessments on English language learners are also summarized. Other relevant factors that influenced this study are presented, as well as recommendations for future research.
CHAPTER II REVIEW OF LITERATURE Key issues and concerns about the No Child Left Behind Act (NCLB) of 2001 and the Adequate Yearly Progress (AYP) are major parts of the review of related literature. Included are the principles and accountability involved in high-stakes testing and the descriptions and accommodations given to the ultimate beneficiary of the efforts exerted by the federal and state policymakers, the school and district administrators – the learners, specifically, the English language learners who strive to be better citizens of this country. Short description of related studies on statewide testing and English language learners (ELLs) are given to show their tie-in with this study. No Child Left Behind (NCLB) Historical Perspective The NCLB Act of 2001 (PL – 107 -110), is the reauthorization of the Elementary and Secondary Education Act (ESEA). The ESEA was first passed in 1965 with the goal of improving the U. S. educational system by providing better education for students in poverty through an increase in services to them. The ESEA provided federal funds for schools but did not require accountability in the use of those funds. In 2003, the Center of Educational Policy clarified why accountability was not part of ESEA in 1965: “At that time, the federal role in education was marginal, most state education agencies had very limited authority and capabilities, and local 15
16 people were extremely wary that more federal aid would bring federal control” (p.iv). The National Assessment of Educational Progress (NAEP) was initiated as a federal testing program at about the same time when ESEA came into existence. NAEP was tasked to report how the nation‟s students were performing on selected items at the three grade levels --- 4th, 8th and 12th. Brennan (2004) reported that there were fears that the NAEP might become a “high-stakes federal testing program” found in some European countries. He explained that, “to help preclude that possibility, it was written into law that NAEP could not report scores for individual students” (p.2). The NAEP evolved through the 1980s and early 1990s from a reporting of item scores to test scores and then, on a trial basis, to a reporting of scores that addressed achievement levels (below basic, basic, proficient, and advanced). It is currently used to confirm state NCLB testing results which, according to Brennan, “is the de facto elevation of NAEP to a federally-mandated highstakes testing program” (p.9). Through the NCLB Act, policymakers in Washington seek to raise academic achievement in the nation by requiring schools to assess all students on specified content areas and report their progress toward proficiency. Focus of NCLB is on core academic subjects as defined in the law: “The term „core academic subjects‟ means English, reading or language arts, mathematics, science, foreign language, civics, and government,
17 economics, arts, history, and geography” (U.S. Department of Education, 2002). The premise of NCLB is that our nation‟s schools are failing. Thus, the purpose of NCLB is raising the achievement of all students and eliminating the achievement gap among students differentiated by race, ethnicity, poverty, disability, and English proficiency. Since this Act redefines the federal role in education policy that has traditionally been a state responsibility, it merits the attention of educators, parents and citizens. Because the NCLB Act has an impact on the teaching and the learning of the core content areas, including languages, language educators need to be informed about it. If a roomful of educators were asked which word or phrase best sums up No Child Left Behind (NCLB), many would say accountability. Others might propose student achievement, proficiency or raised expectations. But perhaps the most accurate word to encapsulate the United States‟ most ambitious federal education law – which proposes to close achievement gaps and aims for 100% student proficiency by 2014 - is testing. Certainly, the focus on holding schools accountable for student achievement on standardized tests sets NCLB apart from previous versions of the law. (Guilfoyle, 2006).
18 Description of the Key Factors There are four key elements in the NCLB Act (Rosenbusch, 2005): (1) Accountability. States are required to establish a definition of student proficiency in the core academic subjects of Reading/Language Arts, Mathematics and Science through prescribed indicators and set a timetable to bring all students in all subgroups up to the defined levels of proficiency by 2013-2014. The school must report to parents their child‟s progress in each targeted academic subject annually, and the state is required to report the results of students‟ performance on the annual tests for every public school to parents and the community. Schools that fail to meet state-defined AYP toward their defined goals for two years are identified as needing improvement. Schools that have not met AYP after four years are subject to restructuring or reconstitution. (2) Testing. States must develop and administer annual tests that define the proficiency that all students are expected to reach in Reading/Language Arts, Mathematics, and Science. States must include a sample of students in fourth and eighth grades in a biennial NAEP in Mathematics and Reading to verify state assessments. NCLB requires that by School Year (SY) 2005-2006, each state must measure every child‟s progress in Reading and Mathematics in each of grades 3 through 8 and at least once during grades 10 through 12. In the meantime, each state must meet the requirements of the previous law reauthorizing ESEA (the Improving America‟s Schools act of 1994) for
19 assessments in Reading and Mathematics at three grade spans (3-5; 6-9; and 10-12). By SY 2007-2008, states must have in place Science assessments to be administered at least once during grades 3-5; grades 6-9; and grades 10-12. States must ensure that districts administer a test of English proficiency to measure oral language, Reading and Writing skills in English to all limited English proficient students, as of SY 2002-2003. Students may still undergo state assessments in other subject areas (i.e., History, Geography, and Writing skills), if and when the state requires it. NCLB requires assessments only in the areas of Reading/Language Arts, Mathematics, and Science. (3) Teacher Quality. Public elementary and secondary school teachers who teach core content areas are required to be “highly qualified,” which is defined as having full state certification (may be attained through alternate routes specified by the state), holding a bachelor‟s degree, and having demonstrated subject matter competency as determined by the state under NCLB guidelines. States are required to develop a plan by the end of 20052006 to ensure that every teacher is highly qualified to teach in his or her core content area. (4) Scientifically-Based Research. The NCLB Act requires that all educational decisions be informed by scientifically-based research as defined in the legislation. The NCLB Act funds for Reading First Grants, for example, are to be used for methods of reading instruction backed by scientifically-based research.
20 Expectations for Parents Due to NCLB (from Collegeboard.com) (1) New standards for students will require that beginning 2005, students in grades 3 through 8 must be tested in Mathematics and English to ensure they are meeting state standards. Students in Grades 10 through 12 will be tested at least once. By 2007, states will begin testing students in Science as well. Results of the yearly tests will be known to parents. NCLB requires that school districts provide parents with an annual “report card” that shows how well students in each school performed. The information is broken down by race, ethnicity, gender, disability status, and other categories so that parents will know how well each school is doing in educating minority students or those with disabilities. (2) By the end of SY 2005-2006, teachers must be “highly qualified” in the subjects they teach. States will determine what skills teachers must have to be “highly qualified”, but the requirements could include a degree in the subject they teach or extra training. States must provide annual report cards about teacher certifications, including the percentage of classrooms in the state not taught by highly qualified teachers. Principals must also maintain information about whether or not their school‟s teachers meet the requirements. (3) Each year, schools must increase the number of students who achieve state standards. At the end of 12 years, all students should be able to pass the tests. Schools that fail to achieve this progress will be targeted
21 for improvements that could include increased funding or staff and curriculum changes. (4) NCLB requires school districts to notify parents if the child‟s school has been identified as needing improvement as a result of failing to increase the number of students meeting state standards. (5) About half of all public schools receive funding to help students from low-income families. If such a school is targeted for improvement and fails after two years, parents can choose to transfer their child to another school or enroll in free tutoring. Parents have this choice for as long as the school fails to adequately perform. Response to NCLB (Rosenbusch, 2005) NCLB has engendered controversy that is centered in part on the increased role of the federal government in educational policy. A majority of Americans believe that decisions about what is taught in public schools should be made at the local level by the school board (61%), rather than at the state level (22%) or the federal level (15%) (Rose & Gallup, 2003). Results of a 2004 survey indicate that they disagree with “the major strategies NCLB uses to determine whether a school is or is not in need of improvement” (Rose & Gallup, 2004, p.2). For example, 83% of those surveyed believe that testing only in English and Mathematics will not yield a fair picture of the school, 73% say it is not possible to judge a student‟s proficiency in English and Mathematics on a single test, and 81% are concerned that basing decisions about school on students‟ performance in
22 English and Mathematics will mean less emphasis on art, music, history and other subjects. In the U.S. Department of Education, there is support for high standards and high expectations for every child, but the NCLB focus on standardized testing is resulting in a narrowing of the curriculum and a “sorting of students” (Marshak, 2003, p.229) and “could halt the development of truly significant improvements in teaching and learning” (Lewis, 2002, p.179). The National Education Association supports the NCLB Act in its goal but views it as an obstacle to improving public education because of its focus on “punishment rather than assistance”, and “mandates rather than support for effective programs” (National Education Association, n.d.). Adequate Yearly Progress (AYP) Purpose and Support to NCLB The No Child Left Behind Act of 2001 (NCLB; Public Law No. 107-110, 115 Stat. 1425, 2002), the most recent reauthorization of the Elementary and Secondary Act of 1965, holds states using federal funds accountable for student academic achievement. States are required to develop a set of highquality, yearly student assessments that include, at a minimum, assessments in Reading/Language Arts, Mathematics and Science. Each year, they must report student progress in terms of percentage of students scoring at the “proficient” level or higher. This reporting is referred to as adequate yearly progress (AYP). A state‟s definition of AYP should also
23 include high school graduation rates and an additional indicator for middle schools to reach the “proficient” level or higher, which must be no more than 12 years after the start date of the 2001 – 2002 school year, provided that the first increase occurs within the first 2 years (Abedi, 2004). AYP will be reported for schools, school districts, and the state for all students. In addition, AYP must be reported for the following subgroup categories of students: (a) economically disadvantaged students, (b) students from major racial and ethnic groups, (c) students with disabilities, and (d) students with limited English proficiency (LEP). According to the educational statistics for 2000 – 2001 school year, the total number of students labeled as LEP in the nation‟s public schools is more than 4.5 million or 9.6% of total enrollment; (National Center for Education Statistics [NCES], 2002). States are continuing to find new ways to calculate AYP under the NCLB, in order to increase the number of schools and districts that meet the student achievement targets set by law. Over the past few years, the U.S. Department of Education (ED) has allowed states to make many changes in the way they determine AYP, including the following: (1) confidence intervals, which make allowances for natural fluctuations in test scores and essentially bolster a school‟s or subgroup‟s percentage of students scoring at proficient levels; (2) performance indices that allow schools to get “partial credit” for the performance of students below the proficient level; (3) retesting, which allows students to retake a different
24 version of the same test and permits schools to use a student‟s best score to count toward AYP, and (4) increased minimum subgroup sizes, which mean that in many schools, subgroups do not get counted for AYP purposes. The changes have the effect of making it easier for the schools to make AYP, early indications are that the number of schools not making AYP has leveled off, despite predictions that this number would increase as proficiency targets rose (Olson, 2005). Changes and Updates In NCLB‟s original conception, determining AYP for a subgroup of students, a school, or a district was already fairly complicated. States had to establish, for every year between 2003 and 2014, a set of ever-increasing state targets in terms of the percentage of students scoring at the proficient level or above on annual tests, with a final goal of 100% proficiency in 2014. If at least 95% of the students in each subgroup are tested, and if all students and subgroups meet the state proficiency targets, the school or district makes AYP. The school has to meet targets for an additional academic indicator, such as the graduation or attendance rate. The law has a “safe harbor” provision: if a school or subgroup fails to meet the state targets, it could still make AYP if it reduces the number of students who are not proficient from the previous year by 10%, and meets its additional academic indicator. Some other state changes that have been approved are briefly summarized below (Center on Education Policy, 2005):
25 Minimum subgroup size. To make AYP, schools and districts must meet achievement targets for each significant subgroup of students enrolled, such as African-American students, low-income students, or students with disabilities. Higher minimum subgroup sizes mean that in many schools, subgroups do not get counted for AYP purposes. Thirteen states increased their minimum subgroup sizes in 2004; ten more did so in 2005. The trend is away from a single minimum size and toward larger subgroup sizes, different subgroup sizes for different subgroups and/or purpose, and the use of formulas for determining subgroup sizes. Georgia is one state that uses a formula approach. Its subgroup size varies according to the size of the school; the minimum size is either 40 students or 10% of a school‟s student population, whichever is greater, with a cap of 75 students. Participation averaging. NCLB requires 95% of the students in every school and every subgroup within a school to take each subject test required by the Act. If this test participation requirement is not met, the school cannot make AYP even if its test scores meet state targets. In March 2004, the Department relaxed this requirement, allowing states to average their participation rates over two or three years, so that a 94% participation rate one year could be balanced by a 96% participation rate the following or previous year. In 2005, six states changed their accountability plans to incorporate this new policy, in addition to the 32 that did so last year.
26 English language learners. Initially the U.S. Department of Education (ED) required all English language learners to be tested with the same grade-level tests as other students. In response to state and local criticism, the Department revised its policy in February 2004 to allow states to exempt immigrant students who are in their first year of enrollment in a U.S. school for less than one year from taking the regular state English Language Arts tests. These students still have to take an English language proficiency test and a Mathematics test, but the results need not count toward AYP. When calculating AYP for the subgroup of English language learners, states can also count the progress of former English language learners for two years after they reach English proficiency. Six more states adopted these changes in 2005, in addition to the 36 states that did so in 2004. Extra time is given for students with disabilities and English language learners to graduate. In 2005, eight states received approval from ED to count students with disabilities and/or English language learners as graduating on time even if they need extra years of high school. Seven states received permission to do this in 2004. For students with disabilities, their individualized education plans would need to call for extra years of high school beyond age 18. English language learners can be counted as graduating on time if it takes five years, or as determined on a case-to-case basis (Center on Education Policy, 2005). Identifying districts for improvement. In 2005, ED approved amendments requested by 13 states to identify a district as being in need of
27 improvement only when it does not make AYP in the same subject and across all three grade spans (elementary, middle and high school) for two consecutive years. In 2004, 18 states made this change. California attempted to have ED accept a relatively lenient method that exempted districts where low-income students reached a certain level on state tests. ED rejected that method, and California settled on the grade span approach instead (Davis & Sack, 2005). Annual measurable objectives. Eleven states changed their annual score targets in 2005; four states did so in 2004. For example, Florida was allowed to change its schedule of annual measurable objectives so that targets would increase in smaller increments annually, rather than in large increments every three years (Olson, 2005); Virginia did so as well. Several other states, including Alabama, Alaska, New Mexico, and North Carolina, changed their annual targets because they were introducing new assessments. NCLB is a demanding law. The achievement goals are ambitious, and the burden on states and districts of declaring schools in need of improvement and then imposing sanctions on them is high. To try to meet these demands, states have a strong incentive to keep the numbers of schools and districts not making AYP as low as possible. Unable to change the fundamental requirements written into the law, states are using administrative methods to lessen the numbers of schools and districts not making the AYP – confidence intervals, indexing, and other techniques.
28 Education Secretary Margaret Spellings has been more flexible than her predecessor in policies regarding students with disabilities, and in granting special exemptions to some districts in the areas of school choice and supplemental educational services (tutoring). Secretary Spellings has decided to allow the Chicago school district to provide tutoring despite the fact that the district has been identified for improvement (Gewertz, 2005). This exemption was then extended to New York City, Los Angeles, Boston, Memphis, Anchorage, and Dayton. This was a regulatory change. Secretary Spellings went further with four districts in Virginia by suspending a key element of the law itself, invoking a clause in NCLB that allows the Secretary of Education to do so. Her action exempted these districts from the law‟s requirement that they provide school choice before tutoring (Olson, 2005). Secretary Spelling‟s letter to Virginia officials indicates that this is a pilot program intended to raise the numbers of students receiving supplemental educational services (Spellings, 2005). In addition, districts in the five states most affected by Hurricane Katrina were allowed to postpone, for one year, the consequences that follow when a school is in need of improvement, such as tutoring, restructuring, and corrective action (Olson & Davis, 2005). ED‟s willingness to make adjustments based on state and local experience is commendable. But on the downside, parents in many states would now find it difficult to understand what it means when a school does or does not make AYP, and what criteria were used to determine this
29 success or failure. For example, parents in Pennsylvania may see a report card that indicates that their child‟s elementary school has made AYP, but might wonder whether the school is improving or whether it simply made AYP as the result of what might be seen as a new “loophole” in the law. The parents probably would not understand that the school may have made AYP through the use of a 95% confidence interval, safe harbor with a 75% confidence interval, or the Pennsylvania Performance Index as a second safe harbor. In other states, parents of English language learners, students with disabilities, or other subgroups may not realize that raising the minimum subgroup sizes means that their children no longer count for AYP purposes at the school level. They might not realize that the use of confidence intervals allows for considerable leeway in a subgroup‟s test scores not available to larger groups of students, and that this is occurring despite the assertion that improving achievement for subgroups is a major focus of the law. Other drawbacks to the increasing complexity may contribute in the difficulty of discerning clear trends in the number of schools and districts not making AYP, because the rules governing AYP keep changing every year. Amid these changes, it is impossible to determine whether an increase in the number of schools making AYP within a state is due to better teaching and learning or NCLB rule changes. The constant rule changes, particularly the use of large confidence intervals and ever-increasing minimum subgroup sizes, may raise questions about whether the law is being watered
30 down so much that it shortchanges the very groups of disadvantaged children that it aims to help. Public support may wither if the implementation of the law is perceived as deceptive or confusing. As states continue to learn from one another about the new types of flexibility that ED is allowing, and as state achievement targets continue to rise until 2014, changes in AYP policies are likely to occur at a more rapid pace, at the expense of the public‟s ability to understand these changes. More transparency is needed at both the state and federal levels. States must fully and clearly explain their rationales for requesting changes to accountability plans. Once changes are approved by ED, they should be explained in such a way that the public understand how AYP is determined. At the federal level, ED should more systematically and promptly publicize its decisions about what types of changes to state accountability plans are and are not acceptable, and why. The current process of granting changes does not help state officials learn from other states‟ experiences, nor does it help them understand how ED is interpreting the intent of the law. Adequate Yearly Progress (AYP) and Limited English Proficient (LEP) Students Definition of English Language Learners (ELLs) and LEP Limited English Proficient (LEP) students are students who lack sufficient English skills to participate in a regular education, all-English speaking classroom. English Language Learner (ELL), according to Rivera
31 and Stansfield (1998), is a positive way to refer to any LEP student in English. NAEP does not provide a definition of the LEP population; instead it presents criteria for the inclusion of LEP students. NAEP inclusion criteria indicate that: A student who is identified on the Administration Schedule as LEP and who is a native speaker of a language other than English should be included in the NAEP assessment unless: (a) the student has received Reading or Mathematics instruction primarily in English for less than 3 school years including the current year , and (b) the student cannot demonstrate his or her knowledge of Reading or Mathematics in English even with an accommodation permitted by NAEP (NCES, 2001). Due to the importance of LEP subgroups in NCLB accountability and reporting, NCLB provides an operational definition of LEP (NCLB, 2002). According to this definition: The term „limited English proficient‟, when used with respect to an individual, means an individual (a) who is aged 3 through 21; (b) who is enrolled or preparing to enroll in an elementary school or secondary school; (c) who was not born in the United States or whose native language is a language other than English; who is a Native American or Alaska Native, or native resident of the outlying areas; and who comes from an environment where a language other than English has had a significant impact on the individual‟s level of English language proficiency; or who is migratory, whose native language is a language other than English, and who comes from an environment where a language other than
32 English is dominant; and (d) whose difficulties in speaking, reading, writing, or understanding the English language may be sufficient to deny the individual the ability to meet the State‟s proficient level of achievement on State assessments described in section 111(b)(3); the ability to successfully achieve in classrooms where the language of instruction is English; or the opportunity to participate fully in society. The term “English language learner” (ELL) is a recent designation for students whose first language is not English. This group includes students who are just beginning to learn English as well as those who have already developed considerable proficiency. The term reflects a positive focus on what these students are accomplishing – mastering another language- and is preferred by some researchers to the term “limited English proficient” (LEP), the designation used in federal and state education legislation and most national and state data collection efforts (August & Hakuta, 1997; LaCelle-Peterson & Rivera, 1994). The ELL population is highly diverse, and any attempt to describe the group as a whole, as with any diverse group of people, is bound to result in inaccurate generalizations. While this group of students share one important feature - the need to increase their proficiency in English - they differ in many other important respects. ELLs are a diverse cross-section of the public school student population. The primary language, cultural background, socio-economic status, family history, length of time in the
33 United States, mobility, prior school experiences, or educational goals of any student in this group can distinguish him or her from any other ELLs. ELLs represent a rapidly growing, culturally and linguistically diverse student population in the United States. In 2000-2001, LEP students comprised nearly 4.6 million public high school students. The majority were Spanish speakers (79.0%), followed by Vietnamese (2.0%), Hmong (1.6%), Cantonese (1.0%), and Korean (1.0%). Since the 1990-1991 school year, the limited English proficient population has grown approximately 105%, while the overall school population has increased by only 12%. English learners matriculate in schools throughout the nation, but most frequently in large urban school districts in the Sun Belt states, in industrial states in the Northeast, and around the Great Lakes. This trend is changing as immigrants move to more affordable suburban and rural areas and to areas where language-minority families are relative newcomers, such as the Midwest. More than half (56.1%) reside in four states alone: California (32.9%), Texas (12.4%), Florida (5.6%) and New York (5.2%) (Kindler, 2002). English learners represent one in four K – 12 students in California schools (California Department of Education, 2000). This population includes recent immigrants as well as children born in the United States. In the 2000-2001 school year, more than 44% of all LEP students were enrolled in Pre-K through Grade 3; about 35% were enrolled in Grades 4 – 8; and only 19% were enrolled at the high school level (Kindler, 2002). Many LEP students attend schools where most of their
34 peers live in poverty. There are numerous differences among English learners; for example, Spanish-speaking families tend to have lower parental educational attainment and family incomes than Asian-or Pacific-language families (August & Hakuta, 1997). Many criteria are used across the nation for identification of ELLs. Among the most commonly used criteria are Home Language Survey results and scores from English proficiency tests. There are reasons to believe that the Home Language Survey results may not be valid because of parents‟ concern over equity in education for their children, parents‟ citizenship issues, and communication problems (Abedi, 2004b). Similarly, there are concerns about the validity of current English proficiency tests, such as the Language Assessment Scales and other commonly used assessments (Zehler, Hopstock, Fleischman & Greniuk, 1994). Criterion-related validity coefficients, or the correlation between English proficiency tests and other existing valid measure of English proficiency, are not strong, explaining less than 5% of the common variance (Abedi, 2003). Finally, in terms of content and construct validity, there is little evidence that the contents of the existing English proficiency tests align sufficiently with commonly accepted English language proficiency standards, such as standards by Teachers of English to Speakers of Other Languages (Bailey & Butler, 2003). Issues and Other Considerations of LEP Disaggregated progress reports by subgroups mandated by the NCLB legislation will monitor the nation‟s goal of having “no child left behind.”
35 However, there are major issues in this disaggregated reporting among different subgroup categories (students who are economically disadvantaged, students from major racial and ethnic groups, students with disabilities, and LEP students). NCLB requirement for subgroup reporting may give the impression that students in the subgroup categories start the achievement race at about the same level and can progress with other students at about the same rate. This might be an overly optimistic view of the situation of less advantaged learners. By focusing this discussion on the consequences for schools enrolling LEP students, we see how putting into practice the policy may produce invalid assessment and unreliable reporting while exacerbating the burdens of current educators. Following is a discussion of some challenges in AYP measurement and reporting for LEP students. Results of research on the assessment of LEP students suggest a strong confounding of language and performance. LEP students exhibit substantially lower performance than non-LEP students in subject areas high in language demand. Studies suggest that the large performance gap between LEP and non-LEP may not be due mainly to lack of content knowledge. LEP students may possess the content knowledge but may not be at the level of English language proficiency necessary to understand the linguistic structure of assessment tools. Strong confusion of language factors and content-based knowledge makes assessment and accountability
36 complex for LEP students and, very likely, students in other targeted groups. Because of the strong effect of language factors on the instruction and assessment of LEP students, they lag far behind native English speakers. This leads to huge initial differences. LEP students start with substantially lower baseline scores. More important, unless LEP students‟ English language proficiency is improved to the level of native English speakerswhich is not an easy task- they will not be able to move at the same rate on the Adequate Yearly Progress line as do native English speakers. NCLB cannot have much of an effect on the initial performance differences between LEP and non-LEP students. A more sensible question here is whether or not NCLB can provide enough resources to schools with a large number of LEP students to help them increase these students‟ language proficiency to a sufficient extent that they can progress with their native English speaker peers in both instruction and assessment. Inconsistency in LEP classification across and within states makes AYP reporting for LEP students even more complex. If students are not correctly identified as LEP, how can their AYP be reliably reported at a subgroup level? Although NCLB attempts to resolve this issue by providing a definition for this group, its criteria for classifying LEP students may face the same problems as the existing classification system (Abedi, 2003; Zehler, Hopstock, Fleishman & Greniuk, 1994).
37 Inconsistency in the classification of LEP students may lead to more heterogeneity in the LEP subgroup. With a more heterogeneous population, larger numbers of students are needed to provide the statistically reliable results required by NCLB. The population of LEP students in many districts and states is sparse. In many states, there may not be enough students in a district or school to satisfy even the minimum number of 25 students suggested in the literature (Linn, Baker & Herman, 2002). Other researchers have argued that even 25 students may not be enough to provide statistically reliable results and have proposed a minimum group size of 100 students (Hill & DePascale, 2003). Considering a small number of LEP students in many districts and states, the small group size for LEP reporting would be another obstacle in regard to reliable AYP reporting. The LEP subgroup suffers from yet another major problem related to AYP reporting: The lack of stability of this group. In many states and districts across the nation, LEP students‟ level of English proficiency is reevaluated regularly, and if they reach a proficient level of English proficiency, they move out of the LEP subgroup. While this helps the more English-proficient students receive more appropriate instruction and assessment, it results in the LEP subgroup continuing to be low-performing. The students in this group will always be labeled as underachievers, and schools with large number of LEP students will be stuck in the “need for improvement” category.
38 Some states with substantial numbers of LEP students have expressed concern over this issue. They have proposed ideas and negotiated with the federal government to ease the level of possible negative impact that this situation may have on school, district, and state accountability. For example, Indiana and Delaware will continue to include exited LEP students in the LEP subgroup for 2 years after they have been determined to be proficient in English. Georgia plans to include LEP students as long as they still receive services through the English for Speakers of Other Languages program, even if they have met exit criteria (Erpenbach, ForteFast & Potts, 2003). In California, students re-designated as LEP will remain in the LEP category until they reach the proficient or above level on the California Standards Test in English-language arts for 3 consecutive years (California Department of Education, 2003). However, the question of whether this policy will provide a long-term solution to the problem of LEP subgroup instability or serve only as a temporary relief remains unanswered. The measurement of the academic achievement of LEP students is much more complex than what the NCLB legislation conceives. A fair assessment of students in the four targeted subgroup categories requires much more serious consideration than is outlined in the law. Despite attempting to solve the age-old problem of heterogeneity among LEP students, the NCLB seems to perpetuate it, thereby leaving more room for children to be left behind.
39 On the other hand, NCLB‟s attention to students in the four subgroup categories in general and to the LEP population in particular is a step in the right direction. Considering that Title III of NCLB requires assessment of LEP students‟ English proficiency on an annual basis and providing support to states to develop reliable and valid measures of students‟ proficiency is promising. Any decisions concerning assessment for all subgroups, particularly LEP students, must be informed by results of research and experience in the education community. Currently, several tests for measuring students‟ level of English language proficiency exist. Some of these tests have been used for many years by different states and districts. In spite of the existence of such tests, states are developing new English language proficiency tests with funding through NCLB‟s Enhanced Assessment Instruments. A reasonable explanation for this might be that states did not find that the existing tests provided reliable and valid measures of students‟ level of English language proficiency as required by NCLB. If this is the reason for the development of the new tests, then the test developers should be aware of problems in the existing tests to avoid the same problems in the new tests. For example, a careful review of some of the most commonly used language proficiency tests concluded that the tests differ considerably in types of tasks and specific item content and are based on different theoretical emphases prevalent at the time of their development (Zehler, Hopstock, Fleischman & Greniuk, 1994). This suggests that in the case of
40 some of the existing tests, the English language proficiency domain was not operationally defined before the test development process. This and similar studies and reviews should inform the development process of new tests. For example, it is imperative this domain be operationally defined before any effort in developing an English proficiency test. This definition should be based on current developments in the areas of psycholinguistics, developmental psychology, education, linguistics, and psychometrics. Content standards for English for speakers of other languages should also be considered (Bailey & Butler, 2003). In analyzing data from the administration of existing language proficiency tests, researchers have expressed concerns about the reliability and validity of these tests, the adequacy of the scoring directions, and the limited populations on which test norms are based. For example, analyses of several large data sets from different locations across the nation have shown validity problems in predicting LEP classification and lack of power in identifying different levels of English language proficiency among the LEP student population (Abedi, 2003; Abedi, Leon, & Mirocha, 2003). Those involved in the development of new English language proficiency tests should learn from such research and should conduct more analyses on the wealth of data that exists in this area. To be considered valid and reliable measures of English language proficiency, as outlined in the NCLB, new tests must first go through a rigorous validation process. Otherwise, there
41 may not be a reasonable justification to spend the limited NCLB resources on English language proficiency test development (Abedi, 2003). As a final thought, assessment and accountability of LEP students cannot be pursued in isolation of other important factors. An effective education system for LEP students that may lead to a successful AYP outcome should include at least three interactive components: (a) classification, (b) instruction, and (c) assessment. A problem in any one of these components may affect the other two. For example, a student misclassified as LEP student may be assigned a different curriculum and thus receives inappropriate instruction. Alternately, inappropriate instruction may result in low performance that may in turn result in misclassification. While each component has a unique role, they share common ground - the effect of language factors or barriers. Unnecessary linguistic complexity of assessment may threaten the validity and equitability of assessment among LEP students. Complex linguistic structure of instruction may negatively affect LEP students‟ ability to understand classroom instruction, and invalid assessment of students‟ level of English proficiency may result in misclassification. In a positive light, valid assessment may provide diagnostic information that can inform instruction and classification (Abedi, 2003). An effective way to help LEP students reach proficiency in the AYP model is to consider the broader picture using the interactive model. The following are few critical needs:
42 1. Improve current LEP classification and assessment. There is a need to establish a common definition of English language proficiency and substantially improve the validity of LEP instruments. Among other things, validity of LEP assessment can be enhanced by avoiding cultural biases and reducing unnecessary linguistic complexity of assessments. 2. Improve monitoring of progress. Schools need effective and valid data collection methods that can be used to monitor LEP progress at every stage of a student‟s education. Weaknesses must be quickly addressed with appropriate instructional strategies. 3. Improve teacher quality. LEP students need teachers who are well qualified in both language development and content, each of which plays a crucial role in LEP student achievement. The federal government can play a key role in this process by funding and encouraging programs that improve teacher capacity in this dual role. Teachers of LEP students should receive training in content delivery, language sheltering, and the teaching of the academic language. 4. Consider redesignated LEP students as part of the LEP subgroup that established the baseline score. State plans allowing redesignated students to remain in the LEP subgroup for only a limited time are temporary fixes. While new LEP students are added to the subgroup, redesignated students should be retained for AYP reporting. This “semicohort” approach to tracking LEP students allows the progress of
43 redesignated students to be counted toward subgroup AYP progress (Abedi, 2003). Based on the results of the research, policymakers, lawmakers, and decision makers are urged to take appropriate action to correct the inequities resulting from the NCLB in regard to the subgroups targeted by the legislation, particularly the LEP student subgroup. What is encouraging is that states, in collaboration with the federal government, are taking steps to remedy some of these issues. The hope is that these continued efforts will bring more fairness into the assessment of and accountability for LEP students (Abedi, 2003). High Stakes / Statewide Testing The 2001 reauthorization of the Elementary and Secondary Education Act (ESEA), also known as the No Child Left Behind (NCLB), carries testing and accountability requirements that will substantially increase student testing and hold all schools accountable for student performance. This legislation marks a major departure from the federal government‟s traditional role regarding elementary and secondary education. It requires that states administer Reading and Mathematics tests annually in grades 3 – 8 and during one year in high school starting in 2005 – 2006. These requirements will affect almost 25 million students each school year (National Center for Education Statistics, 2002). NCLB requires states to meet adequate yearly progress (AYP) goals to ensure school accountability for student achievement on state tests. Schools
44 that fail to achieve AYP goals face demanding corrective actions, such as replacement of school staff, implementation of new curriculum, extension of the school day or academic year, parental choice options, and, finally, complete reorganization. Today‟s widespread implementation of standards-based reform and the federal government‟s commitment to test-based accountability ensure that testing will remain a central issue in education for the foreseeable future. Test results can provide useful information about student progress toward meeting curricular standards. But when policymakers insist on linking test scores to high-stakes consequences for students and schools, they often overlook lessons from the long history of research (Abrams & Madaus, 2003). Current emphasis on testing as a tool of education reform continues a long tradition of using tests to change pedagogical priorities and practices. In the United States, this use of testing dates back to 1845 in Boston, when Horace Mann, then Secretary of the Massachusetts State Board of Education, replaced the traditional oral examination with a standardized written essay test. Internationally, high-stakes testing extends as far back as the 15th century in Treviso, Italy, where teacher salaries were linked to student examination performance (Madaus & O‟Dwyer, 1999). Principles of Testing Programs A 1988 examination of the effects of high-stakes testing programs on teaching and learning in Europe and in the United States (Madaus, 1988)
45 identified seven principles that captured the intended and unintended consequences of such programs. Current research confirms that these principles still hold true for contemporary statewide testing efforts. Principle 1: The power of tests to affect individuals, institutions, curriculum, or instruction is a perceptual phenomenon. Tests produce large effects if students, teachers, or administrators believe that the results are important. Policymakers and the public generally do believe that test scores provide a reliable, external, objective measure of school quality. They view tests as symbols of order, control and attainment (Airasian, 1988). Today‟s high-stakes testing movement relies on the symbolic importance of test scores. Forty-eight states currently require schools to provide the public with “report cards” (Edwards, 2003). Goldhaber and Hannaway (2001) found that the stigma associated with a school receiving a low grade on the state report card was a more powerful influence on Florida teachers than were the school-level sanctions imposed for poor test results. Principle 2: The more any quantitative social indicator is used for social decision making, the more likely it will be to distort and corrupt the social process it is intended to monitor. In other words, placing great importance on state tests can have a major influence on what takes place in the classrooms, often resulting in an emphasis on test preparation that can compromise the credibility or accuracy of test scores as a measure of student achievement.
46 We can assess whether this principle still applies today by examining the relationship between rising state test scores and scores on other achievement tests. Both old and new studies of this relationship (Amrein & Berliner, 2002; Haladyna, Nolen & Haas, 1991; Klein, Hamilton, McCaffrey & Stecher, 2000; Linn, 1998) show that improvements in the state test scores do not necessarily reflect general achievement gains. We can find examples of this second principle in two recent surveys of teachers‟ opinions. In one national study, roughly 40% of responding teachers reported that they had found ways to raise state-mandated test scores without, in their opinion, actually improving learning (Pedulla, Abrams, Madaus, Russell, Ramos & Miao, 2003). Similarly, in a Texas survey, 50% of the responding teachers did not agree that the rise in Texas Assessment of Academic Skills (TAAS) scores “reflected increased learning and high-quality teaching” (Hoffman, Assaf & Paris, 2001, p. 488). Principle 3: If important decisions are based on test results, then teachers will teach to the test. Curriculum standards and tests can focus instruction and provide administrators, teachers, and students with clear goals. A substantial body of past data and recent research confirms that as the stakes increase, the curriculum narrows to reflect the content sampled by the test (Jones et al., 1999; Madaus, 1991; McMillan, Myran, & Workman, 1999; Pedulla et al., 2003; Stecher, Barron, Chun & Ross, 2000). New York State, where the state department of education is requiring schools to spend more time on the NCLB-tested areas of Reading and
47 Mathematics, provides an example on how such pressure encourages schools to give greater attention to tested content and decrease emphasis on non-tested content. According to one school principal, “the art, music, and everything else are basically out the window… something has to go” (Herszenhorn, 2003). Principle 4: In every setting where a high-stakes test operates, the examination content eventually defines the curriculum. Pressure and sanctions associated with a state test often result in teachers using the content of past tests to prepare students for the new test. Several studies have documented that an overwhelming majority of teachers feel pressure to improve student performance on the state test. For example, 88% of teachers surveyed in Maryland and 98% in Kentucky believed that they were under “undue pressure” to improve student performance (Koretz, Barron, Mitchell & Keith, 1996a, 1996b). As an outgrowth of this pressure, the amount of instructional time devoted to specific test preparation often increased. Studies have found that teachers are spending a sizable amount of instructional time and using a variety of test-specific methods to prepare students for their state tests (Herman & Golan, n.d.; Hoffman, Assaf, & Paris, 2001). In North Carolina, 80% of elementary teachers surveyed “spent more than 20% of their total instructional time practicing for the end-ofgrade tests” (Jones et al., 1999, p. 201). A national survey found that teachers in high-stakes states were four times more likely than those in low-
48 stakes setting to report spending more than 30 hours a year on test preparation activities, such as teaching or reviewing topics that would be on the state test, providing students with items similar to those on the test, and using commercial test-preparation materials from previous years for practice (Pedulla et al., 2003). Principle 5: Teachers pay attention to the form of the questions of high-stakes tests (short-answer, essay, multiple-choice, and so on) and adjust their instruction accordingly. A wide variety of research confirms that test format does influence instruction in both positive and negative ways. Studies in states that require students to formulate and provide written responses to test questions show an increased emphasis on teaching writing and higher-level thinking skills (Taylor, Shepard, Kinner & Rosenthal, 2003). For example, in Kentucky, 80% of teachers surveyed indicated that they had increased their instructional emphasis on problem solving and writing as a result of the portfolio-based state test (Koretz, Barron, Mitchell, & Keith, 1996a). In several studies, teachers have reported decreases in the use of more time-consuming instructional strategies and lengthy enrichment activities (Pedulla et al., 2003). A study found that the format of the state test may adversely affect the use of technology for instructional purposes: One-third of teachers in high-stakes states said that they were less likely to use computers to teach writing because students were required to construct handwritten responses on the state test (Russell & Abrams).
49 Principle 6: When test results are the sole or even partial arbiter of future education or life choices, society treats test results as the major goal of schooling rather than as a useful but fallible indicator of achievement. Almost 100 years ago, a chief inspector of schools in England described this principle in a way that resonates today: Whenever the outward standard of reality (examination results) has established itself at the expense of the inward, the ease with which worth (or what passes for such) can be measured is ever tending to become itself the chief, if not sole, measure of worth. And in proportion as we tend to value the results of education for their measurableness, so we tend to undervalue and at last ignore those results which are too intrinsically valuable to be measured (Holmes, 1911). In the next five years, almost half of U.S. states will require students to pass a state-mandated test as a requirement for graduation (Edwards, 2003). As a result, a passing score on the state test is the coin of the realm for students, parents, teachers, and administrators. The social importance placed on state test scores ensures that students‟ successful performance on the state test is the ultimate goal for schools. Local press coverage on school pass rates and anecdotal evidence that scores on the state test may influence local real estate sales show the importance of test performance as a surrogate for education quality. Principle 7: A high-stakes test transfers control over the curriculum to the agency that sets or controls the examination. State standards-based reform efforts leave the details and development of testing programs to state
50 departments of education and whomever the department contracts with to construct the test. This system shifts the responsibility for determining curricular priorities and performance standards away from local school administrators or classroom teachers and often results in a one-size–fits-all curriculum and test. Falmouth, Massachusetts, provides a recent noteworthy example of how a high-stakes state test can override local control. Under the threat of losing state funding and the licensure of the school principal and superintendent, the Falmouth School Committee reversed a decision to award diplomas to special-needs students who failed the Massachusetts state examination, thus shattering the hopes of a student seeking admittance to a nonacademic culinary degree program (Myers, 2003). Accountability in Testing No one denies the importance of accountability. The relationship between test scores and accountability, however, is not as simple as some people think. The seven principles formulated in 1988 have been acted out in state after state in the past 15 years and clearly reveal the serious flaws in the practice of using a single high-stakes measure to hold all students and schools accountable. Cut-off scores that place students in such performance categories as needs improvement, basic, proficient, or advanced are arbitrary. The subjective methods used to categorize students into performance categories often lack validity (Horn, Ramos, Blumer & Madaus, 2000). Most
51 policymakers and the public do not understand the psychometric underpinnings of the tests. Issues that might seem trivial to them, such as the assumptions made when running computer programs that produce scaled scores, and even basic decisions about rounding, have significant consequences when categorizing students. Like any measurement tool that produces a number, test scores are fallible. Yet most state laws do not consider margin of error when interpreting students‟ scores. Misguided executive decisions, poorly conceived legislation, understaffing, unrealistic reporting deadlines, and unreasonable progress goals can cause numerous errors in test scores (Rhoades & Madaus, 2003). One single test can only sample knowledge and cannot give a full picture of what students know and can do. As an illustration, Harlow and Jones‟s (2003) interviews with students showed that on the science portion of the Third International Mathematics and Science Study (TIMSS), the students had more knowledge about concepts than their written answers had demonstrated for more than half of the test questions. Conversely, the interviews suggested that for one-third of the items, students lacked a sound understanding of the information assessed even though they had given the correct response. A fundamental principle in social science research is to always use at least two methods when studying social science phenomena because relying on only one method can produce misleading results. We need to enhance
52 state testing programs by including multiple measures of student achievement. Measuring in a variety of ways does not mean giving students multiple opportunities to take the same test, but rather incorporating other methods of measurement or additional criteria, such as teacher judgments, when making decisions about grade promotion and graduation (Harlow & Jones, 2003). As districts, schools, and teachers respond to federal and state-based accountability policies, we must step back from a blind reliance on test scores. We need to acknowledge that tests, although useful, are also fallible indicators of achievement. We also need to recognize that when test scores are linked to high-stakes consequences, they can weaken the learning experiences of students, transform teaching into test preparation, and taint the test itself so that it no longer measures what it was intended to measure (Harlow & Jones, 2003). Effects of High-Stakes Testing on Student Motivation and Learning Current generation of policymakers did not invent high-stakes testing. Tests of various sorts have determined which immigrants could enter the United States at the turn of the 20th century, who could serve in the armed forces, who was gifted, who needed special education, and who received scholarships to college. But the NCLB Act of 2001 aims to make high-stakes testing more pervasive than ever before, mandating annual testing of students in grades 3 – 8 in Reading and Mathematics.
53 Federal legislators who overwhelmingly passed this act into law apparently assumed that high-stakes testing would improve student motivation and raise student achievement. Because testing programs similar to those required by NCLB already exist in many states, we can put that assumption to the test. Eighteen states currently use examinations to grant or withhold diplomas: Alabama, Florida, Georgia, Indiana, Louisiana, Maryland, Minnesota, Mississippi, Nevada, New Jersey, New Mexico, New York, North Carolina, Ohio, South Carolina, Tennessee, Texas and Virginia. Most of these states also attach to their state assessments a broad range of other consequences for students, teachers, and schools. The experiences of these states can help predict how the new nationwide program of high-stakes testing will affect student achievement. Unfortunately, the evidence shows that such tests actually decrease student motivation and increase the proportion of students who leave school early. Further, student achievement in the 18 high-stakes testing states has not improved on a range of measures, such as the National Assessment of Educational Progress (NAEP), despite higher scores on the state‟s own assessments. (Amrein & Berliner, 2003). High-stakes testing assumes that rewards and consequences attached to rigorous tests will “motivate the unmotivated” to learn (Orfield & Kornhaber, 2001). The “unmotivated” are usually identified as low socioeconomic students in urban schools, often African Americans and Latinos.
54 Researchers have found that when rewards and sanctions are attached to performance on tests, students become less intrinsically motivated to learn and less likely to engage in critical thinking. In addition, they have found that high-stakes testing cause teachers to take greater control of the learning experiences of their students, denying their students opportunities to direct their own learning. When the stakes get high, teachers no longer encourage students to explore the concepts and subjects that interest them. Attaching stakes to tests apparently obstruct students‟ path to becoming lifelong, self-directed learners and alienates students from their own learning experiences in school (Sheldon & Biddle, 1998). Dropout rates are climbing throughout the United States and many researchers hold high-stakes testing at least partly to blame (Rothstein, 2002). Some researchers found that dropout rates were 4 to 6 percent higher in schools with high school graduation examinations. Another study reported that students in the bottom quintile in states with high-stakes testing were 25% more likely to drop out of high school than were their peers in states without high-stakes testing (Jacob, 2001). Researchers in yet another study found that failing these tests significantly increased the likelihood that even students with better academic records would drop out (FairTest & Massachusetts CARE, 2000). More and more teenagers are exiting formal schooling early to earn a General Educational Development (GED) credential (Murnane, Willett, & Tyler, 2000). Although young people who have earned such alternative
55 degrees do not technically count in dropout statistics, many of them undoubtedly left school because of their concerns about passing rigorous graduation tests. Students who repeat a grade are significantly more likely to drop out of school (Goldschmidt & Wang, 1999). In states where promotion to the next grade hinges on passing the state exams, high-stakes testing policies contribute to higher dropout rates in the long run. Even before they actually take the test, struggling students are more likely to be retained in grade if they attend schools in high-stakes testing environments. By holding lowachieving students back, schools ensure that these students have more of the knowledge necessary to perform well on high-stakes testing the next year and keep low-performing students‟ test scores out of the composite test performance in the grades in which high-stakes testing matter. In Texas, students from racial minorities and low socio-economic backgrounds are being retained in Grade 9 at very high rates before taking the Texas Assessment of Academic Skills (TAAS) in Grade 10. Many teachers retain students if they doubt their potential to pass TAAS the following year. McNeil (2000) estimated that half of all minority students enrolled in Texas high schools are technically enrolled as freshmen. Although some of them are 9th graders for the first time, thousands of others have been retained in the 9th grade once or even twice. Other researchers (Haney, 2000, 2001; Klein, Hamilton, McCaffey & Stecher, 2000; Yardley, 2000) have verified her numbers. In 1998, one in every four African American and Latino 9 th
56 graders in Texas was retained (Fisher, 2000). After these students are retained, thousands of them drop out of school. Common problems of high-stakes testing programs are quite likely to affect the breadth and depth of student learning. If schools narrow the curriculum they teach; make heavy use of drill activities tied to the state test; cheat by over-identifying language minority and special education students and then keeping these students from taking these tests; retain poorly performing students in grade; and encourage those who are at least likely to pass the state‟s test to drop out, then scores on state tests will almost certainly go up. But have students really learned any more than they did before high-stakes testing policies were instituted (Fisher, 2000)? Other Considerations of Assessment and Testing Although NCLB now requires all students to be accounted for in any state‟s assessment system, this has not always been the case (Anderson, 2004). In the past, groups of students such as English language learners or students in Special Education were systematically excluded from large scale assessments (State, 1999), or their scores were not reported (Thurlow, Neilson, Tellucksingh, & Ysseldyke, 2000). In the 2002-2003 school year, the Texas Assessment of Knowledge and Skills (TAKS) replaced the Texas Assessment of Academic Skills (TAAS) as the primary statewide assessment program. TAKS is designed by legislative mandate to be more comprehensive than its predecessors and encompasses more of the state-mandated curriculum, the Texas Essential
57 Knowledge and Skills (TEKS), at more grade levels than TAAS did. The high school level assessments, administered at Grades 9, 10 and 11, are grounded in the high school TEKS curriculum. By law, students for whom TAKS is the graduation testing requirement must pass exit level tests in four content areas – English Language Arts, Mathematics, Science, and Social Studies – in order to graduate from a Texas public high school (Technical Digest, 2003-2004). In Texas, there is evidence that the numbers of black and Hispanic students in Special Education rose between 1994-1998 while the state implemented its statewide testing program which excluded some Special Education students from the reported scores (Haney, 2000). It is interesting to note that while the numbers of African-American and Hispanic students are over-represented in Special Education, about eight to nine percent of English language learners are identified as receiving Special Education services in the US (D‟Emilio, 2003, June; Zehler, Fleischman, Hopstock, Pendzick, & Stepherson, 2003). Labeling schools can have an impact on teacher and student morale (Anderson, 2004). Certainly, poor test scores or poorly explained assessment systems can result in decreased student motivation (Lane & Stone, 2002). Teachers have also reported that the high-stakes nature of some assessments can have a negative impact on student morale (Flores & Clark, 2003). Although some teachers have reported that their English
58 language learners can reach the high standards set for them, they may need more time than other students (Hood, 2003). For English-language learners, the additional requirements of an exit examination could increase dropout rates (Anderson, 2004). Hispanic students, many of whom are English-language learners, have higher dropout rates than the population as a whole (Barro & Kolstad, 1987; Kaufman, Alt, & Chapman, 2001). In another study, teachers reported that increasing emphasis on test scores cause them to dislike their jobs (Hinde, 2003). In a study examining the discussion and journal entries from teachers, Flores and Clark (2003) found that teachers were not against accountability and viewed it as distinct from statewide testing, but also thought that an over-emphasis on testing resulted in unbalanced curriculum and inappropriate instructional decisions. In order for teachers to make specific changes to instruction, the assessments needs to be clear as to what skills are being assessed (Popham, 2003). Some teachers may react to low test scores of English language learners by teaching to the test while others may ignore the impact of the test scores all together (Alderson & Wall, 1993). Testing cannot be divorced from socio-cultural, economic, and psychological issues (Solano-Flores & Trumbull, 2003). ELLs, for instance, may not score any differently in an assessment even when allowed to use a dictionary (Albus, Thurlow, Liu & Bielinski, 2005). This is further complicated by the theory forwarded by Wang and Koda (2005) that ELLs as
59 a group may have diverse styles in developing English Language proficiency. Therefore modifications are adapted to teach ELLs the academic content of a lesson, and at the same time support the acquisition of a new language (Major, Fitzmaurice, Bunta, & Balasubramanian, 2005). A study of washback from a test in Hong Kong demonstrated that change in the assessment could change the ways in which teachers and students interacted (Cheng, 1999). School Administrators and teachers, as well as students, need to be motivated to change the way learning takes place and also be invested in demonstrating achievement on the assessments in order for washback to instruction to take place and be successful (Lane & Stone, 2002). Related Studies In a study on “Intended and Unintended Consequences of Statewide Testing for ESL Curriculum and Instruction”, Anderson (2004) examined what positive or negative impact assessment systems have on the curriculum and instruction of English language learners in one Midwestern school district. The researcher used focus groups and interviews to obtain views of educators on the consequences of statewide testing for ELLs. Positive consequences that were identified included more teacher collaboration, changes in curriculum and instruction, better alignment between ESL and content area curricula and more focus on reading and writing. Negative consequences included student and teacher frustration, more teaching to the test occurring, and a narrowed curriculum. Educators
60 in the study also identified problems with the accountability system and made recommendations for how it could be improved (Anderson, 2004). Another study on “Inclusion of Students with Limited English Proficiency in National Assessment of Educational Progress (NAEP): Classification and Measurement Issues” conducted by Abedi (2004) reported the major concerns over classification and measurements for students with limited English proficiency (LEP). Issues included the poor operational definition of English proficiency construct and validity concerns on the existing language proficiency tests. The study discussed issues concerning the classification of ELLs and elaborated on factors that impact decisions to include ELLs in NAEP assessments. With funding through a competitive bidding process authorized under the NCLB section on Enhanced Assessment Instruments, there are national efforts underway to develop English proficiency tests that can be used to provide valid measures of students‟ level of English proficiency (Abedi, 2004). Wall (2000) made a microethnographic case study entitled “A Case Study of Secondary School Efforts Toward English Language Learner Success in a Standards-Based Reform System.” This study was designed to describe and interpret the site-based decision-making process of a collaborative study group of high school educators as they focused on the appropriate participation of ELLs in a district wide, standards-based, reform initiative. The research question which guided the study was: From what perspectives and with what outcomes does a collaborative group of site-
61 based, high school educators deliberate the participation of ELLs in a standards-based reform system which mandates high stakes assessments? Three themes emerged from the study: (a) personal discovery, (b) informed action, and (c) instructional advocacy. These themes suggested phases of sociolinguistic accommodation through which educators progress in their reform-based deliberations regarding appropriate approaches to support ELLs in a high-stakes assessment system (Wall, 2000). This study on the impact of high-stakes testing on ELLs in major urban high schools in Texas showed quantitatively how the percentage of ELLs enrolled in a school affects the school‟s performance in the State‟s assessment. Qualitatively, it gathered the input and feedback of educators on the different concerns included in the study: (a) purpose of TAKS, (b) changes caused by TAKS, (c) consequences of TAKS, (d) recommendations to improve TAKS, and (e) needs of ELLs. Summary As stated in chapter I, the purpose of this study was to determine the impact of high-stakes testing on ELLs. This was shown in both the quantitative and qualitative dimensions of the study. Data obtained from TEA were used to determine whether there is a relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005 and 2006. To support the quantitative aspect, this study explored what
62 certified ESL teachers, non-certified ESL teachers who teach ELLs, administrators, and district ESL personnel viewed as the impact that high stakes standardized assessments have on ELLs, ESL curriculum and instruction, and what they observed as actually occurring. The mandates and key elements of the NCLB were geared towards improving the achievement of students in the different public schools of the United States. The measure of adherence was channeled through the AYP that the different schools and districts of the different states monitor and report. High-stakes testing became the measuring stick that gauged the achievement of students in the different core subject areas. Issues and concerns were centered on the ELLs regarding the different moves and accommodations given to this special subgroup of learners. Feedback regarding the issues and concerns of the different studies and researches included both positive and negative dimensions.
CHAPTER III METHODOLOGY Introduction Standardized testing and assessments have become necessary facets of American education. Consequently, accountability testing is currently implemented in practically every state in the U.S. Since the purpose of this increased level of accountability is to ensure that all students are receiving a quality, standards-based education, it is important to document the consequences of the system to ensure that the intended reforms are taking place. One of the goals of the accountability system should be to document any negative opposing impact that could occur so that interventions can be developed so that these consequences can be minimized. The purpose of this study was to determine the impact of highstakes testing on ELLs. This is shown in both the quantitative and qualitative dimensions of the study. Data obtained from TEA were used to determine whether there is a relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005 and 2006. To support the quantitative aspect, this study explored what certified ESL teachers, non-certified ESL teachers who teach ELLs, administrators, and district ESL personnel viewed as the impact that high stakes
64 standardized assessments have on ELLs, ESL curriculum and instruction, and what they observed as actually occurring. The study also addressed concerns regarding the validity of student evaluations, and the common inferences made about student performance in these assessments. There is a need to know how public schools that have diverse student attributes can be held accountable on the basis of one uniform and universal standard. Since the standardized assessments are given in English, schools with predominantly Hispanic populations may already be at a disadvantage through no fault of their own. What needs to be examined is whether standardized assessments facts are free from linguistic and cultural bias as viewed by teachers of ELLs. Research Questions Quantitative Is there a relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005 and 2006? Null Hypotheses HO1: There is no statistically significant relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in English Language Arts given in 2003, 2004, 2005 and 2006.
65 HO2: There is no statistically significant relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in Mathematics given in 2003, 2004, 2005 and 2006. Qualitative The major question answered by this study was: What are the anticipated and observed consequences of statewide testing specifically, TAKS, on ELLs, ESL curriculum and instruction as viewed by certified ESL teachers, non-certified ESL teachers who teach ELLs, school administrators, and district ESL personnel? This major question was explored using the following probes: 1. Why is TAKS given as a statewide test? 2. What are the intended consequences of this statewide testing? (Or what has happened because of TAKS?) 3. What problems have occurred related to or because of TAKS? 4. What changes were caused by this statewide testing? 5. What are your recommendations to improve this statewide testing? 6. What needs to be done for the ELLs to improve their performance in general and specifically for this statewide test?
66 Research Methods Both descriptive and comparative research techniques were employed in the explanatory design of the mixed methods study. Fraenkel and Wallen (2003) stated that Creswell describes the two types of mixed methods. 1. In a triangulation design, the researcher simultaneously collects both quantitative and qualitative data, compares results, and then uses those findings to see whether they validate each other (p. 443). 2. In an explanatory design, the researcher first collects and analyzes quantitative data, and then obtains qualitative data to follow up and refine the quantitative findings (p. 443). For this study, the explanatory design was used. Quantitative data for this research were gathered through TEA to determine if a relationship existed between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005 and 2006. Qualitative data were obtained through the on-line, open-ended questionnaire and individual and focus group interviews about the varied ways in which standardized assessments impacted ELLs. For the qualitative research component, the study used the crosssectional, open-ended questionnaire. A cross-sectional, open-ended
67 questionnaire collects information from a sample that has been drawn from a predetermined population. Furthermore, the information is collected at just one point in time, although the time it takes to collect the data may take anywhere from a day to a few weeks or more (Fraenkel & Wallen, 2003). The study also utilized descriptive research methods. Isaac and Michael (1995, p. 46) describes this type of research as: “to describe systematically a situation or area of interest factually or accurately.” A survey study also falls under the classification of descriptive research. Van Dalen (1979) lists the purpose of survey studies: 1. To collect detailed factual information that describes existing phenomena. 2. To identify problems or justify current conditions and practices. 3. To make comparisons and evaluations. 4. To determine what others are doing with similar problems or situations and benefit from their experience in making future plans and decisions. Research Design Since the study utilized the explanatory design of the mixed methods, the investigator first gathered quantitative data from Texas Education Agency (TEA) regarding the major urban high schools in Texas. TEA records personnel assisted in accessing and retrieving data from the TEA website. Qualitative data were obtained through the on-
68 line, open-ended questionnaire and individual and focus group interviews; views and opinions of the respondents were gathered and collated to validate and support the quantitative data. Quantitative Data From the Texas Education Agency, the following data regarding the urban high schools were gathered: the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005 and 2006. Qualitative Data Qualitative data were obtained using an on-line, open-ended questionnaire given to principals, assistant principals, ESL district personnel, ESL certified teachers and non-ESL certified teachers who were purposively sampled for the study and through the individual and focus group interviews using open-ended questions about the varied ways in which standardized assessments impact ELLs. Pilot Study Two Houston Independent School District schools, not included in the main study were selected for the pilot study. Quantitative data were obtained regarding the schools’ percentage of English language learners enrolled in a school and the percentage of all their 10th grade students passing TAKS in the two core areas of English Language Arts and
69 Mathematics. This was for the four school years starting with the first school year 2002 – 2003, when TAKS was administered. During the pilot study the on-line questionnaire underwent pretesting with three basic considerations: (1) administered the pre-test under conditions comparable to those anticipated in the final study; (2) analyzed the results to assess the effectiveness of the trial questionnaire to yield the information desired; and (3) made appropriate additions, deletions, and modifications to the questionnaire (Isaac & Michael, 1995). Qualitative data resulting from an on-line open-ended questionnaire on the six different concerns listed below were tabulated combining the results from the two schools. Results were categorized using the NVivo software package but the categories were modified based on the expert opinion of the respondents belonging to the focus groups. The frequencies for the responses by the different respondents (teachers, school administrators and district ESL personnel) pertaining to the different categories were tallied and percentages were computed. Listing of categories was based on the total frequencies; those categories identified most by the respondents were listed first followed by those with lower frequencies. The different concerns included the following: (1) Purpose of TAKS; (2) Consequences of TAKS; (3) Problems Related to TAKS; (4) Changes Caused by TAKS; (5) Recommendations to Improve TAKS; and (6) Needs of ELLs.
70 Results of the focus group and one-on-one interviews were validated against the results of the on-line questionnaire and provided explanation or support for the answers given. The categories for the different responses were affirmed or modified by the focus groups. Population and Samples Quantitative Data The TEA provided the data on the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005 and 2006. The researcher used purposive sampling in selecting schools for this study. Purposive sampling is based on the assumptions that the investigator wants to discover, understand, and gain insight and therefore must select a sample from which the most can be learned (Merriam, 1998). While not a random sampling of high schools, the sample is not intended to create results that can be generalized to all major urban high schools in the U.S. A purposive sampling was used in order to provide a representative sample of the major urban high schools in Texas in order to gain in-depth insight into what impact might be occurring. The impact that might emerge from this study might occur in other high schools, but it is important to take into account the characteristics of the high schools as well as the assessment system in the state in order to
71 extrapolate from the findings and make comparisons with other situations (Patton, 1990). Qualitative Data The on-line, open-ended questionnaire was given to the principals, assistant principals, certified ESL teachers and non-ESL certified teachers handling ELLs of the selected schools and to the district personnel: Total 1) ESL Teachers 2) Non-Certified ESL Teachers 3) Principals 4) Assistant Principals 5) District ESL Personnel Total 30 30 10 20 8 98
The different focus groups consisted of ESL certified and non-ESL certified teachers handling ELLs. One-on-one interviews involved the selected principals and the selected district ESL personnel. The same schools and district personnel who answered the on-line questionnaire were included in the focus groups and one-on-one interviews. Selection of the participants in the focus group interviews utilized the snowballing technique. Participants will identify others whose input or experience will also be valuable to the study (Krathwohl, 1993).
72 Since the researcher has the obligation to respect and protect the rights and wishes of the research participants, the following actions were done: (1) the researcher protected anonymity of the participants by using computer-given codes for the responses; and (2) the researcher informed the participants about the purpose of the survey. The security of the raw data gathered through the records sections of TEA and the selected schools, responses to the on-line questionnaire and the transcripts of the interviews was assured in order to protect the anonymity of the participants and to uphold the trustworthiness of the study. The above concerns regarding trustworthiness and confidentiality of data or information were shared with the participants when the researcher contacted them through e-mail, telephone, mail, or in person. Instrumentation Quantitative data were accessed and retrieved from the TEA website regarding the major urban high schools in Texas. Data were organized for computations utilizing the SPSS software package, Version 14.0. The on-line, open-ended questionnaire provided one of the bases for the qualitative data. The triangulation method included categorizing the responses to the online, open-ended questionnaire into emergent themes, interviewing the focus groups of teachers and assistant
73 principals and one-on-one interviews with the principals and district ESL personnel. According to Fraenkel and Wallen (2003) personal interview is probably one of the most effective ways there is to enlist the cooperation of respondents in a survey; rapport can be established, questions can be clarified, unclear or incomplete answers can be followed up and so on. Patton (1990) expounds that the purpose of interviews is to gain access to those areas of the participants’ experiences or thought which cannot be observed. Consequently, interviews will play a significant role in data collection, a role which generally cannot be duplicated by other means (Dexter, 1970). Fraenkel and Wallen (2003) cite the following advantages of openended questions in survey research (a) allows more freedom of response; (b) easier to construct; and (c) permits follow-up by interviewer. But there are also disadvantages: (a) responses tend to be inconsistent in length and content across respondents; (b) both questions and responses may be subject to misinterpretation; and (c) responses are harder to tabulate and synthesize. However, these disadvantages can be minimized through the use of the NVivo software package, expert help from the focus groups in classifying categories, follow-up interviews with the focus groups and one-on-one interviews.
74 Validity and Reliability For validity and reliability, the following expert opinions were considered. “Validity, I mean truth: interpreted on the extent to which an account accurately represents the social phenomena to which it refers” (Hammersley, 1990, p. 57). “Reliability refers to the degree of consistency with which instances are assigned to the same category by different observers or by the same observer on different occasions” (Hammersley, 1990, p. 67). The triangulation method involving the analysis of the qualitative data, collation of data from the on-line questionnaire and interviews assured the validity and reliability of the survey questions and the explanatory design of the mixed methods study. For the quantitative dimension of the study, validity and reliability measurements were derived from the TAKS report prepared by TEA. Validity is a process of collecting evidence to support inferences made from scoring results of an assessment. In the case of TAKS, test results are used to make inferences about the students’ knowledge and understanding of the Texas Essential Knowledge and Skills (TEKS). Test reliability indicates the consistency of measurement. TAKS test reliabilities are based on internal consistency measures, in particular on the Kuder-Richardson Formula 20 (KR-20) for tests involving dichotomously scored (multiple choice) items and on the stratified coefficient alpha for tests involving a mixture of dichotomous and polytomous (essay-prompt and short answer) items.
75 In order to build trustworthiness in the qualitative aspect of the study, four different criteria were considered to meet this need: (1)credibility, which aims to produce findings that are believable and convincing; (2) transferability, which attempts to apply findings in one setting to other contextually similar settings; (3) dependability, which addresses the question concerning which findings are consistent with those of other similar investigations; and (4) confirmability, which ensures that both the process and the product are auditable (Isaac & Michael, 1995). Research Procedures Quantitative After appropriate permissions for data gathering were obtained, records personnel of TEA were contacted and arrangements made as to process and assistance regarding acquisition of data for the study. The dry-run or pilot study with the two HISD schools facilitated the above process. Qualitative The questions in a survey, and the way they are asked, are of crucial importance (Fraenkel & Wallen, 2003). The authors refer to Floyd Fowler who points out that there are four practical standards that all survey questions should meet: 1. Is this a question that can be asked exactly the way it is written?
76 2. Is this a question that will mean the same thing to everyone? 3. Is this a question that people can answer? 4. Is this a question that people will be willing to answer, given the data collection procedures? (Fowler, 1984). After the questionnaire was refined based on the suggestions of the focus groups during the pilot study, the questionnaire was placed on-line to respondents of the study. Prior to this, the researcher contacted the respondents in person, by phone, by email or mail. Furthermore, the researcher arranged dates with the different schools and districts for the focus group and one-on-one interviews. Data Collection and Recording Quantitative The data for the major urban high schools regarding the percentage of ELLs enrolled in a school and the percentage of all Grade 10 students passing the TAKS tests in English Language Arts and Mathematics during the four years when TAKS was administered, starting school year 2002 – 2003 were obtained from the TEA website. Qualitative An on-line, open-ended questionnaire was answered by the principals, assistant principals, ESL teachers, and non-certified teachers handling ELLs of the selected major urban high schools in Texas. District ESL personnel were also requested to answer the same questionnaire. The focus groups offered expert opinions regarding the categories to use
77 in classifying the responses to the questionnaire. Further clarification was requested from the principals and the district ESL personnel during the one-on-one interviews. Results of the questionnaire were placed in categories suggested by the focus groups after initial classification was done through the NVivo software system. Transcripts of the interviews and focus groups were entered into the NVivo software system (version 7.0) and coded according to themes that emerged in the data. NVivo provides a sophisticated way of electronically organizing interview transcripts for analysis and classification into themes and allowed the researcher to work with a large amount of transcript data. The themes that emerged from the data were compiled and compared between high schools. While NVivo was a valuable sorting tool that allowed the researcher to code, sort, and recall data in different ways, the researcher developed and created codes for the responses gathered. The analysis was done by the researcher using NVivo’s capabilities to sort out the complexities of the rich data from the interviews and focus groups. A program such as NVivo can help the researcher ensure that the qualitative data were well-organized (Weitzman, 2000). One of the strengths of collecting qualitative data is the richness of the information that can be collected and which can capture a theme in a more complete way than the researcher may be able to summarize. This evidence directly from the data was used to show a clear connection
78 between the data and the identified themes (Marshall, 1990). The rich description of the themes from the participants’ own words also aids in verifying that the themes identified are those that the participants actually voiced (Creswell, 1998; Krueger & Casey, 2000). The researcher triangulated quantitative data analysis, qualitative data analysis, and interviews in order to strengthen the credibility of the survey study. By combining multiple observers, theories, methods, and data sources, researchers can hope to overcome the intrinsic bias that comes from single-method, single-observer and single-theory studies (Denzin, 1970). With the mix of analyses, the author has better tools to discuss the impact of statewide testing on ELLs, ESL curriculum and instruction. Data Analysis Quantitative Descriptive statistics and analyses were performed to test each variable. After the data were examined and properly inputted, the next step was to compute for Pearson r correlation coefficients using the SPSS statistical package and test for statistical relationship at p < 0.05. For other analyses, the predictor variable is the percentage of ELLs enrolled in a school and the outcome variables were the percentages of all students passing the Grade 10 TAKS tests in English Language Arts and Mathematics. For each of the years under study, two separate Pearson r correlations were computed; the percentage of ELLs enrolled in a school
79 was compared with the English Language Arts results to determine if they have significant relationship and the other comparison was with the Mathematics results. The SPSS computations showed three different results in tabular form: (1) the means and the standard deviations of the percentage of ELLs enrolled in a school and the percentages of all students passing the TAKS tests in English Language Arts and Mathematics during the four years under study; (2) Pearson r correlation coefficients to determine if there was significant relationship between the percentage of ELLs enrolled in a school and the percentages of all students passing the English Language Arts and Mathematics TAKS tests given in 2003, 2004, 2005 and 2006; and, (3) regression analysis using the percentage of ELLs enrolled in a school as the predictor variable to predict the percentage of students passing in the 10th grade TAKS tests in the core areas of English Language Arts and Mathematics. Qualitative The information for the qualitative portion of the study included the emergent themes shown as categories in the frequency distribution table. The frequency distribution is a table in which all score units are listed in one column and the number of individuals receiving each score appears as frequencies in the second column (Isaac and Michael, 1995). Frequencies were tallied and percentages were computed. Categories with higher percentages were listed first followed by those with lower percentages. An overview preceded each table giving the emergent
80 themes mostly cited by the respondents. Anecdotal records followed the tables - these are the views and opinions of the respondents regarding the different concerns included in the study. Summary In this study the researcher considered the aspects of procedural consistency, neutrality of findings, and truth value to assure the study of trustworthiness. “Valid inquiry in any sphere… must demonstrate its truth value, provide the basis for applying it, and allow for external judgments to be made about the consistency of its procedures and the neutrality of its findings or decisions” (Erlandson, 1993). Quantitative data that were sourced as aggregate data from the TEA website included the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005 and 2006. Qualitative data were collated from the responses of selected respondents to the on-line questionnaire regarding the anticipated and observed consequences of the statewide testing, specifically TAKS, on ESL curriculum and instruction as viewed by ESL teachers, school administrators and district ESL personnel. Interviews were conducted with the focus groups and one-on-one interviews involved the principals and district ESL personnel.
81 Presentation of data included: (a) the quantitative data analysis on the correlation between the percentage of ELLs enrolled in a school and the percentage of all students passing the 10th grade TAKS in English Language Arts and Mathematics and the regression analysis using the percentage of ELLs enrolled in a school as predictor variable and the percentage of all students passing the 10th grade TAKS in ELA and Mathematics as outcome variables; (b) qualitative data analysis classifying responses to the on-line, open-ended questionnaire as different emergent themes; and (c) anecdotal records from the interviews with the different focus groups, principals, and district ESL personnel. The relationship between the percentage of ELLs enrolled in a school and the percentage of all students passing the 10th grade TAKS in each of the core areas of English Language Arts and Mathematics was determined using the SPSS program for Pearson r correlation. The regression analysis resulted to linear regression equations predicted the percentage of all students passing in 10th grade TAKS in English Language Arts and Mathematics using the percentage of ELLs enrolled in a school as predictor variable. Emergent themes were categorized through the NVivo software package and suggestions of the focus groups. The anecdotal records expressed the views and opinions of the respondents regarding the following and focused on the ELLs: (a) the purpose of the statewide, high stakes TAKS; (b) intended consequences of TAKS; (c) problems related to
82 TAKS; (d) changes caused by TAKS; (e) recommendations to improve performance in TAKS; and, (f) the needs of ELLs.
CHAPTER IV ANALYSIS OF DATA The purpose of this study was to determine the impact of highstakes testing on English Language Learners (ELLs). This was shown in both the quantitative and qualitative dimensions of the study. Both quantitative and qualitative dimensions of the study provided the status of high-stakes testing as it affected ELLs and how it influenced efforts in schools to improve performance of students, particularly ELLs. Data obtained from Texas Education Agency (TEA) were used to determine whether there was a relationship between the percentage of ELLs enrolled in a school and the percentage of all students passing the 10th grade Texas Assessment of Knowledge and Skills (TAKS) tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005, and 2006. To support the qualitative aspect, this study explored what certified English as a Second Language( ESL) teachers, noncertified ESL teachers who teach ELLs, administrators, and district ESL personnel viewed as the impact that high stakes standardized assessments had on ELLs, ESL curriculum and instruction, and what they observed as actually occurring. With the No Child Left Behind (NCLB) Act of 2001, student achievement has been placed in the forefront of quality education. Aligned to this effort is statewide testing aimed at assessing students‟ performance and status of the school. In this chapter, the relationships
84 between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005, and 2006 are shown. The results of the Statistical Package for the Social Sciences (SPSS) computations show the: (1) means and the standard deviations of the percentage of ELLs enrolled in a school and the percentages of all students passing the 10th grade TAKS tests in English Language Arts and Mathematics during the four years under study; (2) Pearson r correlation coefficients between the percentage of ELLs enrolled in a school and the percentages of all 10th grade students passing the English Language Arts and Mathematics TAKS tests given in 2003, 2004, 2005, and 2006; and, (3) regression analysis using the percentage of ELLs as the predictor variable. Following this quantitative dimension of the study is the qualitative information regarding TAKS and its effects on the ELLs - its intended purpose and consequences, problems encountered related to this statewide testing, changes that occurred due to TAKS and recommendations to improve the performance of ELLs in general and specifically on this high-stakes test. The tabulated results of the openended, on-line questionnaire and the supporting explanation gathered through the individual and focus group interviews comprised the qualitative portion of this study. ESL district personnel, principals,
85 assistant principals, ESL-certified teachers and non-ESL certified teachers handling ELLs provided the needed information. Schools with ELLs are asked to account for their performance in line with NCLB. School and district administrators may get feedback both from the numerical results of high-stakes testing and from inputs of teachers and other personnel responsible for handling ELLs. Since the presence of ELLs in schools is a reality administrators have to face, it may be beneficial to be aware of the views and opinions of concerned personnel in the school system regarding interventions that may improve the curriculum and/or instruction of ELLs. Findings For the quantitative portion of the study, data for the 173 urban high schools were obtained from Texas Education Agency; however, for the year 2005, data available were only for 155 high schools. The data gathered were collated and the SPSS software package was utilized for the needed computations. The following research question was the focus of the quantitative dimension of the study: Research Question - Quantitative Is there a relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005, and 2006?
86 Two null hypotheses were formulated to answer the above question: HO1: There is no statistically significant relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in English Language Arts given in 2003, 2004, 2005, and 2006. HO2: There is no statistically significant relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in Mathematics given in 2003, 2004, 2005, and 2006. The results are presented in the following order: (a) the descriptive statistics (mean, standard deviation, number of cases); (b) the Pearson r correlation coefficients; and, (c) regression analysis showing the linear equations which can be used to predict the outcomes in the 10 th grade TAKS tests in English Language Arts and Mathematics. Tables 4.1.1 to 4.1.4 show that the average percentage of ELLs in the urban high schools of Texas during the four years under study starting from 2003, ranged from 6.94% to 8.30 %; the average percentage of ELLs enrolled in a school was 7.79%.
87 Table 4.1.1 Means and Standard Deviations of ELLs Enrolled in School and All Students Passing the 2003 10th Grade TAKS for English Language Arts and Mathematics. Mean 08.31 Std. Deviation 08.98 N 165
ELLs Enrolled In School All Students Passing ELA All Students Passing Mathematics Table 4.1.2
Means and Standard Deviations of ELLs Enrolled in School and All Students Passing the 2004 10th Grade TAKS for English Language Arts and Mathematics. Mean 08.30 Std. Deviation 09.64 N 147
ELLs Enrolled In School All Students Passing ELA All Students Passing Mathematics
88 Table 4.1.3 Means and Standard Deviations of ELLs Enrolled in School and All Students Passing the 2005 10th Grade TAKS for English Language Arts and Mathematics. Mean 7.62 Std. Deviation 10.80 N 147
ELLs Enrolled In School All Students Passing ELA All Students Passing Mathematics Table 4.1.4
Means and Standard Deviations of ELLs Enrolled in School and All Students Passing the 2006 10th Grade TAKS for English Language Arts and Mathematics. Mean 6.94 Std. Deviation 9.94 N 167
ELLs Enrolled In School All Students Passing ELA All Students Passing Mathematics
89 Tables 4.1.1 through 4.1.4 reflect the descriptive statistics showing the percent of all students passing in English Language Arts were: 62.87% in 2003, 68.28% in 2004, 59.39% in 2005 and 78.05% in 2006. The descriptive statistics showing the results of all students passing in Mathematics were: 61.85% in 2003, 53.57% in 2004, 47.68% in 2005 and 50.13% in 2006. Based on the above tables, Tables 126.96.36.199 and 188.8.131.52 compare the results of 10th grade TAKS in English Language Arts and Mathematics. Table 184.108.40.206 Comparison of Results in 10th Grade English Language Arts TAKS Percent ELLs Enrolled in School 8.30 8.29 7.62 6.94 Means-All Students Passing Current Year 62.87 68.28 59.39 78.05 Means-All Students Passing Previous Year -62.87 68.28 59.39
Year 2003 2004 2005 2006
Difference (+/-) -+5.41 -8.89 +18.66
Percent Increase (Decrease) -08.61 -13.02 031.42
90 Table 220.127.116.11 Comparison of Results in 10th Grade Mathematics TAKS Percent ELLs Enrolled in School 8.30 8.29 7.62 6.94 Means-All Students Passing Current Year 61.85 53.57 47.68 50.13 Means- All Students Passing Previous Year -61.85 53.57 47.68
Year 2003 2004 2005 2006
Difference (+/-) --8.28 -5.89 +2.45
Percent Increase (Decrease) --13.39 -10.99 05.14
Tables 4.2.1 to 4.2.4 show the obtained Pearson r correlation coefficients for the correlations between percent of ELLs enrolled in a school and the percent of all students passing in 10th grade TAKS test in English Language Arts and Mathematics. The obtained Pearson r correlation coefficients between the percentage of ELLs enrolled in a school and the percentage of all students passing in 10 th grade TAKS were: -0.349 in 2003, -0.392 in 2004, -0.297 in 2005 and -0.398 in 2006 for English Language Arts; -0.293 in 2003, -0.351 in 2004, -0.382 in 2005 and -0.356 in 2006 for Mathematics. All the computed Pearson r correlation coefficients were significant at the 0.05 level, two-tailed. We therefore reject the null hypotheses. All the obtained Pearson r correlation coefficients were negative for both content areas; this inverse relationship indicated that as the percentage of ELLs increased, the
91 percentage of all 10th grade students passing in each of the core areas of English Language Arts and Mathematics decreased. Table 4.2.1 Pearson Correlations: 2003 10th Grade TAKS for English Language Arts and Mathematics. Percent of All Students Passing ELA -.349(**) .00000 Percent of All Students Passing Math -.293(**) .00000
Pearson Correlation Percent of ELLs Enrolled Sig. (2-tailed)
** Correlation is significant at the 0.05 level (2-tailed). Table 4.2.2 Pearson Correlations: 2004 10th Grade TAKS for English Language Arts and Mathematics. Percent of All Students Passing ELA -.392(**) .00000 Percent of All Students Passing Math -.351(**) .00000
Pearson Correlation Percent of ELLs Enrolled Sig. (2-tailed)
** Correlation is significant at the 0.05 level (2-tailed).
92 Table 4.2.3 Pearson Correlations: 2005 10th Grade TAKS for English Language Arts and Mathematics. Percent of All Students Passing ELA -.297(**) .00000 Percent of All Students Passing Math -.382(**) .00000
Pearson Correlation Percent of ELLs Enrolled Sig. (2-tailed)
** Correlation is significant at the 0.05 level (2-tailed). Table 4.2.4 Pearson Correlations: 2006 10th Grade TAKS for English Language Arts and Mathematics. Percent of All Students Passing ELA -.398(**) .00000 Percent of All Students Passing Math -.356(**) .00000
Pearson Correlation Percent of ELLs Enrolled Sig. (2-tailed)
** Correlation is significant at the 0.05 level (2-tailed). A regression analysis for the data in 2003 is shown in Tables 4.2.5 and 4.2.6. The correlation coefficient between the percentage of ELLs enrolled in a school and the percentage of all students passing in 10th grade TAKS in English Language Arts for 2003 is -0.349. The Pearson r between the percentage of ELLs enrolled in a school and the percentage of all students passing in 10th grade TAKS in Mathematics is -0.293. Both correlation coefficients are significant at p < 0.05, two-tailed.
93 The percentage of all students passing in 10th grade English Language Arts TAKS can be predicted using the linear regression equation: Ŷ = 68.71 – 0.70X, where X is the percentage of ELLs enrolled in a school. For example, using the percentage of ELLs data for 2003 (X = 8.30), the predicted value for percent of all students passing 10 th grade English Language Arts is equal to 68.71 – 0.70(8.30) or 62.90. Actual result was 62.87. The percentage of all students passing in 10 th grade Mathematics TAKS can be predicted using the linear regression equation: Ŷ = 67.49 – 0.68X, where X is the percentage of ELLs enrolled in a school. A similar example for Mathematics gives a predicted value of 67.49 – 0.68(8.30) or 61.85. Actual result was 61.85. Table 4.2.5 Coefficients for Percentage of All Students Passing the 2003 10 th Grade English Language Arts TAKS. Model Unstandardized Coefficients B Std. Error 68.71 1.89 0 -.70 0.16 Standardized Coefficients Beta T Sig.
(Constant) Percent ELLs
36.37 -.349 0-4.55
a. Predictor: Percentage of ELLs Enrolled in a School b. Outcome variable: Percentage of All Students Passing in 10 th Grade English Language Arts TAKS
94 Table 4.2.6 Coefficients for Percentage of All Students Passing the 2003 10 th Grade Mathematics TAKS. Model Unstandardized Coefficients B Std. Error 67.49 2.22 0 -.68 0.18 Standardized Coefficients Beta T Sig.
(Constant) Percent ELLs
30.43 -.293 -3.74
a. Predictor: Percentage of ELLs Enrolled in a School b. Outcome variable: Percentage of All Students Passing in 10 th Grade Mathematics TAKS Results of the regression analysis for the data in 2004 are shown in Tables 4.2.7 and 4.2.8. The correlation coefficient between the percentage of ELLs enrolled in a school and the percentage of all students passing in 10th grade TAKS in English Language Arts for 2004 is -0.392. The Pearson r between the percentage of ELLs enrolled in a school and the percentage of all students passing in 10 th grade TAKS in Mathematics is -0.351. Both correlation coefficients are significant at p < 0.05, two-tailed. The percentage of all students passing in 10 th grade English Language Arts TAKS can be predicted using the linear regression equation: Ŷ = 73.76 – 0.66X, where X is the percentage of ELLs enrolled in a school. For example, using the percentage of ELLs data for 2004 (X = 8.29), the predicted value for percent of all students passing 10 th
95 grade English Language Arts is equal to 73.76 – 0.66(8.29) or 68.29. Actual result was 68.28. The percentage of all students passing in 10 th grade Mathematics TAKS can be predicted using the linear regression equation: Ŷ = 60.34 – 0.82X, where X is the percentage of ELLs enrolled in a school. A similar example for Mathematics gives a predicted value of 60.34 – 0.82(8.29) or 53.54. Actual result was 53.57. Table 4.2.7 Coefficients for Percentage of All Students Passing the 2004 10 th Grade English Language Arts TAKS. Model Unstandardized Standardized Coefficients Coefficients B Std. Beta Error 73.76 1.74 0 -.66 0.14 -.392 T Sig.
(Constant) Percent ELLs
a. Predictor: Percentage of ELLs Enrolled in a School b. Outcome variable: Percentage of All Students Passing in 10 th Grade English Language Arts
96 Table 4.2.8 Coefficients for Percentage of All Students Passing the 2004 10th Grade Mathematics TAKS. Model Unstandardized Coefficients B Std. Error 60.34 2.42 Standardized Coefficients Beta T Sig.
a. Predictor: Percentage of ELLs Enrolled in a School b. Outcome variable: Percentage of All Students Passing in 10 th Grade Mathematics TAKS A regression analysis for the data in 2005 is shown in Tables 4.2.9 and 4.2.10. The correlation coefficient between the percentage of ELLs enrolled in a school and the percentage of all students passing in 10 th grade TAKS in English Language Arts for 2005 is -0.297. The Pearson r between the percentage of ELLs enrolled in a school and the percentage of all students passing in 10th grade TAKS in Mathematics is -0.382. Both correlation coefficients are significant at p < 0.05, two-tailed. The percentage of all students passing in 10th grade English Language Arts TAKS can be predicted using the linear regression equation: Ŷ = 63.61 – 0.55X, where X is the percentage of ELLs enrolled in a school. For example, using the percentage of ELLs data for 2005 (X = 7.62), the predicted value for percent of all students passing
97 10th grade English Language Arts is equal to 63.61 – 0.55(7.62) or 59.42. Actual result was 59.39. The percentage of all students passing in 10 th grade Mathematics TAKS can be predicted using the linear regression equation: Ŷ = 53.97 – 0.83X, where X is the percentage of ELLs enrolled in a school. A similar example for Mathematics gives a predicted value of 53.97 – 0.83(7.62) or 47.65. Actual result was 47.68. Table 4.2.9 Coefficients for Percentage of All Students Passing the 2005 10 th Grade English Language Arts TAKS. Model Unstandardized Coefficients B Std. Error 63.61 2.03 0-.55 0.15 Standardized Coefficients Beta t Sig.
(Constant) Percent ELLs
31.28 -.297 0-3.59
a. Predictor: Percentage of ELLs Enrolled in a School b. Outcome variable: Percentage of All Students Passing in 10 th Grade English Language Arts TAKS
98 Table 4.2.10 Coefficients for Percentage of All Students Passing the 2005 10 th Grade Mathematics TAKS. Model Unstandardized Standardized Coefficients Coefficients B Std. Beta Error 53.97 2.29 0-.82 0.17 -.382 t Sig.
(Constant) Percent ELLs
a. Predictor: Percentage of ELLs Enrolled in a School b. Outcome variable: Percentage of All Students Passing in 10 th Grade Mathematics TAKS To show that the percentage of ELLs enrolled in a school affected the percentage of all students passing the 10th grade TAKS tests in English Language Arts and Mathematics, the regression analysis results are shown in Table 4.2.11 and Table 4.2.12. The correlation coefficient between the percentage of ELLs enrolled in a school and the percentage of all students passing in 10th grade TAKS in ELA for 2006 is -0.398. The Pearson r between the percentage of ELLs enrolled in a school and the percentage of all students passing in 10th grade TAKS in Mathematics is -0.356. Both correlation coefficients are significant at p < 0.05, twotailed. The percentage of all students passing in 10th grade English Language Arts TAKS can be predicted using the linear regression equation: Ŷ = 81.85 – 0.55X, where X is the percentage of ELLs enrolled
99 in a school. For example, using the percentage of ELLs data for 2006 (X = 6.94), the predicted value for percent of all students passing 10 th grade ELA is equal to 81.85 – 0.55(6.94) or 78.03. Actual result was 78.05. The percentage of all students passing in 10 th grade Mathematics TAKS can be predicted using the linear regression equation: Ŷ = 55.70 – 0.80X, where X is the percentage of ELLs enrolled in a school. A similar example for Mathematics gives a predicted value of 55.70 – 0.80(6.94) or 50.15. Actual result was 50.13. Table 4.2.11 Coefficients for Percentage of All Students Passing the 2006 10 th Grade English Language Arts TAKS. Model Unstandardized Coefficients B Std. Error 81.85 1.24 0-.55 0.10 Standardized Coefficients Beta t Sig.
(Constant) Percent ELLs
66.28 -.398 -5.37
a. Predictor: Percentage of ELLs Enrolled in a School b. Outcome variable: Percentage of All Students Passing in 10 th Grade English Language Arts TAKS
100 Table 4.2.12 Coefficients for Percentage of All Students Passing the 2006 10 th Grade Mathematics TAKS. Model Unstandardized Coefficients B Std. Error 55.70 2.08 0-.80 0.17 Standardized Coefficients Beta t Sig.
(Constant) Percent ELLs
26.78 -.356 -4.67
a. Predictor: Percentage of ELLs Enrolled in a School b. Outcome variable: Percentage of All Students Passing in 10 th Grade Mathematics TAKS Research Question – Qualitative After the refinements were done during the pilot study, an openended, on-line questionnaire was given to the principals, assistant principals, ESL district personnel, ESL certified teachers and non-ESL certified teachers who were purposively sampled from urban high schools in Texas. The major question answered by this study was: What are the anticipated and observed consequences of the statewide testing specifically, TAKS, on ESL curriculum and instruction as viewed by certified ESL teachers, non-certified ESL teachers who teach ELLs, school administrators, and district ESL personnel?
101 This major question was explored using the following probes: 1. Why is TAKS given as a statewide test? 2. What are the intended consequences of this statewide testing? (Or what has happened because of TAKS?) 3. What problems have occurred related to or because of TAKS? 4. What changes were caused by this statewide testing? 5. What are your recommendations to improve this statewide testing? 6. What needs to be done for the ESL students to improve their performance in general and specifically for this statewide test? Before the above questions for the qualitative study are answered, the following tables give the demographic information regarding the respondents. Demographic Characteristics of the Respondents Demographic information regarding the respondents included gender, age, current position, highest degree earned, years of experience in public education and certification(s) achieved. The open-ended, on-line questionnaire for the qualitative aspect of the study was intended for 98 respondents. Six principals responded, together with 9 assistant principals, 6 ESL district personnel, 15 ESL certified teachers and 19 non-ESL certified teachers – a total of 55 respondents, for a response rate of 56%. Table 4.3 shows that 69% of the respondents were female and 31% were male.
102 Table 4.3 Distribution of Respondents by Gender. Gender Female Male N=55 The age of the respondents is shown in Table 4.4. With respect to age, 31% of the respondents were between 41 – 50 years old and 51 – 60 years old. The group of 31 – 40 years old comprised 22% of the respondents; 9% of the respondents were older than 60 years old and the youngest between 21 – 30 years old was 7% of the group. Table 4.4 Distribution of Respondents by Age. Age 21-30 31-40 41-50 51-60 > 60 N=55 It is shown in Table 4.5 that the non-ESL certified teachers comprised 35% of the respondents and 27% were ESL certified teachers. The administrators accounted for the remaining 38%: 16% were Frequency 04 12 17 17 05 Percent (%) 07 22 31 31 09 Frequency 38 17 Percent (%) 69 31
103 assistant principals, 11% were principals, and 11% were ESL district personnel. Table 4.5 Distribution of Respondents by Professional Position. Current Position Principal Assistant Principal ESL District Personnel ESL Certified Teacher Non-ESL Certified Teacher N=55 Table 4.6 displays that 53% of the respondents have Master‟s degrees, 40% have Bachelor‟s degrees and 7% have Doctorate degrees. Table 4.6 Distribution of Respondents by Highest Degree Earned. Highest Degree Bachelor‟s Degree Master‟s Degree Doctorate Degree N=55 Considering years of experience in public education, it is shown in Table 4.7 that 40% of those who participated in the study had more than 20 years of public school experience. Thirty-one percent had 11 – 20 Frequency 22 29 04 Percent (%) 40 53 07 Frequency 06 09 06 15 19 Percent (%) 11 16 11 27 35
104 years of public school experience, 16% had 5 years or less, and 13% had been in the public schools for 6 – 10 years. Table 4.7 Distribution of Respondents by Years of Experience in Education. Years of Experience 1–5 6-10 11-20 > 20 N=55 Table 4.8 displays the certifications of the ESL district personnel, principals, assistant principals, ESL certified and non-ESL certified teachers who handle ELLs that participated in this study. Some of these administrators and teachers had more than one certification. Based on total number of respondents, 47% had certification in English Language Arts, 20% in Mid-Management, 18% in Mathematics, 16% in Frequency 09 07 17 22 Percent (%) 16 13 31 40
Principalship, and 13% in Special Education.
105 Table 4.8 Distribution of Respondents by Certification. Certification English Language Arts Mid-Management Mathematics Principal Special Education Bilingual Educational Diagnostician Biology Sociology Supervisor Reading Librarian Superintendent History Technology Marketing Counseling Vocational Home Economics Social Studies Composite N=55 Frequency 26 11 10 09 07 02 02 02 02 02 02 02 02 02 01 1 1 01 01
106 The findings regarding the qualitative portion of the study were presented as follows: (1) the nine open-ended responses to the on-line questionnaire; (2) frequency tables showing the emergent themes, frequencies and percentages; and, (3) anecdotal views and opinions of district ESL personnel, principals, assistant principals, ESL-certified teachers and non-certified ESL teachers handling ELLs regarding the issues or concerns. Answers of some respondents belonged to more than one emergent theme; the total number of answers may have exceeded the total number of respondents. The percentage shown after the total responses given for each emergent theme was computed based on total number of respondents. Why is Texas Assessment of Knowledge and Skills (TAKS) given as a Statewide Test? Why is TAKS given as a statewide test? Responses are shown in Table 4.9. Of the 55 respondents, 40% viewed TAKS as a tool to gauge knowledge in the core academic areas. Twenty-nine percent (29%) of the respondents considered TAKS as a means to determine the school‟s effectiveness and performance (Exemplary, Recognized, Academically Acceptable, or Academically Unacceptable), while 16% regarded TAKS as an instrument to appraise the effectiveness of the state curriculum. Only 7% indicated that this statewide test is mandated by law and is aligned with NCLB.
Table 4.9 Why is TAKS Given as a Statewide Test to ELLs? Emergent Themes Gauge Skills/TEKS School Accountability Assess State Curri-Std Mandated by NCLB Com Schools/Districts Generate Revenue Test College Prep Political Appeasement No Response Dist.(6) 3 0 1 1 1 1 0 0 0 Prin (6) A.Prin (9) 3 3 2 1 0 0 0 0 0 0 6 0 1 1 1 1 0 0 ESLTeach (15) 6 3 3 0 1 0 1 1 0 NonTeach (19) 7 5 4 2 0 1 0 0 1 Tot/%(55) 22/40 0 16/29 00 9/16 04/7 0003/5 0003/5 00 2/4 00 1/2 00 1/2
108 One ESL district personnel (EDP-03) gave this opinion about TAKS: “It is administered to measure the knowledge and skills of Texas students,” and “it provides a comparative data table for critics.” An assistant principal (AP-06) supported this view: “TAKS is to help gauge students‟ basic skills” and “to make schools accountable for students‟ learning.” Two high school principals (P-06 and P-04) referred to the TAKS as an instrument of accountability: “…currently, TAKS is the State Accountability System test in order to comply with NCLB.” A more comprehensive response regarding the purpose of TAKS was given by non-ESL certified teacher (non-ESL-14) who handles ELLs: “At the state level, the TAKS is the state education agency‟s tool for determining the accreditation status of schools and school districts…With the enactment of the No Child Left Behind Act, the TAKS is now being used also as a tool, among other indicators, for determining the attainment or non-attainment of an adequate yearly progress (AYP). It is expected that a school carrying standards-based curriculum and intervention, especially one that is predominantly attended by minority populations, would show improving performance until the targeted year 2013 - 2014 when all children have met standards, which is the essence of the phrase „No Child Left Behind.‟ The TAKS performance is disaggregated into subgroups, so that again, there is a way of determining whether
109 achievement gaps are being bridged. Much of the accountability for student improvement on knowledge and skills in the four academic areas falls on schools. The other indicators of adequate yearly progress are completion rate and attendance.” In addition to being an instrument of accountability, respondents suggested that assessment results provide data for comparative analysis. According to one assistant principal (AP-09), TAKS is given “to compare/contrast statewide similarities and differences” in achievement levels “between schools” and across districts. During one of the focus group sessions, teachers gave these comments regarding TAKS: “The test administration is the same for all students. We have to follow the guide from the state. But, getting the kids ready for the test is different. We don‟t do anything differently, but we should. It‟s going to take more than what we are doing right now to ensure these kids (ELLs) graduate with the same diploma, meeting the same standards, as the general population.” Knowing what TAKS is for, does not necessarily translate into unanimity in terms of agreeing on what schools must do to respond to the challenge that the test brings. Respondents‟ opinions however, indicate the sense that schools need to adapt, and do things differently if they expect to achieve some level of parity between the TAKS achievement scores of ELLs and their non-ELL counterparts.
Anticipated Results of ELLs on English Language Arts and Mathematics TAKS Table 4.10 shows the responses to the following question: What are the anticipated results on the statewide testing, specifically English Language Arts and Mathematics portion of the Texas Assessment of Knowledge and Skills (TAKS), on English language learners (ELLs)? Almost half (49%) of those who responded expected ELLs to have at least average scores in TAKS. However, 16% observed ELLs‟ performance to be lower than non-ELLs, and 15% projected low results for ELLs because of their lower English language skills.
Table 4.10 What are the Anticipated Results of Statewide Testing for ELLs? Emergent Themes Average Scores Everyone Pass/Meet AYP Lower than Non-ELLs Low: Language Barrier Curriculum Improvement Dismal Cause Dropouts Gap in Scores to Lessen Team Effort Improvement Political Modified Diploma Not Sure Dist.(6) 1 2 1 1 2 0 1 1 0 0 0 0 Prin (6) 2 1 2 1 0 0 0 0 0 0 0 0 A.Prin (9) 3 3 0 1 0 1 0 1 0 0 0 0 ESLTeach(15) 3 3 3 3 0 0 1 0 0 1 1 1 NonTeach(19) Tot/%(55) 7 16/29 2 3 2 2 1 0 0 2 0 0 0 0 11/20 009/16 08/15 0004/7 0002/4 00 2/4 00 2/4 00 2/4 00 1/2 00 1/2 00 1/2
112 One high school principal (P-04) anticipated that the: “results of statewide testing are that all students will be expected to meet minimum state standards.” Another high school principal (P-02) expressed with some sense of exasperation that: “English language learners tend to score extremely low on the ELA and Mathematics portions of the TAKS. The state and the federal governments expect us to work miracles with these groups of students. If a student is in the U.S. for a year or two, he/she needs extra remediation to get a significant gain in TAKS scores. He may not necessarily pass the TAKS, but he will make gains. Keep in mind the NCLB ruling. Making any gains is not enough. The students must perform at 70% or better on all four sections of TAKS. At this point, all administrators in poverty-stricken areas are in fear of losing their jobs due to the low TAKS scores. We are indeed under extreme pressure!” One principal (P-05) indicated that the TAKS “will have a negative impact on the English language learners because of their difficulty in reading and writing in English.” The same negative outlook was expressed by an assistant principal (AP-03) who bluntly predicted that: “ELL students will fare worse than the general population.” A non-ESL teacher handling ELLs (Non-ESL-14) proposed a measure to address the feared, and somewhat expected underachievement of ELL Learners:
113 “Title I programs are instituted to address the intervention needs of ELLs so that they perform well in statewide testing. The said federal assistance is aimed at incrementally improving the performance of disadvantaged groups, which include the ELLs and the ethnic population they comprise. In my particular school, intervention translates into identifying those who are at risk in this group, breaking down their previous performances into specific areas and conducting the needed mentoring.” During a focus group interview, one member of the group expressed this view regarding the anticipated results TAKS : “I think that if we are going to require ELLs to be as successful on TAKS as any native English speaker then those students need specific language classes in school. ELLs need English classes whose sole purpose is teaching them to understand and speak English, with basic grammar skills included. In addition to a language class, they do still need a literature-based class in which they are learning to read and analyze text, as well as write both creatively and analytically. Also, having tutoring available, either built into the school day, or after school, during which ELLs practice speaking and listening in English would do a world of good. Teachers would not need to be the only tutors either, we could tap into a college‟s community service group, or even take advantage of peer tutoring.”
114 Actual Results of ELLs in English Language Arts and Mathematics TAKS Table 4.11 summarizes the responses for the question: What are the actual results of TAKS, specifically English Language Arts and Mathematics, on ELLs? Responses were related to how the ELLs in the schools actually performed on TAKS. Results were more negative than positive. Twenty-two percent (22%) observed dismal or failing performance, 18% noticed performance of the ELLs to be lower than non-ELLs and 11% mentioned that low results frustrate ELLs.
Table 4.11 What are the Actual Results of Statewide Testing for ELLs? Emergent Themes Dismal/Failing Lower than Non-ELLs Average Results No Response Frustrated ELLs Better than Average Don‟t Know Gap Narrowed Showed Areas to Improve Political Teacher Teach TAKS Dist.(6) 0 3 0 0 3 0 0 0 0 0 0 Prin (6) 1 1 2 0 0 2 0 0 0 0 0 A.Prin (9) 5 1 2 0 0 1 0 0 0 0 0 ESLTeach(15) 1 3 2 4 2 0 0 0 1 1 1 NonTeach(19) 5 2 3 4 1 0 2 2 0 0 0 Tot/%(55) 12/22 0 10/18 009/16 08/15 0006/11 000 3/5 00 02/4 00 02/4 00 01/2 00 01/2 00 01/2
116 Actual results in TAKS by ELLs are shared by an assistant principal (AP-03): “ELL students do fare worse than the general population.” The same observation was given by a non-ESL certified teacher (Non- ESL- 03): “Actual results show that ELLs perform poorer than native speakers and naturally so for they are very much handicapped as far as their understanding of the intricacies of the language is concerned. Their scores result in schools being „below standard‟, so to speak.” A non-ESL certified teacher (Non-ESL-14) offered a contrasting view by sharing the actual experience in their district: “I do not have the exact statistical report as far as the performances of ELLs are concerned, but my school district came up with a report lauding the improvement across grade levels in Mathematics and English Language Arts among ELLs, except in 9th Grade Mathematics. The district superintendent also mentioned about the narrowing of the achievement gap, and cited the percentage points of improvement.” Intended Consequences of TAKS for ELLs Table 4.12 summarizes the responses to the question: What are the intended consequences of this statewide testing as it relates to ELLs? With respect to the intended consequences of TAKS, 18% of the respondents expressed the understanding that statewide testing is
117 intended to eventually result in ELLs performing as well as the rest of the students and 18% agreed that TAKS is a tool to assess ELLs. Eleven percent (11%) of the respondents expressed confidence that ELLs can improve academically and eventually join the mainstream, and 11% demonstrated awareness that TAKS may improve the graduation rate. Teachers (9% of the respondents) expressed their wish for an opportunity to teach ELLs individually with the end in mind of improving their performance in TAKS.
Table 4.12 What are the Intended Consequences of TAKS for ELLs? Emergent Themes Same Level of Expectations Assess ELLs ELLs Improve-Mainstream Improve Graduation Rate ELLs Learn English ASAP School Accountability Individual Teaching of ELLs Equal Learn-Opportunities Improve Teacher Practices Not Sure No Response Dist.(6) 1 1 2 0 0 1 0 0 0 1 0 Prin (6) 2 1 0 1 0 1 0 1 0 0 0 A.Prin (9) 0 1 1 1 2 1 0 1 1 1 0 ESLTeach(15) 4 4 1 2 2 1 2 0 1 0 0 NonTeach(19) 3 3 2 2 2 1 3 1 1 1 1 Tot/%(55) 10/18 0 10/18 0006/11 006/11 00 06/11 0000 5/9 00 005/9 0 003/5 00 003/5 0 00 03/5 1/2
119 An assistant principal (AP- 09) stated that statewide testing is intended to ensure equality of learning opportunities for minority groups. Another assistant principal (AP-03) saw the tests as eventually resulting in the elevation of standards and the improvement of ELL students‟ English language skills. A number of non-ESL certified teachers held similar views: “With this statewide testing, ELLs can show how much they have gotten into the „mainstream‟ of the American educational system. In so doing, the government will get an „assurance‟ that graduates of this educational system (or future adults of this society) are competent and can function well in… this system” (Non-ESL-03). “The ultimate objective of this educational reform is for everyone... particularly minority populations, to be educationally competent so that they become more and more socio-economically integrated into the American mainstream” (Non-ESL-14). While the respondents demonstrated competent awareness of the intended consequences of the TAKS, a few were quick to point out the challenges that testing poses. A principal (P-02) stated: “I agree with the No Child Left Behind. We care about each and every student, but you can‟t compare a 5th generation student with a student who has only experienced a month or two in the U.S.A. and cannot speak the English language or may not have had formal education in his/her home country. These students are an
120 extremely difficult challenge. This challenge is rarely seen in the suburban schools.” On the other hand, one non-ESL certified teacher (Non-ESL-09) observed that the intended consequence of the TAKS: “Is to let ELL students stay in high school for more than 4 years because they can not graduate without passing the TAKS exit level…it should serve as a challenge to ELL kids to strive more and work harder because …we all agree that at some point, Mathematics in itself is a language…so anybody who has this class is learning a new language… ELLs should not use language as a barrier to learning Mathematics.” An English teacher of a focus group expressed this opinion regarding how statewide assessment affects ELLs: “Statewide assessments that are intended for non-ESL students have detrimental effects on the ELL populations. Many are indeed good students, but they get their results back and only see failure. The educator also sympathizes because he/she sees firsthand how far the student has come, but there is no state test for progress. It becomes frustrating for all parties involved. Sometimes administrators assume the teacher is not doing his/her job because the scores come back so low.”
121 What Has Happened to TAKS, As It Relates to ELLs What has happened because of TAKS, as it relates to ELLs? Responses given to this question are shown in Table 4.13. Twenty-six percent (26%) of the respondents reported that there is a high failure rate among ELLs. Other respondents observed that ELLs experienced a diminished self-esteem because of low scores (reported by 11% of the respondents) and tests caused frustration and exasperation on the part of ELLs (also seen by 11% of the group). There is also pressure on ELLs and the schools they attend to improve performance as noted by 11% of the respondents. Eleven percent (11%) of the respondents reported that there is now an overwhelming emphasis on test performance in their campuses.
Table 4.13 What Has Happened to ELLs Because of TAKS? Emergent Themes Higher Failure Rate-ELLs Dropout Rate Increased ELLs Have Low Self-Esteem Pressure on School/ELLs Emphasis Test Performance Frustration/Exasperation Focus on Needs of ELLs Teachers Teach to TAKS Continue Testing Changes ELLs Repeat TAKS Often Acceptable Test Scores Get Rid of TAKS No Response Dist.(6) 1 2 0 0 1 1 1 0 0 0 0 0 0 Prin (6) 1 1 1 1 0 1 1 0 0 0 1 0 0 A.Prin (9) 5 1 1 1 0 0 0 0 1 0 0 1 0 ESLTeach(15) 3 1 1 1 3 2 1 2 0 2 1 1 1 NonTeach(19) 4 2 3 3 2 2 1 2 2 0 0 0 1 Tot/%(55) 14/26 0 7/13 006/11 06/11 00 6/11 00 6/11 00 4/7 00 4/7 00 03/5 00 02/4 00222/4 00 2/4
123 District personnel (EDP-03) noted that “many of the ELL students have dropped out, fallen behind and in some instances stopped caring. Those who successfully passed the TAKS are concerned with their family and friends, so they share the failure. An assistant principal (AP-02) stated that TAKS has resulted in an “increase in dropout rate, low self esteem.” An ESL teacher (ESL-01) has observed that “many ELLs fear TAKS and do not think they will ever be able to pass it… many drop out of school altogether.” Another ESL teacher (ESL-05) noticed something similar: “Every year ELLs either pass and exit the program, or fail… they experience a sense of failure.” “The test has caused frustration and exasperation on the part of many students because the stakes are too high. They are only judged on this one test” (ESL – 10). Similarly, a nonESL teacher (Non- ESL- 03) observed that: “Because of TAKS (and its results), ELLs have become the “trail enders” (the ones at the end of the trail). Not performing well in these tests and being “left behind” by the “better” students has given them lower self-esteem and feelings of frustration. Failing the test one too many times may result in feelings of frustration and, possibly, rejection on their part.” An ESL teacher (ESL-07) observed that “ELLs have been penalized, a large portion must re-take TAKS multiple times in order to
124 pass, and schools have been held accountable for the failure of that population of students.” A more optimistic response was expressed by a non-ESL teacher (Non- ESL-14) regarding what happened to ELLs because of TAKS: “School administrators and educators have become more aware of the standards and expectations that students must reach for them to become competent human resource in the workplace and society. Not only has there been a gap among ethnic populations, with the ELLs at the disadvantaged end of the spectrum; there has also been a gap between competencies of exiting high school students and the actual skills and scholastic aptitude required in postsecondary education. Falling on the shoulder of the academic workforce, the TAKS has pushed these school stakeholders to intensify their efforts of addressing the needs of ELLs. Student expectations and academic objectives are being laid out more clearly, and their assessments more meticulously scrutinized with students, as part of instruction, in order for them to become more aware and self-reflective in their learning. There is a culture of concern for statistical performance, broken down furthermore into numbers vis-à-vis chunks of knowledge and skills in a specific subject area. Even ELLs have become familiar with the language of testing.”
125 Problems of ELLs, Due to TAKS What problems have occurred, if any, to ELLs related to or because of TAKS? Responses are summarized in Table 4.14. Of the 55 respondents, 24 % mentioned the higher dropout rate as one of the problems experienced by ELLs due to TAKS. At least one from each group expressed this problem. Graduation rate decreased for ELLs due to this statewide test - this was echoed by 22% of the respondents. Another negative aspect is perceived to be the low esteem of ELLs as seen by 13% of the group; 15% of the respondents realized the need to support ELLs.
Table 4.14 What Problems Have Occurred for ELLs Due to TAKS? Emergent Themes Higher dropout rate Low graduation rates Support for ELLs Low self-esteem Instruction is test oriented Increased achievement gap Ostracized due to low scores Tutorials/staff development True ability not measured No problem Limited reading ability Rush ELLs to learn English No Answer Dist.(6) 3 2 0 0 1 0 0 0 0 0 0 0 0 Prin (6) 2 0 1 1 0 0 0 1 0 2 0 0 0 A.Prin (9) 1 1 1 1 0 2 1 0 2 0 0 0 0 ESLTeach(15) 4 4 4 3 2 1 0 1 0 0 0 0 0 NonTeach(19) 3 4 2 2 1 1 2 1 1 1 2 1 1 Tot/%(55) 13/24 0 11/20 008/15 07/13 00004/7 000 4/7 00 03/5 00 03/5 00 03/5 00 03/5 00 02/4 01001/2 01001/2
127 Problems experienced by principals included the following: “Some have dropped out; others have done well enough to finish high school but fear college. Others have lost all hope and courage to continue” (P-03). “The results of TAKS have caused the English language learners to feel as if they are failures” (P-05). A non-ESL teacher (Non-ESL-03) observed the same problem: “ELLs … may just disengage themselves from any school activities and studies, and may, ultimately, check out of school.” This was echoed by an assistant principal (AP-07) who observed that ELLs “leave high school, complete all credits but do not receive diploma due to failing statewide exam.” Though challenged, a principal (P-02) remained optimistic: “Some of our English language learners‟ hopes and dreams of passing the state exams never become a realization, but we don‟t give up on them. We keep them and continue to work with them individually so that they may succeed.” The ELLs‟ performance in TAKS was explained by a non-ESL teacher (Non-ESL-14): “ELLs enroll into their grade levels not quite equipped with the prerequisite skills to perform” according to standards. “Even with the administration of the TAKS in the native language of the ELLs, there still exists an academic gap, because the testing language is not essentially native but academic, an orientation with which ELLs are not familiar. Only 4 years old, the TAKS as a federal tool
128 for adequate yearly progress is battling against a culture (focused on) economic survival. The TAKS measures scholastic aptitude and academic achievement - attributes that make up the profile for college readiness, or postsecondary education.” It was also observed that ELLs are more likely to need employment to survive. ELLs predominantly come from the working class, and they view attendance in school as a bureaucratic legal requirement rather than as an opportunity to develop skills necessary for social mobility. Changes Which Occurred as a Result of Statewide Testing, as They Relate to ELLs Table 4.15 displays the responses to the question: What changes have occurred as a result of statewide testing as it relates to ELLs? Due to the low performance of ELLs, 35% of those who responded to the questionnaire saw the need to focus and intervene on behalf of ELLs. Although 16% observed no changes because of TAKS, 15% realized that there will be more pressure at home and school to help ELLs pass TAKS. Thirteen percent (13%) of the respondents affirmed their commitment to be accountable for the ELLs.
Table 4.15 What Changes Have Occurred for ELLs Due to TAKS? Emergent Themes Focus/Intervention for ELLs None More pressure: school/home Accountability for ELLs Teacher development Uncertain Emphasis: test performance Positive pressure to perform Instruction is test oriented Fewer Exemptions Need effective leadership Not welcome: public schools Dist.(6) 2 0 1 1 0 2 2 0 0 1 0 0 Prin (6) 3 0 1 2 1 0 0 0 0 0 0 0 A.Prin (9) 2 3 1 2 1 0 0 0 0 0 1 0 ESLTeach(15) 4 3 3 1 2 2 0 1 0 1 0 0 NonTeach(19) 8 3 2 1 1 1 1 1 2 0 2 1 Tot/%(55) 19/35 0 9/16 008/15 07/13 000 5/9 000 5/9 00 3/5 00 2/4 00 2/4 00 2/4 00 1/2 010 1/2
130 In response to what should be done regarding the expected changes, a non-ESL teacher (Non-ESL-11) suggested that “the changes that have occurred include more intervention programs in classrooms, on campuses and in districts.” This move was also supported by another non-ESL teacher (Non-ESL-14): “There is a growing number of ELLs being motivated to perform according to standards. These are the ones who can actually see beyond high school graduation upon passing the exit-level TAKS. Opportunities for postsecondary education exist along with standardized testing, and so the high school journey for these ELLs seeing the long-term meaning of the TAKS gives them a greater sense of direction. The TAKS has also created positive pressure among this group of ELLs in that they have become aware of specific academic competencies they are expected to attain, thereby creating in them greater self-efficacy and confidence for postsecondary education.” An assistant principal (AP-06) observed that: “The changes that have occurred in some schools have been teachers receiving more staff development in how to instruct students whose first language is not English. More responsibility has been placed on teachers in all classes to help strengthen the students‟ ability to pass the test.”
131 A non-ESL teacher (Non- ESL- 09) also saw that “changes in teaching strategies are affected…teachers should be able to teach using English as a medium of instruction and yet accommodate the ELLs in their classes.” A principal (P-02) offered this strategy to cope with changes due to TAKS: “We are focused on each child. The teachers are aware of their strengths and weaknesses. Teachers meet with administrators to talk about the students and what strategies they will use to help the ELL attain the level of academic achievement that the state and the school require.” Recommendations to Improve Performance of ELLs in Statewide Testing Respondents gave their recommendations on how to improve the overall performance of ELLs in TAKS; these are shown in Table 4.16. Recommendations involved: (1) different/fairer test; and, (2) better assistance from the school through the teachers and the curriculum. Twenty percent (20%) of the respondents clamored for a different and fair test, while 9% opted for a test to be given 5 to 7 years after ELLs started schooling in the country. Eighteen percent (18%) recognized the need for better prepared teachers; 15% requested for modifications in teaching; and 11% proposed a paced curriculum for ELLs.
Table 4.16 What Recommendations are suggested for Improvement of ELLs Performance on TAKS? Emergent Themes Different/fairer test Better prepared teachers Teaching modifications Paced curriculum Test ELLs later (5-7years) Increase overall performance Intense English programs Bilingual teachers for ELLs Monitor ELLs Test preparation classes Smaller classes-create PLCs Involvement of parents Dist.(6) 1 1 2 2 0 0 0 0 0 1 0 0 Prin (6) 0 3 2 0 0 0 1 0 1 0 0 0 A.Prin (9) 0 0 1 1 1 1 3 0 1 0 1 0 ESLTeach(15) 5 4 1 0 2 2 0 0 1 0 1 0 NonTeach(19) 5 2 2 3 2 2 0 3 0 1 0 2 Tot/%(55) 11/20 0 10/18 008/15 06/11 000 5/9 000 5/9 444 4/7 00 3/5 00 3/5 00 2/4 00 2/4 010 2/4
133 ESL district personnel (EDP-03) recommended: “State legislators (individuals who formulate the TAKS test) need to speak with ELL teachers as well as the ELL students” to develop a more appropriate test which measures achievement. Two ESL teachers (ESL-06 and ESL-07) offered the following changes in the administration of the TAKS to ELLs: “There should be varying levels of the TAKS test for ELLs. When students fail the TAKS, it‟s not because they are incapable of passing…but the test is not a very ELL-friendly exam. If TAKS is „supposed‟ to be a basic skills test for non-ELLs, then a test should be devised to test the basic skills that ELLs have acquired since they have learned English. It is not fair that a student that has been speaking English their entire life takes the same test as a person that has only been speaking English 3 or more years in school.” “Have an „alternate‟ state test for the students who enter the country/school at such a late age, which they realistically cannot be expected to have enough of a grasp of English to pass the TAKS. This of course would be a small portion of the ELLs. ” A non-ESL teacher (Non-ESL-16) proposed “that ELLs be given at least 5 years of residency before they are given a statewide test. Teachers need to be certified to teach ELLs so that the “problem” can be addressed properly.”
134 An assistant principal (AP-02) recommended this plan of action: “allow them more time to learn the English language before administering the TAKS Exit Level Test; schedule them in intense English/Reading classes for 75% of the school day.” An even more drastic move was suggested by a non-ESL teacher (Non-ESL-17) who suggested that we “do away with TAKS as a state test” altogether. Another non-ESL teacher (Non- ESL-14) recommended the following regarding TAKS: “Curriculum and instruction must translate standards into classroom experience that takes into account their socio-cultural milieu, their prior knowledge and skills and their linguistic repertoire. It would help if those who are academically lagging behind, or enrolled at a level several years above their actual level, are given remedial courses so that they do not necessarily have to study material that is incoherent to them, but would bring them up from where they are to where they are supposed to be. Smaller classes or intervention groups would perhaps ensure that greater attention is given to the individual ELL‟s need. Since intrinsic motivation for academic study is undermined by the survival mode, perhaps a culture of career orientation can be instituted where professionals from the community could conduct career activities in schools. This will indirectly motivate towards ELLs
135 seeing the significance of standardized testing. Increased involvement of parents is a big factor in this intrinsic motivation.” Recommendations by a focus group included the following: (1) “Much, much smaller classes for one thing, but it never happens because it always comes down to money”; (2) “Home and community push for excellence. Adult business tutors brought in on a weekly basis”; and (3) “I believe that technology, certain computer programs, may be beneficial to the ELL population.” Recommendations of Greatest Value for ELLs‟ Success on Statewide Testing From the recommendations given, the respondents identified what will be of greatest value for ELLs‟ success on statewide testing. These suggestions are given in Table 4.17. Intervention for ELLs was recommended by 27% of the respondents, while 16% emphasized quality instruction, and 13% suggested a modified test. A more intensive English program was proposed by 9% of the group, while 9% also observed the need for help from home or for ELLs to have meaningful tutoring in school.
Table 4.17 What are the Recommendations, with the Greatest Value, are offered for ELLs Success on TAKS? Emergent Themes Interventions Quality Instruction Modified Test Intensive Programs for ELLs Tutoring More ESL Teachers Practice Test 5 or More Years Residency Bilingual/ESL Classes None/Don‟t Know Smaller Class Modified Diploma Effective School Leadership Dist.(6) 2 1 0 1 0 0 2 0 0 0 0 0 0 Prin (6) 3 2 0 0 0 0 0 0 1 1 0 0 0 A.Prin (9) 1 1 1 2 0 0 1 0 2 1 0 0 1 ESLTeach(15) 3 0 3 2 2 2 0 1 0 0 1 1 0 NonTeach(19) Tot/%(55) 6 15/27 5 3 0 3 2 1 3 0 1 0 0 0 0 9/16 007/13 0 5/9 0005/9 0004/7 00 4/7 00 4/7 00 3/5 00 3/5 00 1/2 00 1/2 00 1/2
137 A principal (P-03) considered this scenario as beneficial to the ELLs: “Better trained ESL teachers, more training for the traditional teachers and training for administration on an assets-based model rather than deficit. Also setting up more parent and community involvement. Parents do care and want the best for their children but they need to feel comfortable coming to school and made to feel welcomed.” An assistant principal (AP-09) suggested that: “Bilingual/ESL classes should be continuously offered. Certified Bilingual/ESL teachers should be recruited to offer even more assistance to these students.” This suggestion finds support from an ESL teacher (ESL-03) who stated that: “ELLs must be taught the skills that will allow them to assimilate into a new society at a rigorous level. The test should reflect their needs and also be more culturally sensitive.” A non-ESL teacher (Non-ESL-14) averred that: “The remedial intervention program for ELLs whose academic skills fall below their actual grade level would greatly help in an immediate manner.” Yet another non-ESL teacher (Non-ESL-17) suggested that “ESL teachers and regular teachers focus on teaching skills that would be most beneficial to the students based on their future aspiration (work or college after high school).”
138 Regarding statewide testing, an assistant principal (AP-06) proposed that: “The test should be written in the language the student speaks. Are we testing for academic skills? Or, are we trying to hold some schools more accountable than others when giving tests? Schools that have a higher minority rate must have teachers who are more specialized in special education or bilingual education.” During one of the interviews, a principal stated a rather grim scenario involving ELLs: “Standards are going to be difficult for them to achieve due to their lack of language and academic skills. It takes an ELL student five to seven years to acquire language skills adequate for the rigor of high school curriculum. These students eventually pass but not before causing tremendous challenges on our respective campuses. These students show up on our „dropout lists.‟ They have a huge impact on all our student data. For the most part these students come from dysfunctional homes. The father is in Mexico. Mother is here trying to raise a house full of kids- often forced to move from place to place due to financial difficulties. These students seldom stay in one school very long. I think the family unit should move here together and stay until the student graduates from high school.”
139 Another principal offered a radical suggestion regarding what can be done to handle ELLs regarding the issue of statewide testing: “If I could, I would send them (ELLs) all away from this school. They would all be placed in a school designed to introduce, develop, and build their language and culture skills, they bring our scores down. That shouldn‟t be any surprise. Their skills are weak. Their language is weak. Many of these students develop discipline and attendance problems. Our ISD provides a „new comers‟ program for ESL students. The campus is a separate campus-away from the main high school. Students remain on the „new comers‟ campus for about six months. Then, when they are ready, the students are transitioned onto the main campuses. They ruin our state scores; and wreak havoc on AYP scores. We have a large population of these kids. Our school data indicates that we have 19 percent ESL students. That does not, however, include the monitored kids. Most ESL students eventually pass the state exam but we will never get above „acceptable‟ status if we continue to try to teach and educate these kids on our campuses, within the regular population. The faculty and staff must work ten times harder than schools with few ESL kids.”
140 A principal explained how a strategy worked in their campus to help ELLs: “ELL students who continue to be monitored are placed in the appropriate English Language Arts course and are served by certified ESL instructors. As with all of our students, tutorial assistance is provided one hour each week after school for the four key academic content areas. Tutorials begin the second week of school and continue throughout the year (generally increasing in number prior to TAKS testing in the spring). Teachers closely monitor student performance and parents are contacted at the first sign of difficulty. Informal parent teacher conferences are held, and the student is encouraged to take every advantage of the assistance provided by the instructional staff. Students are held accountable for their study time. Note that our students are expected to study approximately one and a half hours per school night. Students and parents sign a contract to this effect prior to being accepted at the school. If informal parent conferences are not successful in providing the assistance and motivation to improve academic success, the Intervention Assistance Team is employed.” ELLs enrolling in a school during anytime during the year is a reality school administrators and teachers have to face in public schools.
141 They have to be ready to map out a course of action for each ELL to provide guidance and education during the stay of that student in their school. Discussion Research Question - Quantitative Is there a relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005, and 2006? The descriptive statistics showing the means of the 10th grade TAKS for English Language Arts and Mathematics do not indicate improvement in performance despite the decrease in the percent of ELLs enrolled in a school. Percent ELLs was 8.30% in 2003, 8.29% in 2004, 7.62% in 2005 and 6.94% in 2006. However, it is in the computations for Pearson r correlation that the significant relationship was determined. Using the SPSS software package for the TEA data of the sampled schools, the obtained Pearson r correlation coefficients were: -0.349 in 2003, -0.392 in 2004, -0.297 in 2005 and -0.398 in 2006 for English Language Arts; -0.293 in 2003, -0.351 in 2004, -0.382 in 2005, and -0.356 in 2006 for Mathematics. All of the computed Pearson r correlation coefficients were significant at the 0.05 level, two-tailed. All of the obtained Pearson r correlation coefficients were negative for both content areas; this inverse relationship indicated that as the percentage
142 of ELLs increased, the percentage of all 10th grade students passing in each of the core areas of English Language Arts and Mathematics decreased. A regression analysis using the percentage of ELLs enrolled in a school as a predictor variable resulted to a linear regression equation which can predict the percentage of all students passing in 10 th grade TAKS in English Language Arts and Mathematics. Any school which has ELLs should be aware of the number of these types of students. If there is a surge in the influx of ELLs sometime during the year, the school should carefully plan on how to prepare them for future testing. Both curriculum and instruction for ELLs may be put under review for improvement to better address this growing population of ELLs. Research Question - Qualitative What are the anticipated and observed consequences of the statewide testing specifically, TAKS, on ESL curriculum and instruction as viewed by certified ESL teachers, non-certified ESL teachers who teach ELLs, school administrators, and district ESL personnel? This major question was explored using the following probes: 1. Why is TAKS given as a statewide test? Respondents gave the following answers: (1) TAKS is given as a tool to gauge knowledge in the core areas or what is supposed to be taught; (2) TAKS is considered as a means to determine the school‟s status (Exemplary, Recognized, Academically Acceptable, or Academically
143 Unacceptable); (3) TAKS is regarded as an avenue to assess the state curriculum or standards; and (4) this statewide test is mandated by law and is aligned with NCLB. The reasons why TAKS is given have been understood by the different school personnel. 2. What are the intended consequences of this statewide testing? Respondents expressed the understanding that statewide testing is intended to eventually result in ELLs performing as well as the rest of the students. Respondents are confident that ELLs can improve academically and eventually join the mainstream; they are also aware that TAKS is a requirement for graduation. Teachers expressed their wish for an opportunity to teach ELLs individually with the end in mind of improving their performance in TAKS. School personnel affirmed their commitment on improving the situation of ELLs in school. 3. What problems have occurred related to or because of TAKS? Respondents identified the following as problems encountered by ELLs due to TAKS: (1) higher dropout rate; (2) graduation rate decreased for ELLs; and (3) low self-esteem of ELLs. These problems are rooted on the difficulty ELLs face in passing the TAKS tests. 4. What changes were caused by this statewide testing? Due to TAKS, schools experienced the negative reality that there is a high failure rate among ELLs. Other unpleasant realities included low self-esteem because of low scores and tests caused frustration and exasperation on the part of ELLs. There is mounting pressure on the
144 school and ELLs and emphasis is placed on test performance. Changes require action to improve the situation of the school, particularly those aspects which are related to the ELLs. Low scores have to be improved and ELLs need counseling on ways on how to have a positive view of anything one meets in life. 5. What are your recommendations to improve this statewide testing? Recommendations involved two main factors: (1) deferment of the test, possibly a different but fair test; and, (2) better assistance from the school through the teachers and the curriculum, modifications in teaching and possibly a paced curriculum for ELLs. Since one requirement of NCLB is testing and results indicate progress, there is need for school personnel to continually monitor ELLs and devise strategies to better help these students. 6. What needs to be done for the ESL students to improve their performance in general and specifically for this statewide test? The following recommendations were seen as more helpful to ELLs: Specific interventions for ELLs, quality instruction, a more intensive English program, help from home and meaningful tutoring in school. Both the school and the home, together with the community have to be partners in getting involved with ELLs.
145 Summary Since the researcher utilized the explanatory design of mixed methods, both quantitative and qualitative aspects were considered in this study. From the data gathered from TEA and after utilizing the SPSS software package for Pearson r correlation, the obtained results indicate that there existed significant, negative relationships between the percent of ELLs enrolled in a school and the percentage of 10th grade students passing in the English Language Arts and Mathematics TAKS tests given in 2003, 2004, 2005, and 2006. The null hypotheses were then rejected. The regression analysis provided the linear regression equations to predict the percentage of all students passing in 10 th grade TAKS tests in ELA and Mathematics using the percentage of ELLs enrolled in a school as the predictor variable. For the qualitative dimension of the study, the respondents were mostly certified in English Language Arts, Mathematics, and Special Education for teachers and some process of administrative certification for ESL district personnel, principals and assistant principals. As a statewide test, TAKS was seen as a tool to gauge knowledge and skills in the core areas. According to the respondents, students who took TAKS were expected to pass and have at least average scores. Unfortunately, most of the respondents saw dismal or low results in their respective schools, especially among ELLs. Consequently, focus had been
146 centered on this group of students and recommendations included testing them at a later date by using a modified test, and having interventions to improve the overall performance of ELLs. Administrators and teachers realized the need to improve the quality of instruction and provided interventions especially geared towards improving the academic performance of ELLs. It is hoped that both instruction and curriculum for ELLs were directed towards improving the plight of ELLs.
CHAPTER V SUMMARY, CONCLUSIONS, IMPLICATIONS AND RECOMMENDATIONS Summary The purpose of this study was to determine the impact of highstakes testing on English Language Learners (ELLs). This was shown in both the quantitative and qualitative dimensions of the study. Both quantitative and qualitative dimensions of the study provided the status of high-stakes testing as it affected ELLs and how it influenced efforts in schools to improve performance of students, particularly ELLs. Data obtained from Texas Education Agency (TEA) were used to determine whether there was a relationship between the percentage of ELLs enrolled in a school and the percentage of all students passing the 10th grade Texas Assessment of Knowledge and Skills (TAKS) tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005, and 2006. To support the qualitative aspect, this study explored what certified English as a Second Language( ESL) teachers, noncertified ESL teachers who teach ELLs, administrators, and district ESL personnel viewed as the impact that high stakes standardized assessments had on ELLs, ESL curriculum and instruction. Review of literature included the important consideration of the No Child Left Behind Act (NCLB). NCLB holds states using federal funds accountable for student academic achievement. States are required to develop a set of high-quality, yearly student assessments that include, at
148 a minimum, assessments in Reading/Language Arts, Mathematics and Science. NCLB requires states to report Adequate Yearly Progress (AYP) for all students and for subgroups, including students with limited English proficiency (Abedi, 2004). Phrases such as “student achievement,” “proficiency,” “raised expectations” and “testing” are implications of NCLB. Certainly, the focus on holding schools accountable for student achievement on standardized tests sets NCLB apart from previous versions of the law. (Guilfoyle, 2006). The focus of the study is the ELLs. The term “English language learner” is a recent designation for students whose first language is not English. This group includes students who are just beginning to learn English as well as those who have already developed considerable proficiency. The driving force behind including English language learners in statewide accountability testing is the legislation requiring it. In order to continue to receive Title I funds through NCLB, states must set high standards for all students and implement accountability systems to measure progress towards those standards. NCLB specifically states that English language learners must be included in statewide accountability testing, that their scores must be disaggregated so that it can be seen how they are achieving as a subgroup, and that the assessment system must accommodate their linguistic needs (“NCLB”, 2002).
149 High-stakes testing -- using standardized scores to impose consequences affecting teachers and students – has been embraced widely in recent years as a way to hold educators and students accountable for their performance. Experts say the movement is one of the most significant shifts in U.S. education in decades (Whoriskey, 2006). The goal of statewide accountability testing for English language learners (or for all students for that matter) is to improve standardsbased practices. The intended “washback” of including English language learners in standards-based assessment has been described as providing “the leverage needed to raise expectations for English language learners, and the emphasis on higher level skills should improve the quality of teaching provided to them” (Lachat, 1999, p.60), “feedback that will allow instructional leaders to improve instructional programs” (LacellePeterson & Rivera, 1994, p.64), and will ideally “…help students reach the standards by (a) influencing what is taught and how it is taught (i.e., „washback‟ to instruction), (b) providing data to guide instructional modifications, and (c) targeting resources to schools they are most needed” (Rivera & Vincent, 1997, p.336). In addition, Mehrens (2002) states that large-scale assessments have two major purposes: to drive reform and to gauge if reform policies have had an impact on student learning. These goals are especially important for English language
150 learners who often face socio-economic, cultural, and linguistic challenges to academic achievement. Demographic Data Total respondents who answered the on-line questionnaire totaled 55 – 35% are non-ESL certified teachers and 27% are ESL-certified teachers. The administrators accounted for the remaining 38%- 16% are Assistant Principals, 11% are Principals and 11% are ESL District Personnel. Conclusions The analysis of the quantitative data in Chapter IV led the researcher to draw the following conclusions: 1. The descriptive statistics showing the means of the 10th grade TAKS for ELA and Mathematics do not indicate improvement in performance despite the decrease in the percent of ELLs enrolled in a school. 2. All the obtained Pearson r correlation coefficients to determine whether there is a relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005, and 2006 were all significant at the 0.05 level, two-tailed. Both null hypotheses were rejected.
151 3. The negative Pearson r correlation coefficients implied that as the percentage of ELLs in a school increased, performance on both English Language Arts and Mathematics decreased. 4. The linear regression equations may be used to predict outcomes in 10th grade TAKS tests in ELA and Mathematics using the percentage of ELLs enrolled in a school as the predictor variable. The analysis of the qualitative data in Chapter IV led the researcher to draw the following conclusions: 1. Texas Assessment of Knowledge and Skills (TAKS) were perceived by respondents as a tool to gauge knowledge in the core areas. 2. ELLs were expected to have at least average scores on TAKS. 3. There was a difference in the expected and actual results. Respondents observed dismal or failing performance of ELLS in actual results in TAKS. This was evident by the high failure rate of ELLs in their respective schools. 4. Higher dropout rate and lower graduation rates of ELLs were problems encountered due to TAKS. 5. Respondents favored a different test for ELLs. 6. Respondents believed that interventions were needed to help ELLs perform better. Implications The research data gathered in the course of this study suggested that while there was a common perception that ELLs performed poorly
152 on high stakes testing, there was no unanimity among professionals in the field of education regarding the viability of options that might be considered in addressing the low achievement level of ELLs. This was not necessarily relevant, although it suggested that the appreciation of the problem and its causes lent itself to biases and distortions depending on the personal circumstance and perspective of those presenting these options. It was clear from the study that schools needed to do things differently, if they expected ELLs to perform better on standardized assessments. The major implications of the study were as follows: 1. The performance of schools in high stakes testing was affected by the size and proportion of ELLs taking the test. At the same time, ELLs were not evenly distributed across campuses. The performance of schools on standardized tests was influenced to a degree by the voluntary segregation in many districts of ethnic groups who speak English only as an adopted language. This was a phenomenon that was beyond the power of school districts to address, and that required wide coordination among various government agencies to develop an appropriate policy response. 2. The extended deferment of standardized tests administered in English to ELLS should be considered. The primary goal of these tests was to measure learning, that might be more accurately accomplished if the assessment was done in the language the student was most proficient. There was the expectation that the
153 student will eventually be proficient in English as well. Since a second language is acquired in degrees, it might be reasonable to assume that ELLs would not readily have the same facility for English as a native speaker. Administering the test in English before the ELL student was ready for it would compromise the stated goal of measuring learning as accurately as possible. 3. Learning is transmitted through communication. Due to the unique linguistic characteristic of ELLs, unique strategies, modifications, and instructions need to be used to maximize their capacity to learn concepts and skills. It is futile to assume that ELLs will learn the same way as native speakers of the English language. It follows that education professionals need the specialized training and support to be able to facilitate learning for ELLs. 4. The Language Proficiency Assessment Committee (LPAC) in each campus needs to take a more active role in monitoring the progress of ELLs and devising specific plans to properly respond to the requirements designed specifically for ELLs. They need to undertake a regular evaluation of instruction and curriculum for ELLs and communicate findings and recommendations to all stakeholders – school administrators, teachers, parents and the ELLs.
154 5. Interventions to improve the situation of ELLs should include specific action plans to devise a more intensive English program in schools and a continued emphasis on quality instruction employing strategies suggested by educational experts who have extensively researched on such courses of action. Recommendations for Further Study Based on the results of the study, the researcher recommends the following concerns for further study: 1. A study to determine what additional supports are needed to ensure that English language learners will pass high-stakes tests. 2. A study to identify what data are needed to make fair high-stakes decisions about English language learners (like subject grades, samples of class work and recommendations of teachers and counselors). 3. A study to determine the reasons why English language learners scored lowest among student groups in the Texas Assessment of Knowledge and Skills tests in the core areas of English Language Arts and/or Mathematics. 4. A study to explore different approaches in school campuses regarding handling of English language learners in terms of instruction, curriculum and other pertinent or related aspects (such as some sort of evaluation - academic, social, financial, etc.)
155 that may guide administrators and teachers to effectively handle English Language learners. 5. A study to determine the performance of 10th grade English language learners compared to non-English language learners and non-classified students based on the different objectives of the Texas Assessment of Knowledge and Skills tests in either or both Mathematics and English Language Arts. 6. A study to determine the impact of high stakes testing on English language learners as viewed by parents and students. 7. A study to explore different instruments to measure academic performance of English language learners. 8. A study to determine if there is significant difference between performance in the different core areas of English language learners belonging to different language groups. This study affirmed the expected outcome that a significant relationship existed between the percentage of ELLs enrolled in a school and the percentage of all students passing the 10 th grade TAKS tests in both core areas of English Language Arts and Mathematics. The regression analysis predicted that as the percentage of ELLs in a school increased, the performance in the statewide, high-stakes testing in terms of all students passing the 10th grade TAKS tests decreased. The respondents of the study considered the Texas Assessment of Knowledge and Skills (TAKS) as a tool to gauge knowledge in the different core areas.
156 English language learners were expected to have at least average scores on TAKS. There was a difference in the expected and actual results; respondents observed dismal or failing performance of ELLS in the actual results in TAKS. This was evident by the high failure rate of ELLs in their respective schools. Higher dropout rate and lower graduation rate of ELLs were problems encountered due to TAKS. Respondents favored a different test for ELLs, possibly given at a later date after ELLs had studied in the country for at least several years. Respondents believed that interventions were needed to help ELLs perform better. Both the school and the home, together with the community have to be involved in preparing ELLs to be better prepared for their present and future roles in the American society.
REFERENCES Abedi, J. (2004). The No Child Left Behind Act and English Language Learners: Assessment and Accountability Issues. Educational Researcher, 33(1), 4-14. Abedi, J. (2004). Will You Explain the Question? Principal Leadership (High School Ed.), 4(7), 27-31. Abedi, J., Hofstetter, C. & Lord, C. (2004). Assessment Accommodations for English Language Learners: Implications for Policy-Based Empirical Research. Abedi, J. (2004). Inclusion of Students with Limited English Proficiency in NAEP: Classification and Measurement Issues. CSE Report 629 CRESST/ University of California, Los Angeles. Abedi, J. (2004). Will You Explain the Title? Principal Leadership (High School Ed.), 4(7), 27-31. Abedi, J., Leon, S., & Mirocha, J (2003). Impact of student language background on content-based performance: Analyses of extant data (CSE Tech. Rep. No. 603). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing. Abrams, L.M., & Madaus, G.F. (2003). The Lessons of High-Stakes Testing. Educational Leadership, 61(3), 31-35.
Airasian, P. W. (1988). Symbolic validation: The case of state-mandated, high-stakes testing. Educational Evaluation and Policy Archives, 10(4), 301-313. Albus, D., Thurlow, M. L., Liu, K. K., & Bielinski, J. (2005). Reading Test Performance of English Language Learners Using an English Dictionary. The Journal of Educational Research, 98(4), 245-254. Albus, D., Thurlow, M. L., & Liu, K. K. (2002). 1999-2000 Participation and Performance of English Language Learners Reported in Public State Documents and Web Sites (LEP Projects Report 3). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Alderson, J. C., & Wall, D. (1993). Does Washback Exist? Applied Linquistics, 14(2), 115-129. Amrein, A. L., & Berliner, D. C. (2003). The Effects of High-Stakes Testing On Student Motivation and Learning. Educational Leadership, 60(5), 32-38. Anderson, M. E. (2001). Item Response Differences Among SpanishSpeaking ESL Students and Native English-Speaking Students on a Standardized Reading Test. Unpublished Manuscript. Anderson, M. E. (2004). Intended and Unintended Consequences of Statewide Testing for ESL Curriculum and Instruction. (UMI No. 3137152).
Anderson, M. E., Thurlow, M. L., Swierznin, B., Liu, K. K., & Bielinski, J. (2000). Bilingual Accommodations for Limited English Proficient Students on Statewide Reading Tests: Phase (No. 31 Minnesota report). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Artiles, A. J., Rueda, R., Salazar, J. J, & Higareda, I. (2005). Within Group Diversity in Minority Disproportionate Representation: English Language Learners in Urban City Schools. Exceptional Children, 71(3), 283-300. August, D., & Hakuta, K. (1997). Improving Schooling for Minority Children: A Research Agenda. Washington, D.C.: National Academy Press. Barro, S.M., & Kolstad, A. (1987). Who Drops Out of High School? Findings from High School and Beyond. Washington, D.C.: Center for Education Statistics, Office of Research and Improvement, U.S. Department of Education. Brennan, R. L. (2004). Revolutions and Evolutions in Current Educational Testing. Center for Advanced Studies in Measurement and Assessment: CASMA Research Report #6. California Department of Education. (2000). California demographics data. Retrieved February 24, 2003, from http://www.cde.ca.gov/demographics/
160 Center on Educational Policy (2005). From the capital to the classroom: Year 3 of the No Child Left Behind Act. Washington, DC: Author. Chamot, A. U., & O’Mailley. J.M. (1994). The CALLA Handbook: Implementing the Cognitive Academic Language Learning Approach. Reading, MA: Addison-Wesley. Cheng, L. (1999). Changing Assessment: Washback on Teacher Perceptions and Actions. Teaching and Teacher Education, 15, 253-271. Cheng, L. (2003). Looking at the Impact of a Public Examination Change on Secondary Classroom Teaching: A Hong Kong Case Study. Journal of Classroom Interaction, 38(1), 1-10. Cizek, G.J. (2003). When Teachers Cheat. Education Digest, 68(6), 28-31. Cobb, T. (2000). One Size Fits All? Francophone Learners and English Vocabulary Tests. Canadian Language Review, 57(2), 295-324. Collier, V.P. (1989). How Long? A Synthesis of Research on Academic Achievement in a Second Language. TESOL Quarterly, 23(3), 509531. Coltrane, B., (2002).English Language Learners and High-Stakes Tests: An Overview of the Issues. (November, 2002). Center for Applied Linguistics. Council of Chief State School Officers. (1992). Recommendations for Improving and Assessment and Monitoring of Students with
161 Limited English Proficiency. Washington, DC: Council of Chief State School Officers. Creswell, J.W. (1998). Qualitative Inquiry and Research Design: Choosing Among Five Traditions. Thousand Oaks, CA: Sage. Davis, M. R. & Sacks, J. L. (2005). California, U.S. Department of Education strike deal on NCLB rules [Electronic version]. Education Week. Datnow, A., Borman, G.D., Stringfield, S., & Overman L.T. (2003). Comprehensive School Reform in Culturally and Linguistically Diverse Contexts: Implementation and Outcomes from a Four-Year Study. Educational Evaluation & Policy Analysis, 25(2), 143-170. D’Emilio, T. E. (2003). LEP Student Participation in Special Education: Over or Under-Representation? Paper Presented at the CCSSO Conference on Large-Scale Assessment, San Antonio, Texas Denzin, H. (1970). The research act: A Theoretical Introduction to Sociological Methods. New York: McGraw-Hill. Dexter, L.A. (1970). Elite and Specialized Interviewing. Evanston, IL: Northwestern University. Edwards, V. (Ed.). (2003, Jan. 9). Quality Counts 2003: If I can’t learn from you… (Education Week Special Report), 12(17). Bethesda, MD: Editorial Projects in Education. Erlandson, D., Harris, E., Skipper, B., & Allen, S. (1993). Doing Naturalistic Inquiry. Newbury Park: Sage.
162 Erpenbach, W.J., Forte-Fast, E., & Potts, A. (2003). Statewide educational accountability under NCLB (An Accountability Systems and Reporting State Collaborative on Assessment and Student Standards Paper). Washington, DC: Council of Chief State School Officers. Escamilla, K., Mahon, E., Riley-Bernal, H., & Rutledge, D. (2003). HighStakes Testing, Latinos, and English Language Learners: Lessons from Colorado. Bilingual Research Journal, 27(1), 25-47. FairTest (2000). Flores, B.B., & Clark, E.R. (2003). Texas Voices Speak About HighStakes Testing: Preservice Teachers, Teachers, and Students, Current Issues in Education (Vol.6). Fowler, F. J. (1984). Survey Research Methods. Beverly Hills: Sage. Fraenkel, J. & Wallen, N. (2003). How to Design and Evaluate Research In Education (5th ed.). New York, NY: McGraw-Hill. Freeman, Y.S., & Freeman, D., E. 2002). Closing the Achievement Gap: How to Reach Limited-Formal-Schooling and Long-Term English Learners. Portsmouth, NH: Heinemann. Garcia, G. E., & Pearson, P. D. (1994). Assessment and diversity. In L. Darling-Hammond (Ed.), Review of Research in Education (Vol. 20). Washington, DC: American Educational Research Association. Gewertz, C. (2005). Ed. Dept. allows Chicago to provide NCLB tutoring [Electronic version]. Education Week.
Goldhaber, D., & Hannaway, J. (2001). Accountability with a kicker: Observations on the Florida A+ Plan. Paper presented at the annual meeting of the Association of Public Policy and Management, Washington, DC. Goldschmidt, P. and Wang, J. (1999) "When Can Schools Affect Dropout Behavior: A Longitudinal Multilevel Analysis," American Educational Research Journal, vol. 36, no. 4, pp.715-738, Winter. Guilfoyle, C. (2006). “NCLB: Is There Life Beyond Testing?” Educational Leadership, vol. 64, no. 3, pp. 8-13, November 2006. Haladyna, T., Nolen, S., & Haas, N. (1991) Raising standardized achievement test scores and the origins of test score pollution. Educational Researcher, 20(5). 2-7 Hammersley, M. (1992). What’s Wrong with Ethnography? London: Routledge. Haney, W. (2000). The Myth of the Texas Miracle in Education, Education Analysis Policy Archives (Vol.8). Harlow, A., & Jones, A. (2003, July). Why students answer TIMSS science test items the way they do. Paper presented at the annual conference of the Australian Science Education Research Association, Melbourne, Victoria, Australia. Herman, J., & Golan, S. (n.d.). Effects of standardized testing on teachers and learning (CSE Technical Report 334). Los Angeles:
164 National Center for Research on Evaluation, Standards, and Student Testing. Herszenhorn, D. (2003, July 23). Basic skills forcing cuts in art classes. The New York Times, p. B1. Heubert, J. (2000).High-Stakes Testing: Opportunities and Threats. In Pines, M (ed.). The Continuing Challenge: Moving the Youth Agenda Forward (Policy Issues Monograph 00-02, Sar Levitan Center for Social Policy Studies. Baltimore, MD: Johns Hopkins University Press. Heubert, J.P. (2003). First, Do No Harm. Educational Leadership, 60(4), 26-30. Heubert, J., & Hauser, R. (Eds.) (1999). High stakes testing for tracking, promotion, and graduation. Washington, DC: National Academy Press. Hill, R.K., & DePascale, C. A. (2003). Reliability of no child left behind accountability designs. Educational Measurement: Issues and Practice, 22(3), 12-20. Hinde, E.R. (2003). The Tyranny of the Test: Elementary Teacher’s Conceptualizations of the Effects of State Standards and Mandated Tests on Their Practice, Current Issues in Education. (Vol.6). Hoffman, J. V., Assaf, L., & Paris, S. G. (2001). High Stakes Testing in Reading: Today in Texas, Tomorrow? The Reading Teacher.
165 Holmes, E. (1911). What is and what might be: A study of education in general and elementary in particular. London: Constable. Hood, L. (2003). Immigrant Students, Urban High Schools: the Challenge Continues. Revised 09/16/2003, from http://www.carnegie.orge/immigrantstudents.pdf. Horn, C. (2003). High-Stakes Testing and Students: Stopping or Perpetuating a Cycle of Failure? Theory Into Practice, 42(1), 30-41. Horn, C., Ramos, M., Blumer, I., & Madaus, G. (2000). Cut scores: Results may vary. Chestnut Hill, MA: National Board on Educational Testing and Public Policy, Boston College. Isaac, S. & Michael, W. (1997). Handbook in Research and Evaluation for Education and the Behavioral Sciences (3rd ed.) San Diego, CA: EdiTs/Educational and Industrial Testing Services. Johnson, D. (2001). Performance Pentagon: Five Strategies to Help All Students Make the Grade. NASSP Bulletin, 85, 40-55. Jones, M., Jones, B., Hardin B., Chapman, L., Yarbough, T., & Davis, M. (1999). The impact of high-stakes testing on teachers and students in North Carolina. Phi Delta Kappan, 81(3), 199-203. Kaufman, P., Alt, M. N., & Chapman, C. D. (2001). Drop-Out Rates in the United States: 2000 (No.NCES 2002-114). Washington, DC: National Center on Educational Statistics. Kinder, A. L. (2002). Survey of the states’ limited English proficient students and available educational programs and services, 2000-
166 2001 summary report. Washington, DC: National Clearinghouse for English Language Acquisition and Language instruction Educational Programs. Klein, S., Hamilton, L., McCaffrey, D., & Stecher, B. (2000). What do test scores in Texas tell us? Education Policy Analysis Archives, 8(49). [Online]. Available: http://epaa.asu.edu/epaa/v8n49 Koretz, D., Barron, S., Mitchell, K., & Keith, S. (1996a). Perceived effects of the Kentucky Instructional results information system (KIRIS) (MR-792-PCT/FF). Santa Monica, CA: Rand. Koretz, D., Barron, S., Mitchell, K., & Keith, S. (1996b). Perceived effects of the Maryland school performance assessment program (CSE Technical Report 409). Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing. Krathwohl, D. (1993). Methods of Educational and Social Science Research: An Integrated Approach. New York: Longman. Krueger, R.A., & Casey, M. A. (2000). Focus Groups: A Practical Guide for Applied Research (3rd ed.). Thousand Oaks, CA: Sage Publications. Lacelle-Peterson, M., & Rivera, C. (1994). Is it Real for all Kids? A Framework for Equitable Assessment Policies for English Language Learners. Harvard Educational Review, 64(1), 55-75. Lachat, M. A. (1999). What Policy Makers and School Administrators Need to Know About Assessment Reform for English Language
167 Learners. Northeast and Islands Regional Educational Laboratory at Brown University. Providence. Lane, S., & Stone, C. A. (2002). Strategies for Examining the Consequences of Assessment and Accountability Programs. Educational Measurement: Issues into Practice, 21(1), 23-30. Lewis, A. (2002). A Horse Called NCLB. Phi Delta Kappan, 84(3), 179-180. Linn, R. (1998). Assessments and Accountability (CSE Technical Report 490). Boulder, CO: CRESST/University of Colorado at Boulder. Linn, R. L. (2000). Assessment and Accountability. Educational Researcher, 29(2), 14-16. Linn, R. L., Baker, E. L., & Herman, J. L. (2002). Minimum group size for measuring adequate yearly progress: The CRESST line. Los Angeles: University of California, Center for the Study of Evaluation/National Center for Research on Evaluation, Standards, and Student Testing. Liu, K. & Thurlow, M. (1999). Limited English Proficient Students’ Participation and Performance on Statewide Assessments: Minnesota Basic Standards Reading and Math, 1996-1998. National Center on Educational Outcomes. Madaus, G. (1988). The influence of testing on the curriculum. In L. Tanner (Ed.), Critical issues in curriculum (pp. 83-121). Chicago: University of Chicago Press.
168 Madaus, G. (1991, January). The effects of important tests on students: Implications for a national examination or system of examinations. Paper prepared for the American Educational Research Association Invitational Conference on Accountability as a State Reform Instrument, Washington, DC. Madaus, G., & O’Dwyer, L. (1999). A Short History of Performance Assessment. Lessons learned. Phi Delta Kappan, 80(9), 688-695. Major, R.C., Fitzmaurice, S.M., Bunta, F., & Balasubramanian, C. (2005). Testing the Effects of Regional, Ethnic, and International Dialects of English on Listening Comprehension. Language Learning 55 (1) 37-69. Marshak, D. (2003). No Child Left Behind: A Foolish Race Into the Past.Phi Delta Kappan, 85(3), 229-231. Marshall, C. (1990). Goodness Criteria: Are They Objective or Judgment Calls? In E. G. Guba (Ed.), The Paradigm Dialog (pp. 188-197). Thousand Oaks, CA: Sage. Maxwell, J. (2005). Qualitative Research Design. An Interactive Approach (2nd ed.). Thousand Oaks, CA: Sage Publications. McLaughlin, M.W., & Shepard, L.A. (1995). Improving Education Through Standards-Based Reform. Stanford, CA: The National Academy of Education. McMillan, J., Myran, S., & Workman, D. (1999, April 19-23). The Impact of Mandated Statewide Testing on Teachers’ Classroom
169 Assessment and Instructional Practices. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Quebec, Canada. McNeil, L.M. (2000). Contradictions of school reform: Educational costs of standardized testing. New York: Routledge. Mehrens, W. A. (1998). Consequences of assessment: what is the evidence? Vice Presidential Address for Division D, annual meeting of the American Educational Research Association, San Diego. Menken, K. (2000).What Are the Critical Issues in Wide-Scale Assessment of English Language Learners? Framing Effective Practice, 23-27. Merriam, S. B. (1998). Qualitative Research and Case Study Applications in Education, San Francisco, CA: Jossey-Bass. Miller & Fredericks (2000). Research on High-Stakes Tests and Instruction. Retrieved October 9, 2005 from http://vnweb.hwwilsonweb.com/hww/results/results_single_fullte xt.jhtml. Murnane, R. & Levy, F. (1996). Teaching the New Basic Skills. New York: The Free Press. Murnane, Richard J., John B. Willett, Yves Duhaldeborde, and John H. Tyler. 2000. How important are the cognitive skills of teenagers in predicting subsequent earnings? Journal of Policy Analysis and Management 19 (4) 547-568.
170 Myers, K. (2003, July 16). A dream denied: Aspiring chef rethinks her future as Falmouth school board bows to state pressure on MCAS. Cape Cod Times. Available: www.capecodonline.com. National Center for Education Statistics. (2002). Public school student, staff, and graduate counts by state: School year 2000-01 (NCES Publication 2002-348). Washington, DC: Author. National Clearinghouse for Bilingual Education. (1997). High-Stakes Assessment: A Research Agenda for English Language Learners. Retrieved June 21, 1999, from http://www.ncbe/gwu.edu/ncbepubs/reports/highstakes.htm#Te sting. National Commission of Excellence in Education. (1983). A Nation at Risk: The Imperative for Educational Reform: a Report to the Nation and the Secretary of Education, United States Department of Education. Washington, D.C.: The Commission: [Supt. of Docs., U.S. G.P.O. distributor]. No Child Left behind Act of 2001, 20 U.S.C. 6301 et.seq. (2002). O'Byrne, B. (2001). Needed: A Compass to Navigate the Multilingual English Classroom. Journal of Adolescent & Adult Literacy 44 (5) 440-9. Olson, L. (2005). Defying predictions, state trends prove mixed on schools making NCLB targets [Electronic version]. Education Week.
171 Orfield, G. & Kornhaber, M. L. (2001). Raising standards or raising barriers? Washington, DC: The Century Foundation Press. Patton, M. Q. (1990). Qualitative Evaluation and Research Methods (2nd ed.). Thousand Oaks, CA: Sage Publications. Pedulla, J., Abrams, L., Madaus, G., Russell, M., Ramos, M., & Miao, J (2003). Perceived effects of state-mandated testing programs on teaching and learning: Findings from a national survey of teachers. Chestnut Hill, MA: National Board on Educational Testing and Public Policy, Boston College. Popham, J. W. (2003). The Seductive Allure of Data. Educational Leadership, 60(5), 48-51. Reyes, P. & Fletcher, C. (2003). Successful Migrant Students: The Case of Mathematics. Journal of Curriculum and Supervision, 18(4), 306333. Rex, L. A. & Nelson, M. C. (2004). How Teachers’ Professional Identities Position High-Stakes Test Preparation in Their Classrooms. Teachers College Record, 106(6), 1288-1331. Rhoades, K., & Madaus, G. (2003). Errors in standardized tests: A systemic problem. Chestnut Hill, MA: National Board on Educational Testing and Public Policy, Boston College. Rivera, C., & Stansfield, C. (1998). Leveling the Playing Field for English Language Learners: Increasing the Participation in State and Local Assessments Through Accommodations. In R. Brandt (Ed.),
172 Assessing Student Learning: New Rules, New Realities (pp. 65-92). Arlington, VA: Educational Research Service. Rivera, C., Stansfield, C., Scialdone, L., & Sharkey, M. (2000). An analysis of state policies for the inclusion and accommodation of ELLs in state assessment programs during 1998 – 1999 (Executive Summary). Washington, DC: The George Washington University, Center for Equity and Excellence in Education. Rivera, C., & Vincent. (1997), High School Graduation Testing: Policies and Practices in the Assessment of English Language Learners. Educational Assessment, 4(4), 335-355. Rose, L.C. & Gallup, A.M. (2003). The 35 th Annual Phi Delta Kappa/ Gallop Poll of the Public’s Attitudes Toward the Public Schools. Phi Delta Kappan, 85.1, 41-56. Rosenbusch, M. H. (2005). The No Child Left Behind Act and Teaching and Learning Languages in U.S. Schools. The Modern Language Journal, 89(2), 250-261. Russell, M., & Abrams, L. (in press). Instructional uses of computers for writing: The impact of state testing programs. Teachers College Record. Sheldon K. and Biddle, B. (1998). Standards, accountability and school reform: Perils and pitfalls. Teachers' College Record 100 (1): 164180. Sirkin, R. M. (2006). Statistics for the Social Sciences. (3rd ed.). Thousand
173 Oaks, CA: Sage Publications. Solano-Flores, G., & Trumbull, E. (2003). Examining Language in Context: The Need for New Research and Practice Paradigms in the Testing of English Language Learners. Educational Researcher, 32(2), 3-13. Spellings, M. (2005). Decision Letter on Request to Amend Virginia accountability plan. Retrieved November 5, 2005 from http://www.ed.gov/print/admins/lead/account/letters/acva5.ht ml. Stake, R. (1999). The Goods on American Education. Phi Delta Kappan, 668-672. Stecher, B., Barron, S., Chun, T., & Ross, K. (2000). The effects of the Washington state education reform on Standards and Student Testing. Sum, A. (1999). Literacy in the Labor Force. Washington, DC: National Center for Education Statistics. Taylor, G., Shepard, L., & Kinner, F. (2003). A Survey of Teachers’ Perspectives on High-Stakes Testing in Colorado: What Gets Taught, What Gets Lost (CSE Technical Report No. 588). Santa Cruz, CA and Los Angeles, CA: Center for Research on Evaluation, Diversity, and Excellence and CRESST. Technical Digest, 2003 – 2004
174 Texas Assessments of Knowledge and Skills (TAKS), (2005). Data Resources and Research. Texas Education Agency (TEA) from http://www.tea.state.tx.us/data.html. Thurlow, M. L., Nelson, J. R., Teelucksingh, E., & Ysseldyke, J. E. (2000). Where’s Waldo? A Third Search for Students with Disabilities in State Accountability Reports (Technical Report No.25). Minneapolis, MN: National Center on Educational Outcomes. Thompson, G. L. The Real Deal on Bilingual Education: Former Language-Minority Students Discuss Effective and Ineffective Instructional Practices. Educational Horizons, 78(20, 80-92. U. S. Department of Education. (2002). Elementary and Secondary Education. Title IX. General Provisions. Part A, Section 9101. Definitions.  Core Academic Subjects. Washington, D.C.: U.S. Dept. of Education. U.S. Department of Education, Office of the Undersecretary, Planning and Evaluation Service. (1999). Promising results, continuing challenges: The final report of the national assessment of Title I. Washington, DC: Author U.S. Department of Education. (1995). The Improving America’s Schools Act of 1994: Reauthorization of the Elementary and Secondary Education Act. Washington, D.C.: U.S. Dept. of Education. Van Dalen, D.B. (1979). Understanding Educational Research (4th ed.). New York: McGraw-Hill.
175 Wall, M. E. (2004). A Case Study of Secondary School Efforts Toward English Language Learner Success in a Standards-Based Reform System. Wang, M. & Koda, K. (2005). Commonalities and Differences in Word Identification Skills among English Second Language Learners. Language Learning, 55 (1) 71-98. Weitzman, E.A. (2000). Software and Qualitative Research. In N.K. Denizin & Y.S. Lincoln (Eds.), The Handbook of Qualitative Research (2nd ed., pp. 803-820). Thousand Oaks, CA: Sage. Whoriskey, P. (2006).Political backlash builds over high-stakes testing. Public support wanes for tests seen as punitive. Washington Post Company. Winograd, M. (2002). Equity: A Prerequisite for Reform. Principal Leadership, 2(8), 42-47. Wright, W. E., (2006).A Catch-22 for Language Learners. Educational Leadership. November 2006/Volume 64/Number 3. Zehler, A. M., Fleischman, H. L., Hopstock, P. J., Pendzick, M. L., & Stephenson, T. G. (2003). Descriptive Study of Services to LEP Students and LEP Students with Disabilities. (No. 4 Special Topic Report: Findings on Special Education LEP. students). Arlington, Va.: Development Associates, Inc.
176 Zehler, A. M., Hopstock, P. J., Fleischman, H. L., & Greniuk, C. (1994). An Examination of Assessment of Limited English Proficient Students. Special Issues Analysis Center,Task Order Report. Arlington, VA: Development Associates.
APPENDIX A IRB
179 PRAIRIE VIEW A&M UNIVERSITY The Texas A&M University System P. O. Box 4149 Prairie View, Texas 77446-4149 Office of the Vice President Research and Development March 21, 2006 TO: Mr. Arthur L. Petterway, Principal Investigator Doctoral Student, Education Leadership and Counseling Dr. Ben DeSpain, EDLC – Faculty Advisor Marcia C. Shelton, Compliance Officer, Regulatory Research Institutional Review Board IRB Protocol Review – Protocol Status v. 936.857.4494 f. 936. 857.2255
Title: A Mixed Mthods Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas Protocol Number: 200-103 Review Category: Full Review - (primary reviewer –L. Myers) Approval Date: February 13, 2006 The approval determination was based on the following Code of Federal Regulations (CFR) 46.101(b) (2). Research involving the use of educational tests (cognitive, diagnostic,
aptitude, achievement), survey procedures, interview procedures or observation of public behavior, unless: (i) information obtained is recorded in such a manner that human subjects can be identified, directly or through identifiers linked to the subjects; and (ii) any disclosure of the human subjects' responses outside the research could reasonably place the subjects at risk of criminal or civil liability or be damaging to the subjects' financial standing, employability, or reputation.
Remarks: The Institutional Review Board – Human Subjects in Research, Prairie View A&M University has reviewed and approved the above referenced protocol. Your study has been approved for one year –February 13, 2006- February 12, 2007. As the principal investigator of this study, you assume the following responsibilities: Renewal: Your protocol must be re-approved each year in order to continue the research. You must also complete the proper renewal forms in order to continue the study after the initial approval period. Adverse events: Any adverse events or reactions must be reported to the IRB immediately. Amendments: Any changes to the protocol, such as procedures, consent/assent forms, addition of subjects, or study design must be reported to and approved by the IRB. Completion: When the study is complete, you must notify the IRB office and complete the required forms.
Date: 14-Jan-2006 00:33:10 -0600 From: <email@example.com> To: <firstname.lastname@example.org> Subject: Exam Confirmation - Clinical Research Training
Registration Summary: 01/14/2006 Arthur L Petterway Ph.D Prairie View A&M University Educational Leadership & Counseling P. O. Box 519 n/a MSC Prairie View TX 77446-0519 United States Tel: (713)924-1622 Fax: (713)924-1619 E-mail: email@example.com This e-mail is to verify that you successfully completed the NIH Clinical Research Training course. You answered 20 out of a total of 25 questions for a final grade of 80%. If you are an NIH principal investigator, you have fulfilled the Training and Education Standard issued by the NIH for conducting clinical research within the intramural research program. Please print this e-mail and retain for your records.
CITI Course in The Protection of Human Research Subjects
Sunday, February 19, 2006 CITI Course Completion Record for Arthur Petterway
To whom it may concern: On 1/15/2006, Arthur Petterway (username=apetterw) completed all CITI Program requirements for the Basic CITI Course in The Protection of Human Research Subjects.
Learner Institution: Texas A&M University Learner Group: Group 2. Learner Group Description: Social and Behavioral Research Investigators and Key Personnel Contact Information: Gender: Male Which course do you plan to take?: Social & Behavioral Investigator Course Only Role in human subjects research: Principal Investigator Mailing Address: 5300 N. Braeswood Blvd #247 Houston Texas 77096 USA Email: firstname.lastname@example.org Office Phone: (713)924-1622 Home Phone: (713)498-8667
The Required Modules for Group 2. are: Introduction History and Ethical Principles - SBR Defining Research with Human Subjects - SBR The Regulations and The Social and Behavioral Sciences SBR Assessing Risk in Social and Behavioral Sciences - SBR Informed Consent - SBR Privacy and Confidentiality - SBR Research with Prisoners - SBR Research with Children - SBR Research in Public Elementary and Secondary Schools - SBR International Research - SBR Internet Research - SBR Conflicts of Interest in Research Involving Human Subjects Texas A&M University Additional optional modules completed:
Date completed 01/15/06 01/15/06 01/15/06 01/15/06 01/15/06 01/15/06 01/15/06 01/15/06 01/15/06 01/15/06 01/15/06 01/15/06 01/15/06 01/15/06 Date completed
For this Completion Report to be valid, the learner listed above must be affiliated with a CITI participating institution. Falsified information and unauthorized use of the CITI course site is unethical, and may be considered scientific misconduct by your institution. Paul Braunschweiger Ph.D. Professor, University of Miami Director Office of Research Education CITI Course Coordinator
APPENDIX B CONSENT FORM
184 CONSENT FORM A MIXED METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGH SCHOOLS IN TEXAS I have been asked to participate in a research study. This study is intended to explore what certified English as a Second Language (ESL) teachers, non-certified ESL teachers who teach English Language Learners (ELLs), campus administrators , and district ESL personnel view as the impact that high stakes assessments have on ELLs, ESL curriculum and instruction , and what they observe as occurring. I was selected to be a possible participant because I am either a campus administrator, a teacher handling ELLs or a district ESL coordinator of the selected major urban high schools in Texas. A total of 118 people have been asked to participate in this study. The purpose of this study is to determine the views and opinions of campus administrators, teachers handling ELLs, and district ESL coordinators regarding the impact of statewide testing, specifically Texas Assessment of Knowledge and Skills (TAKS), on the curriculum and instruction of this special group of students. If I agree to be in this study, I will be asked to answer an on-line questionnaire. I may also form part of the focus group which will have group interviews verifying and validating my views and opinions regarding the open-ended questions. If I am a principal or a district ESL coordinator my interview will be one-on-one. The proceedings will be transcribed so that the researcher can go back to sections for better understanding. I may volunteer another knowledgeable respondent in order that the succeeding interviews will be more productive or meaningful. This study will only take at least two interviews after the online questionnaire has been answered. The risks associated with this study are almost non-existent. The benefits of participation are my contributions to the betterment of the ESL curriculum, instruction and testing. I will receive no compensation for my participation in this research study. A simple ‘thank you’ note will suffice. This study is confidential since the data will be dealt with only by the researcher and transcripts will be kept in a safe storage. The records of this study will be kept private. No identifiers linking me to the study will be included in any sort of report that might be published. Date _____________ Initial________________ Page 1 of 2
Research records will be stored securely and only the researcher, Arthur L. Petterway, will have access to the records. For the transcripts, only Arthur L. Petterway will also have access to the information contained therein. My decision whether or not to participate will not affect my current or future relations with Prairie View A & M University. If I decide to participate, I am free to refuse to answer any of the questions that may make me uncomfortable. I can withdraw at any time without my relations with the University, job, benefits, etc., being affected. I can contact Arthur L. Petterway at (713)748-8303 or email@example.com and Dr. Robert Marshall, at (936)857-4127 Marshall@pvamu.edu with any questions about this study. This research study has been reviewed by the Institutional Review BoardHuman Subjects in Research, Prairie View A & M University. For research- related problems or questions regarding subjects’ rights, I can contact the Institutional Review Board through Ms. Marcia C. Shelton, Research Compliance Officer, Anderson Hall Room 311, PO Box 4149, Prairie View, TX 77446, at 936.857.2541 and at firstname.lastname@example.org. I have read the above information. I have asked questions and have received answers to my satisfaction. I have been given a copy of this consent document for my records. By signing this document, I consent to participate in the study. Signature of the Subject: ______________________________ Date: __________ Signature of Investigator: ______________________________ Date: __________
Original – Researcher Copy – Participant
Date _____________ Initial________________ Page 2 of 2
APPENDIX C INTERVIEW QUESTIONS
INTERVIEW /FOCUS GROUP QUESTIONS Please help us serve English Languages Learners (ELLs) more effectively by taking a few moments to answer these interview questions. Confidentiality will be maintained throughout this process, as only I will have access to the data. Data and information will be kept in a safe home vault for a period of seven years, after which time they will be destroyed. Summarized data will be published in my dissertation. Your participation is greatly appreciated, as you will be making a significant contribution to the English as a Second Language (ESL) curriculum, instruction and testing. 1. Describe your experience with English Language Learners. 2. In your estimation, how many English Language Learners graduate each year from your school? 3. Do you think this number has increased or decreased in recent years? Why? 4. Describe your school’s support system for former English Language Learners. 5. What is your understanding of the Texas Comprehensive Assessment Plan as it relates to ELLs? 6. Should the English Language Learner be held to a separate standard of promotion or to the same standards as the regular population? Why?
188 7. With respect to your school, how do the promotion standards affect your English Language Learner population? What will it take to facilitate the English Language Learners’ success in meeting graduation standards? 8. What concerns need to be addressed before administrating TAKS to the English Language Learner? Who should be responsible in resolving these concerns? 9. Why should English Language Learners be held/or not held liable or accountable to the same standards or requirements? 10. What do you think are the intended consequences of the statewide accountability test (TAKS), specifically in terms of ESL curriculum and instruction? 11. Has the ESL program implemented in schools failed/or served its purpose or the state’s standardized assessment? 12.How would you rate the success of the ESL program in regards to standardized assessment? 13. To what extent do the consequences of a statewide assessment affect the ELL student, educator and district? 14. In what ways, if any, has the curriculum for ESL students changed as a result of their participation in the assessment? 15.What is your opinion of the cause of the low performing scores of ELL students?
189 16.What do you need to do to adequately prepare these students for success with standardized assessments and graduation standards? 17. What are your recommendations for the future testing of the English Language Learner? 18.Given that the criteria being used are high-stakes, what additional supports are needed to ensure that ELLs will be able to meet them? 19.How do alternative assessments (e.g., Spanish language exams) compare to mainstream assessments? 20.When is the use of native language assessments appropriate? 21.How does the placement of accommodations impact comparability with mainstream student performance? 22.Do wide-scale tests with the permitted accommodations fully assess English Language Learners’ knowledge and abilities or does the system need to be fully redesigned such that the needs of these students are addressed in the development of the assessments? 23.Do you think other data collection methods, such as portfolios or other performance assessments, would yield more accurate results with regard to ELLs than traditional assessments? 24.What sorts of information is needed to make fair high-stakes decisions about ELLs (e.g., grades, classroom performance, an array of samples of student work, teacher recommendations)?
190 25.What would be the most beneficial system(s) of accountability to ensure that these students are making progress in what they know and can do in important content areas? 26.What supports are necessary to aid states and districts in their alignment of assessments, standards, curricula, and instruction? TAKS = Texas Assessment of Knowledge and Skills
APPENDIX D ON-LINE/HARD COPY QUESTIONNAIRE
A MIXED METHODS ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGH SCHOOLS IN TEXAS Please help us serve English Languages Learners (ELLs) more effectively by taking a few moments to fill out this questionnaire. The results will be returned to us automatically via the web. Confidentiality will be maintained throughout this process, as only I will have access to the data. A random numeric code will be electronically generated and assigned to each qualitative questionnaire. Data and information will be kept in a safe home vault for a period of seven years, after which time they will be destroyed. Summarized data will be published in my dissertation. Your participation is greatly appreciated, as you will be making a significant contribution to the English as a Second Language (ESL) curriculum, instruction and testing. 1.) GENDER MALE FEMALE
2.) AGE 21-30 31-40 41-50 51-60 61+ 3.) CURRENT POSITION Principal Assistant Principal
ESL Certified Teacher NON-ESL Certified Teacher (who teach ELLs) District ESL Personnel 4.) HIGHEST DEGREE EARNED Bachelors Masters Doctorate 5.) What certifications do you hold?
6.) YEARS OF EXPERIENCE IN PUBLIC EDUCATION 1-5 6-10 11-20 21+ 7.) What are the anticipated results of the statewide testing, specifically, English Language Arts (ELA) and Mathematics portion of the Texas Assessment of Knowledge and Skills (TAKS), on English Language Learners?
8.) What are the actual results of the statewide testing, specifically, ELA and Mathematics portion of the Texas Assessment of Knowledge and Skills (TAKS), on English Language Learners?
9.) Why is the Texas Assessment of Knowledge and Skills (TAKS) given as a statewide test?
10.) What are the intended consequences of this statewide testing as it relates to English Language Learners?
11.) What has happened because of TAKS as it relates to English Language Learners?
12.) What problems have occurred, if any, to the English Language Learners because of TAKS?
13.) What changes have occurred as a result of statewide testing as it relates to English Language Learners?
14.) What general recommendations would you suggest to improve the overall performance of English Language Learners on statewide testing ?
15.) Based on your recommendations above, which will be of greatest value for ELLs success on statewide testing?
Thank you for taking the time to fill out this questionnaire. If you need to contact us - you can click on the following email email@example.com
Submit Form Reset Form
APPENDIX E LETTER TO PARTICIPANTS
197 LETTER TO PARTICIPANTS Arthur L. Petterway 5300 N. Braeswood Blvd. # 247 Houston, Texas 77096 05/12/2006
Dear Sir or Madam, I am presently in the Ph.D. in Education Leadership program at Prairie View A & M University. I am currently conducting a dissertation research on high-stakes testing and English language learners. My dissertation topic is “A Mixed Methods Analysis of the Impact of HighStakes Testing on English Language Learners in Major Urban High Schools in Texas”. I realize how busy you are in meeting the challenges of your work but I hope that you will take time to complete an on-line qualitative questionnaire that I have prepared to gather research data. You can access the qualitative questionnaire at firstname.lastname@example.org. The questionnaire will include a major question on what are the anticipated and observed consequences of the statewide testing, specifically, Texas Assessments of Knowledge an Skills (TAKS), on English as a Second Language (ESL) curriculum and instruction as viewed by certified ESL teachers, non-certified ESL teachers who teach English Language Learners (ELLs), school administrators, and district ESL personnel. Confidentiality will be maintained throughout this process, as only I will have access to the data. A random numeric code will be electronically generated and assigned to each qualitative questionnaire. Data and information will be kept in a safe home vault for a period of seven years, after which time they will be destroyed. Summarized data will be published in my dissertation. Your participation is greatly appreciated, as you will be making a significant contribution to the ESL curriculum, instruction and testing. Should you have any questions or comments, please feel free to contact me at (713)748-8303 or email@example.com. You may Contact my dissertation chair, Dr. Robert Marshall, at (936)857-4127 or Marshall@pvamu.edu. Once again, I appreciate your attention to this matter and look forward to your favorable response. Sincerely, Arthur L. Petterway Ph.D. student – Prairie View A & M University
APPENDIX F REQUEST FOR EXTANT DATA FROM TEXAS EDUCATION AGENCY (TEA)
HOUSTON INDEPENDENT SCHOOL DISTRICT
1700 DUMBLE HOUSTON, TEXAS 77023-3195 PH: (713) 924-1600 FAX: (713) 924-1619
STEPHEN F. AUSTIN SENIOR HIGH SCHOOL
ARTHUR L. PETTERWAY Assistant Principal
Dear Mr. Rod Rowell, As per our telephone conversation on July 3, 2006, I am submitting my request for extant data to TEA.
Requestor Name: Arthur L. Petterway Company Name: PhD Doctoral Student at Prairie View A&M University Address: 5300 N. Braeswood Blvd. #247 City/State/Zip: Houston, Texas 77096 Telephone: Office-(713)924-1600; Home-(713)748-8303; Cell-(832)693-2809 Fax Number: (713)924-1619 Requestor Email Address: firstname.lastname@example.org Brief Summary of Request: I am currently working on my dissertation, A Mixed Methods Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High schools in Texas, at Prairie View A&M University and need your help. This is a Public Information Request to please provide me with the 2002-2003; 2003-2004; 2004-2005; 2005-2006 TAKS Summary Report-(ALL) for 10th grade English Language Arts and Mathematics of all of the high schools in the following school districts:
015907 015910 015915 057905 071902 071905 101912 220901 220905 227901 SAN ANTONIO ISD NORTH EAST ISD NORTHSIDE ISD DALLAS ISD EL PASO ISD YSLETA ISD HOUSTON ISD ARLINGTON ISD FORT WORTH ISD AUSTIN ISD.
Your immediate attention to this matter will be greatly appreciated. If you have any questions, please don't hesitate to let me know. Sincerely, Arthur L. Petterway Assistant Principal
Home of the Mighty Mustangs
From: Woli, Urbe [mailto:Urbe.Woli@tea.state.tx.us] Sent: Tue 7/11/2006 3:23 PM To: Petterway, Arthur L Cc: PIR; Eaton, Jennifer; Woli, Urbe Subject: PIR # 6541 - Grade 10 Campus TAKS Summary Data
Mr. Arthur Petterway,
The 2003 – 2005 TAKS campus summary data you are requesting (see attached PDF document) are available for download at no cost from our website at http://www.tea.state.tx.us/student.assessment/reporting/taksagg/index.html. The 2006 TAKS campus summary data will be posted at the same location by the end of August, 2006. Due to the large size of the downloadable data files, you will need to use SAS or SPSS to process the data files. Please let us know if you prefer that we generate the data for you. To enable us provide accurate data; please list all the statistics you need in the data. Contact me at (512) 4639536 to discuss your data needs. Thank you. Urbe Woli Student Assessment Division Texas Education Agency Tel: (512) 463-9536
VITA ARTHUR L. PETTERWAY 5300 N. Braeswood Blvd, #247 Houston, Texas 77096 EDUCATIONAL HISTORY Prairie View A&M University, Prairie View, Texas, Ph. D. in Education Leadership, Expected Graduation Date August, 2007 Prairie View A&M University, Prairie View, Texas, M.Ed. in Education Administration, August, 1999 Dillard University, New Orleans, Louisiana B.A. in Biology, June, 1969 CERTIFICATIONS Administrator - Principal Teacher - Elementary Mathematics (Grades 1-8) Teacher - Elementary Self-Contained (Grades 1-8) EMPLOYMENT HISTORY 2002 - 2007 1996 - 2002 1994 - 1996 1991 - 2006 1989 - 1991 1978 - 1989 Assistant Principal, Houston ISD Math Department Chair, Houston ISD Middle School Math Teacher, Houston ISD Upward Bound Coordinator, UH/DT President/CEO, Way Refining, Houston Executive Vice President, Manufacturing, Houston 1969 - 1978 Manufacturing Supervisor, General Motors, Dayton, Ohio