You are on page 1of 12


Transforming Education and Supporting Equity Through Opportunity to Learn Standards
Blake Chism
Doctoral Student San Diego State University and Claremont Graduate University

Valerie Ooka Pang, PhD
Professor San Diego State University ______________________________________________________________________________ Abstract Opportunity to Learn (OTL) standards originated as a way to compare mathematics assessment data from different classrooms in a multi-national study. Over the years, OTL has surfaced in various emanations such as a research construct, equity assessment framework, and finally, as a policy orientation. The OTL literature makes the case for its prominence in an effective standards-based accountability system. Through an extensive literature review, Opportunity to Learn standards included seven core areas: Teacher Quality, School Quality, District Quality, Content Delivery, Curriculum, Facilities, and Resources. We believe OTL standards provide a comprehensive assessment for transforming education by ensuring that all students are provided with equity in education; students must have the opportunity to learn the content and skills which schools are held accountable. OTL emphasizes not only educational inputs and student performance outputs, but also the importance of student access to learning. Keywords: opportunity to learn; equity; student access to learning; core standards ______________________________________________________________________________

Opportunity to Learn (OTL) standards is a framework that has developed from the work of policy makers, educators, and researchers who are concerned about the ability of students to have access to subject area content and high quality delivery. Though much of the accountability research in education has focused on student assessment outcomes, OTL presents a more comprehensive frame of reference in which students are provided with the opportunity to learn the content on which they are held accountable in schools. The purpose of this article is to describe the seven core areas of Opportunity To Learn (OTL) standards and to describe how OTL can assist educators in comprehensively examining the opportunities that students have to achieve and to make sure that equity is a core value in schools. We believe that educational equity is dependent upon the ability of districts, schools, and teachers to provide each component. OTL emphasizes not only educational inputs and student performance outputs, but also the importance of student access to learning.


BLAKE CHISM and VALERIE OOKA PANG Many educational policy makers believe that OTL standards more fully address equity in education in comparison to solely utilizing student outcomes (Murphy, 1988; Porter, 1995; Wang, 1998). Through an extensive literature review, Opportunity To Learn standards included seven core areas: Teacher Quality (TQ), School Quality (SQ), District Quality (DQ), Content Delivery (CD), Curriculum (CUR), Facilities (FAC), and Resources (RES). These core elements represent potential benchmarking areas by which school systems can be measured to make certain that students are provided with the opportunities to learn the content and which schools are held accountable. In order to assess student learning, we believe it is vital that educational policy makers, administrators, and educators examine the seven areas. There are as follows:  Teacher Quality is key to student learning because the teacher is the most important component in the classroom. The teacher must be competent in disciplines being taught; decides on and implements effective instructional strategies; uses motivational methods; has effective classroom management skills; attends professional development training; and holds high expectations for students and him/herself. School Quality refers to the cohesive nature of the school and whether there are carefully communicated goals and effective leadership in the building. Elements of schools such as class size, school climate, and diverse staffing are in place. The faculty and staff are professionals and have high academic standards for student learning as a collective organization. District Quality refers to the ability of the organization to provide a comprehensive system where effective achievement is at its core and includes elements such as the recruitment and retention of high quality educators, fair compensation for educators, mandated regular review and adoption of textbooks, required instructional time, common curriculum standards across the district, and additional programs such as Special Education and English as a second language to meet the needs of culturally and linguistically diverse students. Content Delivery refers to the ability of teachers to engage students in learning. This area is highly integrated with the opportunity of students to learn. Instructional strategies, interventions, and lesson planning that teachers choose will determine if students have access to and are able to learn the content being presented. Curriculum should meet federal and state standards; the curriculum must be rigorous and cover depth of content, free from bias, and aligned with district /school curriculum standards. Facilities provide for a variety of elements in schools such as the implementation of enough physical space for student learning, the heating and cooling systems in buildings are effective, proper lighting is present, a safe environment is furnished, and recreational and library facilities are provided.


BLAKE CHISM and VALERIE OOKA PANG  Resources refers to materials and services provided to students such as access to computer laboratories, internet access, up-to-date textbooks, science lab equipment, physical education apparatus, and consumable materials like art and office supplies.

Historical Background of the Opportunity To Learn Standards Opportunity to Learn as a construct first developed during the 1960s when researchers for the International Association for the Evaluation of Educational Achievement (IEA) created a way to validate cross-national studies of mathematics students. Researchers looked for a measure that would evaluate whether a student had learned specific content (Burstein, 1993). Specifically, they believed in the importance of measuring if the content for each item was being taught by the classroom teacher. OTL was introduced as a piece of the First International Mathematics Survey, and was further refined under the Second International Mathematics Survey (SIMS) between 1976 and 1982. The SIMS OTL used survey data to measure three main areas: the degree of similarity between intended and implemented coverage, documented differences in curriculum across national systems, and opportunity to learn within the same country (McDonnell, 1995). The data were collected primarily by asking teachers to examine each item on the test, determine whether they believed that they had taught its content to their students, and report their conclusions on a survey. Though tedious to analyze, this data then allowed researchers to make an accurate comparison of classrooms in different countries based on whether or not the content students were tested on was covered. This became the conceptual foundation for OTL standards. The difference between the intended and implemented content coverage in this study was measured by having systems-level analyst determine what content was contained in the curriculum, and then surveying teachers to see whether they had successfully delivered that curriculum. The distinction is important because both pieces; the curricular content and the delivery may have a considerable impact on student learning. If the curriculum does not contain the content, then students cannot be held responsible for learning it, and if the content is being taught, then teachers should be held responsible for effectively delivering the curriculum to students in a way that provides them the opportunity to learn the subject area content. The curricula study among nations provided details about how math is taught in different countries (McDonnell, 1995). This research showed that the Japanese curriculum had a major emphasis on algebra, and that the US curriculum tended to emphasize a broad range of mathematics, but covered much of the content in a shallow fashion. The study also measured the way that the content was covered across national systems. Some systems, such as the Japanese system, were fairly consistent from classroom to classroom with a median of 85% coverage, whereas the American system had a range of 0% coverage to 100%, with a median at 75% (Schmidt, Wolfe, & Kilfer, 1993). McDonnell (1995) found that the Second International Mathematics Survey (SIMS) OTL data collected, combined with research on the relationship between student achievement and curriculum exposure support a mechanism that improves student achievement and also provides more equitable learning opportunities to learners. The SIMS use of OTL is similar to establishing an exchange rate across two different currencies. Once their value is established against a set of standards, they can be compared to each other in a more meaningful way. OTL was not used to find out if one classroom achieved higher than another, rather it determined if the two classrooms were teaching the items on the test, so that data from the study could be properly analyzed. 21

BLAKE CHISM and VALERIE OOKA PANG No Child Left Behind Legislation: Moving to Outcome Measures The release of “A Nation at Risk” (ANAR) in 1983 by the National Commission on Excellence in Education began the movement away from input-based systems of measuring educational achievement, to a standards and outcome-based model. The assumptions of ANAR pointed to the ineffective educational structure in the United States (US); the workforce was not well trained in content areas like science and as a nation we lost ground rapidly and fell below the abilities of individuals from other countries around the world. It was reported that President Reagan originally intended to abolish the Department of Education, but the release of ANAR increased awareness of the issue of education and moved people to action (Bell, 1987). ANAR helped start the movement toward measuring educational outcomes based on standards. Measuring achievement outcomes against a set of standards, means that schools know what is expected of them and what their students are expected to achieve. The standards-based accountability movement has grown steadily since the early 1980s. The reauthorization of the Elementary and Secondary Education Act (ESEA) called The No Child Left Behind Act of 2001 (NCLB) was signed into law in 2002 under the George W. Bush administration. The legislation mandated that states create standards of achievement for students and schools, and required that they levy penalties for underperformance. The law set achievement benchmarks, but did not specifically stipulate what should be taught, how the content is delivered, what amount of time should be spent on the content, how or by whom it is delivered, nor under what physical conditions content is received. The law punished schools and students for not achieving the goals, but did not take into account whether or not students are provided with the opportunity to learn the required content or achieve the required goals. In contrast, Opportunity to Learn (OTL) standards allow educators, researchers, and policymakers the ability to measure whether or not students are provided with the opportunity to learn the content on which they are being tested. These standards also measure whether or not students have access to quality instructional materials. When used in conjunction with the existing accountability system, OTL can provide key assessments of how the system and schools are doing, and how best to adjust them for improved student achievement.

OTL: A Comprehensive Model of School Processes The collection of statistical data on schooling largely focused on inputs such as per-pupil spending and, to a lesser degree, outputs such as test scores. The late 1980s saw researchers and policymakers move toward a system that looked at comparing schools and schooling conditions across geographic areas (McDonnell, 1995). During this time, the OTL construct became known by a number of different names such as: school process indicators, performance indicators, school delivery standards, and education indicators (Odden, 1990; Ogawa & Collom, 2000; Porter, 1991, 1993 not in references). School performance indicators, such as the Academic Performance Indicator (API) in California, measured school performance based on a number of different areas, such as testing, attendance, graduation rates, language assessment, and so on. Though not necessarily opportunity to learn in spirit, these performance indicators are included here due to the fact that they provide a valuable assessment complement to the OTL framework included below in Figure 1. Education indicators that measure outcomes instead of opportunity, such as the API, should be used as assessment of the outcomes resulting from the inputs and 22

BLAKE CHISM and VALERIE OOKA PANG processes. On their own, they do not measure the school system accurately enough to be considered an OTL standard. Odden (1990) described “education indicators” as measures of educational inputs, processes, and outputs. These included measuring the graduation rates, SAT and ACT scores, per pupil expenditures, teacher quality, level of adequacy of facilities, classroom discipline, and others. The measure of success in education was not only what went into the educational process, or solely what came out, but everything that happened during the schooling process was recorded. Odden’s use of education indicators is comprehensive in its approach when looking at school data. Porter draws the term “school delivery standards” from a report that came out in 1992 by the National Council on Education Standards and Testing (NCEST) entitled Raising Standards for American Education.(no citation or reference) School delivery standards (SDS) placed more focus on schools and the delivery of the curriculum by the teacher. SDS is interesting because in a time when the focus was quickly moving towards output-based accountability, it focused more on input. SDS seems to be a part of the accountability movement of the 1970s when inputs reigned supreme. School delivery standards specified how the curriculum was delivered to students. SDS was a narrow construct, but somewhat aligned with OTL standards. School process indicators (SPI), as described by Porter (1991) focused on the context of schooling rather than inputs and outputs that are traditionally measured. These indicators helped researchers, policymakers, and educators observe what happens in schools. When the outputs are not what is desired by schools, SPI provided feedback on how to correct the problem. In the black box scenario, there are known inputs going into the box, unknown processes within the box, and known outcomes. SPI inspected the black box to determine not only the quality of teaching and curriculum, but also the quality of the school in general, and the opportunities it provides to students. The curriculum and teaching qualities are also main foci of SPI. An adaptation of Porter's model can be found in Figure 1. This new conceptual model shows how the inputs, processes, and outputs can all be observed via use of school process indicators.



Figure 1. Opportunity to learn framework: Educational inputs, processes and outputs. The model takes into account teacher quality, student background, fiscal resources as the inputs, and measures the achievement, participation, and attitudes as outcomes. The primary focus, however, is the processes of schooling. What is the quality of instruction? Are the teachers qualified to teach their subjects? How do the federal, state, and local systems contribute to the quality of education? These processes are the operations within the “black box” and they provide a detailed analysis of a student’s true opportunity to learn. This type of assessment in the form of school process indicators and education indicators was a powerful research tool. It allowed researchers and practitioners to look at all levels of the education process and continually adjust to achieve desired outcomes. Because of the potential of OTL or school process indicators to make positive reforms in education, it was not long before they moved into equity assessment and finally, into the political arena.

OTL: Educational Equity and Access of Students of Color OTL was becoming burdened with education policy by the mid-1990's while the concept of school equity in finance was in full swing. Equity consideration prior to this time focused 24

BLAKE CHISM and VALERIE OOKA PANG solely on input equity, or making sure that resources were equitably distributed among students. The discussion eventually moved from equity inputs, to student access to learning as equity. This was an important step, as Murphy (1988) described, because the allocation of aggregated resources was proven less than effective in producing the outcomes desired. Educational equity assessments shined a light on the unequal distribution of educational resources and processes, such as content, time, teacher quality, and homework (Murphy, 1988). Equitable distribution of resources was never achieved on a large scale, and it remains an unachieved goal to this day. What began to emerge in educational research during this period was the fact that there was still a gap in the educational process as far as delivering the curriculum, teacher quality, and school quality. Wang (1998) examined relevant literature and surmised that many students of color were more likely to be placed in classrooms where they have much less opportunity, even when their ability is considered. Educational research recognized that equity in resource allocation did not translate into desired outputs if the processes were still less than equitable. Baratz-Snowden (1993) wrote that, “if students are to be held accountable for their learning, then schools must be held accountable as well by demonstrating that they provide students with opportunities to learn what they must learn to meet the standards that have been set” (p. 317). This common theme in OTL literature reflects the evolution of OTL standards from only school processes as in earlier years, to the equitable application of those processes. OTL standards for equity assessment promised to identify gaps in the educational process which are needed to provide equitable opportunities for all students. Porter (1995) wrote that OTL advocates see the standards as “an appropriate antidote to the potentially negative effects of high-stakes testing on students who, through no fault of their own, attend schools that provide an inferior education” (p. 21). Because the accountability movement was beginning to gain momentum, and the standards were measured by tests, Porter’s insistence on OTL standards to create and maintain equity for the process seemed to be a foregone conclusion. If all students are required to achieve at certain levels, then there should be a way of ensuring that they are all given equal opportunity to learn the content tested. The research began to show that students in lower ability groups had far fewer opportunities to learn. These students are often provided with less experienced teachers, inferior curriculum, and a chaotic learning environment (Murphy, 1988). Guiton and Oakes (1995) delivered a concise assessment of OTL as an educational equity assessment tool: Moreover, the proposed accountability uses of outcome indicators to administer sanctions and incentives based on individual student and school performance presume an education system that provides adequate human and material resources to ensure a fair opportunity for students to achieve those outcomes and that schools’ efforts predominantly account for student performance. (p. 324) The assumption underlying outcome-based systems is that all things are equal, and that students alone are to be held accountable for what they can achieve regardless of the opportunities presented to them. The research on OTL as an equity assessment proved that there was an enormous gap between the opportunities provided to students at the top and those provided to the bottom. Another key point in the equity assessment field was making sure that schools and teachers were applying resources and skills adequately to provide opportunities to learn. Darling25

BLAKE CHISM and VALERIE OOKA PANG Hammond wrote that OTL can help to determine if schools have the resources needed and are using them appropriately (1994). Unfortunately for students, it was blatantly clear in many cases that the resources were not in place to allow schools to properly provide the needed level of instruction. A good example of this was seen in the state of California; it was eventually settled in court. In the California lawsuit of Williams v. California, the plaintiffs argued that the state did not provide the poorest of its students with proper textbooks, qualified teachers, facilities, and resources (Oakes, 2004; Oakes & Lipton, 2004). The case found that 45,000 teachers were working in California schools without full preparation, and that some 6 million students were being taught by under qualified teachers. Textbooks were either in short supply or of poor quality in almost twice as many of the lower income schools as in upper income schools. A 1996 General Accounting Office (GAO) study found that 42% of California schools had at least one building in disrepair. The irony of the case is that it was determined that California was not providing the poorest of its students the opportunities they needed to learn, but no OTL standards were required to prevent the same issue from occurring in the future. Perhaps this was because OTL standards had already run their course through public policy, and met their political end. The case was similar to Serrano v. Priest in 1976 that argued that California’s school finance system was unconstitutional because it violated the equal protection clause. In the same way, the Williams settlement provides standards for what each student should have in order to have the opportunity to learn.

OTL As A Policy Instrument Federal government data collection historically measured two characteristics; raw test scores as outputs and inputs such as the amount of money spent per student. As mentioned above, research began not only to look at the input and output measures, but also at what happened in between. OTL expanded from the SIMS arena into what became known as classroom or education indicators. The indicators included measures of content delivery standards, course offerings, teacher training and experience, availability, and amount of instructional materials. The 1992 publication of the National Council on Education Standards and Testing (NCEST) report entitled, Raising Standards for American Education, (No citation) helped move OTL to a policy tool. This report changed the name from Opportunity to Learn standards to “school delivery standards,” but the underlying intent was the same. An interesting dynamic arose in this report that ultimately proved to be the back breaker for OTL. As the report was being written, members of NCEST were forced to reach a compromise on whether OTL or school delivery standards should be standardized at the federal level or designed at the local level. The question of whether education reform strategies should build from the local level up or be handed down from the top is not new to education; it is one of the long-standing debates in education and policy in general. What occurred next proved to be detrimental to the evolution process of OTL and signaled the demise of the construct as far as policy implications go. When the No Child Left Behind act (NCLB) was signed into law in 2002, the legislation provided little in the way of OTL and moved much of the burden of achievement onto students. In fact, one phrase seemed to blatantly try to avoid being confused with OTL by stating that students should be able to “achieve” the knowledge and skills thus, NCLB’s wording placed 26

BLAKE CHISM and VALERIE OOKA PANG much more of the accountability of learning on students. (need citation with page numbers and reference) Several sections referring to migratory students and other students who might be considered “at risk” explain that states should provide students with the opportunity to “meet” the strict content standards, not opportunity to learn. In NCLB, the onus to meet the standards is squarely placed on the student’s shoulders as is the burden of learning as stated above. Dougherty (1996) pointed out that OTL standards do just the opposite by taking the responsibility for achievement from the student, and requiring the school to demonstrate that the student has the opportunity to learn what is required of them. Another interesting aspect of NCLB is that the phrase “opportunity for a hearing” is used almost as many times as “opportunity to meet content standards.” The insertion of litigious language into the reauthorization opened the door for educational systems to be corrected in court. The polarizing nature of the political arena took the construct of Opportunity to Learn Standards from a highly functional tool to obsolescence in a few short years. In the intervening years since Goals 2000, the only mention of OTL in literature, educational policy or otherwise, has been in sparse patches of educational research. (need citation-page numbers for al direct quotes)

OTL: Recent Research Recent studies have identified specific OTL “opportunity areas.” For example, Boscardin et al. (2005) measured OTL in terms of three main areas: teacher experience and expertise, content/subject area, and student background. Venezia and Maxwell-Jolly (2007) also identified three similar approaches: content of instruction, the pedagogical process, and instructional resources such as curricular materials, technology, facilities, and use of instructional time. Other studies concluded that curricular materials, textbooks, technology, and equipment vary greatly among schools, and are more likely to be used in the wealthier neighborhoods (Oakes & Lipton, 2004; Oakes & Saunders, 2004). OTL was divided into two main camps. The first one focuses on classroom-level indicators, and is measured using teacher surveys, logs, interviews, and observations. The second camp is focused on school and classroom resources such as textbooks, instructional materials, and traditional inputs. Gearhart et al.(1999), measured students’ OTL in the classroom and examined the extent to which students were provided the opportunity to learn the information taught in the classroom. Gillies and Quijada (2008) measured OTL in terms of total instructional time, hours in a school year, days school is open, teacher attendance and punctuality, student attendance and punctuality, teacher-student ratio, instructional materials per student, time in classroom spent on task, and reading skills taught by grade. Relatively unique among OTL research, which is largely theoretical in nature, the authors created concrete OTL standards such as 850-1000 hours of schooling per year, whether or not (a) there were schools within 1 km of students’ homes, (b) the teacher was present every day, (c) the student was present every day, and (d) there were sufficient instructional materials available every day. They also created an “opportunity index” that measured these inputs in order to produce a rating. The process was used in developing countries to demonstrate the lack of opportunity to learn in those countries. This method could


BLAKE CHISM and VALERIE OOKA PANG easily be expanded and implemented as part of a statewide assessment and accountability system. Stevens (1993) defined her conceptual framework for OTL in terms of content coverage, time on task, emphasis variables, and quality of instructional delivery variables. Her framework focused entirely on variables within the classroom. Additional OTL studies included the importance of content instruction (Porter, 2002), access to information through guidance counselors as OTL (Cooper & Liou, 2007), content exposure and coverage as well as instructional conditions, practices and processes (Muthen et al., 1995). The common denominator of much of the OTL research is that each framework or conceptual model requires that many different data points be continually gathered, assessed, and adjusted in order to provide students with the opportunities they need. The current system of accountability relies almost solely on test scores, or student out-puts and focuses little on analyzing the inputs leading up to student performance measures. It is impossible to change what educators do not measure.

Conclusion Opportunity to Learn presents a holistic framework for educational leaders, policy makers, and teachers to consider as they work towards equitable education for all students. It is vital that student test performances are examined within a comprehensive context where various components such as teachers, schools, districts, content delivery, curriculum, facilities, and resources are considered, otherwise the dominant focus becomes the learner. Research has found that many students from underrepresented communities have not been given the opportunity to learn (Pang, 2010), though they are being held accountable for subject area content (Wang, 1998; Oakes, 2004; Oakes & Lipton, 2004). Equity in education points to the importance of ensuring that each student is provided with what they need in order to access high level content through effective instructional delivery. We believe OTL standards are an important avenue that leaders in education must include in their assessment process. The use of OTL is an equity issue. Without using a comprehensive organizational process of assessment, equity in education will not be provided to students, especially those who are members of marginalized communities. Opportunity to Learn standards provides a wealth of possibilities for further research. Key recommendations include continued study of the core areas in order to define their most salient components. Additionally, one of the most important areas of research is the development of a comprehensive framework for OTL where the creation and validation of assessment tools for measuring the effects of OTL on students opportunities to learn are included. Scholars can also investigate school districts and individual schools developing case studies where OTL standards are used in a process of continual improvement. These case studies will demonstrate the strengths and weaknesses of the Opportunity To Learn framework and suggest revisions so as to more fully provide for equity in education for all students.

References Baratz-Snowden, J. C. (1993). Opportunity to learn: Implications for professional development. The Journal of Negro Education, 62(3), 311-323. 28

BLAKE CHISM and VALERIE OOKA PANG Bell, T. H. (1987). The thirteenth man: A Regan cabinet memoir. New York, NY: Free Press. Boscardin, C. K., Aguirre-Munoz, Z., Stoker, G., Kim, J., Kim, M., & Lee, J. (2005). Relationship between opportunity to learn and student performance on English and algebra assessments. Educational Assessment, 10(4), 307-332. Burstein, L. (1993). Studying learning, growth, and instruction cross-nationally: Lessons learned about why and why not engage in cross-national studies (Prologue). In L Burstein (Ed.), The IEA study of mathematics III: Student growth and classroom processes (pp.27-49). New York, NY: Pergamon. Cooper, R., & Liou, D. (2007, October/November). The structure and culture of information pathways: Rethinking opportunity to learn in urban high schools during the ninth grade transition. The High School Journal, 91(1), 43-56. Darling-Hammond, L. (1994). "National standards and assessments: Will they improve education.?" American Journal of Education, 102(4), 478-510. Gearhart, M., Saxe, G. B., Seltzer, M., Schlackman, J., Ching, C. C., Nasir, N., … Sloan, T. F. (1999, May). Opportunities to learn fractions in elementary mathematics classrooms. Journal for Research in Mathematics Education, 30(3), 286-315. Gillies, J., & Quijada, J. (2008). Opportunity to learn: A high impact strategy for improving educational outcomes in developing countries. Working Paper for Educational Quality Improvement Program (EQUIP). Retrieved from Guiton, G., & Oakes, J. (1995). Opportunity to learn and conceptions of educational equality. Educational Evaluation and Policy Analysis, 17(3), 323-336. McDonnell, L. (1995). Opportunity to learn as research concept and a policy instrument. Educational Evaluation and Policy Analysis, 17(3), 305-322. Murphy, J. (1988). Equity as student opportunity to learn. Theory into Practice, 27(2), 145-151. Muthen, B. O., Huang, L., Jo, B., Khoo, S., Goff, G., Novak, J., & Shih, J. (1995). Opportunity to learn effects on achievement: Analytical aspects. Educational Evaluation and Policy Analysis, 17(3), 371-403. Oakes, J. (2004). Investigating the claims in Williams v. State of California: An unconstitutional denial of education’s basic tools? Teachers College Record, 106(10), 1889-1906. Oakes, J. & Lipton, M. (2004). “Schools that shock the conscience:” Williams v. California and the Struggle for Education on Equal Terms Fifty Years After Brown, in Symposium, Rekindling the spirit of Brown V. Board of Education, 6 Afr.-Am. L. & Pol’Y rep. 152; 11 Asian L.J. 234; 15 Berkeley La Raza L.J. 25; 19 Berkeley women’s L.J. 353; Calif. L. Rev. (2004) Check for APA formatting-not sure of all your information. Is this a secondary source? Oakes, J., & Saunders, M. (2004). Education’s most basic tools: Access to textbooks and instructional materials in California’s public schools. Teachers College Record, 106(10), 1967-1988. Odden, A. (1990). Educational indicators in the United States: The need for analysis. Educational Researcher, 19(5), 24-29. Ogawa, T., & Collom, E. (2000). Using performance indications to hold schools accountable: implicit assumptions and inherent tensions. Peabody Journal of Education, 75(4), 200215. Pang, V. O. (2010). Multicultural education: A caring-centered, reflective approach. San Diego, CA: Montezuma Publishing.


BLAKE CHISM and VALERIE OOKA PANG Porter, A. C. (1991). Creating a system of school process indicators. Educational Evaluation and Policy Analysis, 13(1), 13-29. Porter, A. C. (1995). The uses and misuses of opportunity to learn standards. Educational Researcher, 24(1), 21-27. Porter, A. C. (2002). Measuring the content of instruction: Uses in research and practice. Educational Researcher, 31(7), 3-14. Schmidt, W. H., Wolfe, R. G., & Kilfer, E. (1993). The identification and description of student growth in mathematics achievement. In L. Burstein (Ed.), The IEA study of mathematics III: Student growth and classroom processes (pp. 59-75). New York, NY: Pergamon. Shavelson, R., McDonnell, L., Oakes, J., Caren, N. , & Picus, L. (1987). Indicator systems for monitoring mathematics and science education. Santa Monica, CA: RAND Corporation. Stevens, F. (1993). Applying an opportunity to learn conceptual framework to the investigation of the effects of teaching practices via secondary analyses of multiple case study summary data. Journal of Negro Education, 62(3), 232-248. Venezia, A., & Maxwell-Jolly, J. (2007). The unequal opportunity to learn in California’s schools: Crafting standards to track quality. Policy Analysis for California Education (PACE), Working Paper 07-2. University of California and Stanford University. Persistent link: OR Retrieved from 510177 [Education Resources Information Center. ERIC database ED510177] Wang, J. (1998). Opportunity to learn: The impacts and policy Implications. Educational Evaluation and Policy Analysis, 20(3), 137-156.

Authors Blake M. Chism is a doctoral student in the joint Ph.D. program in Education with San Diego State University and Claremont Graduate University. His research interests include School Finance, Educational Assessment, and Educational Technology. He currently works as an educational consultant for Dell Services focusing on technology integration in K-12 and Higher Education.

Valerie Ooka Pang is a professor in the School of Teacher Education at San Diego State University. She has published several books such as Multicultural Education: A Caringcentered, Reflective Approach (2010) and Struggling To Be Heard: The Unmet Needs of Asian American Children with L. Cheng (1998). In addition, Pang has published in various journals including Educational Researcher, Harvard Educational Review, The Kappan, The Journal of Teacher Education, Action in Teacher Education, and Social Education. Pang has consulted with organizations such as Sesame Street, Family Communications (Producers of Mr. Roger’s Neighborhood), and Scott Foresman. Pang was a senior Fellow at the Annenberg Institute at Brown University and honored by organizations such AERA, San Diego State, and the University of Washington’s College of Education.