ANALYSIS

Quantitative approaches to assessing classrooms have the ability to be more systematic and can
broadly be broken down into two major categories: checklists and rating/categorization scales.
Checklists typically require the classroom observer to mark the presence or absence of the item in
the classroom. They are very low inference methods and can catalog a range of desired constructs
from artifacts in the classroom to the practices of the teacher. Rating or categorization scales are
often higher inference methods, capable of focusing on the quality of specific behaviors as well as
on their frequency of occurrence in the classroom (Waxman et al., 2004). As assessments of
quality, rating or categorization scales have much greater potential to be fed back to teachers to
change their practices.

Instruments to assess teacher practices and classroom processes used in low- and middle-income
countries (LMICs) have primarily focused on measuring student use of their time in the classroom,
primarily due to the framework put forth by the Global Campaign for Education (2002) that set the
learner in the center of all education quality endeavors. In the past, however, instruments such as
the Stallings (Stallings, 1978) were based on Carroll’s (1963) model of school-based learning that
focused on the importance of a learner’s time engaged in learning as well as his or her learning
rate. Though this modality could still provide a general picture of classroom alignment with policy
and expectations, understanding specific behaviors run the risk of being underreported when using
a snapshot method unless intervals are quite frequent (UNESCO, 2016).

At one time, time on task was the prevailing method behind classroom observation, but more
recent findings question this mode of measurement. Even as they support it, Benavot and Gad
(2004: 293) note that, “researchers disagree over the magnitude of this [time engaged in learning
and learning rate] relationship, the relative importance of various intervening factors, and the
nature of the socio-economic contexts in which the relationship is more or less salient.” Therefore,
though defined as systematic observation instruments, time and frequency measurement
instruments, such as the Stallings Snapshot, are far too narrow in scope to meet the need of
measuring the quality use of classroom time.1 Furthermore, time on task measures cannot capture
teacher competencies around “soft skills” such as emotional support nor can they differentiate
levels (i.e. quality) of instruction (Bruns et al., 2016).

Additionally, many internationally used instruments are intervention-specific. The main drawback to
tailoring measures to evaluate the specific facets of an intervention is that the measure becomes
insensitive to experimental contrast, and use and comparison of data in other evaluations and
contexts are not possible. Only a few instruments – primarily the various iterations of the Stallings
Observation System (SOS) – have been used across different contexts, cultures, and interventions
(Bruns, 2011; Crouch, 2008). The issue is further compounded by the fact that most instruments in
use internationally do not provide a clear conceptual framework that serves as a foundation for
instrument validity. The development of a high-quality instrument for LMICs would require further
psychometric development. Furthermore, while professional development and pedagogy need to
be based on research and existing best practices, it is also of the highest importance that the
research that informs it continues to be localized, adapted, and refined to the day-to-day realities
of the teacher’s context (Burns and Lawrie, 2015; Vavrus, 2002).