Teaching and Teacher Education 41 (2014) 22e33

Contents lists available at ScienceDirect

Teaching and Teacher Education

journal homepage: www.elsevier.com/locate/tate

Using generic and content-specific teaching practices in teacher

evaluation: An exploratory study of teachers’ perceptions
Charalambos Y. Charalambous*, Andreas Komitis, Maria Papacharalambous,
Afroditi Stefanou
Department of Education, University of Cyprus, Cyprus

h i g h l i g h t s

Generic and content-specific practices are equally endorsed for teacher evaluation.

Unlike scholars, teachers do not rate these practices as distinctly important.

Teachers consider these practices more important than feasible to implement.

Teacher-centered practices are deemed more important and feasible to implement.

Student-centered practices are considered less important and feasible to implement.

a r t i c l e i n f o a b s t r a c t

Article history: Generic and content-specific teaching practices have largely been employed in parallel in teacher eval-
Received 19 August 2013 uation and classroom observation systems. Giving voice to teachers, we examine whether both types of
Received in revised form practices are endorsed by teachers as criteria for evaluation purposes. Analyzing the responses of 589
22 February 2014
Cypriot primary schoolteachers to a survey, we found that, unlike researchers, teachers did not appraise
Accepted 5 March 2014
these practices as distinctly important; rather teachers’ perceptions seemed to be informed by the
Available online 31 March 2014
centrality of teachers’ role during instruction. Teachers also rated these practices as more important than
feasible to implement. Implications for teacher evaluation and teacher education are provided.
Keywords:
Content-specific teaching practices
Ó 2014 Elsevier Ltd. All rights reserved.
Generic teaching practices
Teacher evaluation
Teacher perceptions

1. Introduction Hammerness, & McDonald, 2009), namely strategies, routines, or

Accumulated research evidence over the past decades suggests
that teachers matter for student learning (Hanushek & Rivkin,
2010; Nye, Konstantopoulos, & Hedges, 2004; Rockoff, 2004;
Strong, 2011). In particular, teacher effects have been found to be
larger than school effects (Muijs & Reynolds, 2000), cumulative,
and lasting over a sequence of years (Konstantopoulos & Chung,
2011; Rivers & Sanders, 2002). Acknowledging teachers’ critical
role in student learning, during the last few years, researchers are
increasingly attending to what they call “high leverage” or “core”
teaching practices (Ball, Sleep, Boerst, & Bass, 2009; Grossman,
* Corresponding author. Department of Education, University of Cyprus, Theo-
phanides Building, Rm 508, 11-13 Dramas Str., Nicosia 1077, Cyprus. Tel.: þ357 22
892989; fax: þ357 22 894488.
E-mail address: cycharal@ucy.ac.cy (C.Y. Charalambous).
activities whose proficient enactment by teachers is likely to lead to
comparatively large advances in student learning (cf. Lampert,
2010).
Despite this heightened interest in teaching practices, pertinent
scholarly work seems to have largely moved along two parallel
paths. Building on earlier works in the late 1970s (e.g., Brophy,
1979), several scholars (e.g., Danielson, 2007; Kyriakides &
Creemers, 2008; Pianta, La Paro, & Hamre, 2006) have focused on
generic practices, namely practices that cut across different subject
matters. Sparked by Shulman’s (1986) plea to attend to the subject
matter itself, another growing body of literature has focused on
content-specific practices, which are more relevant for particular
subject matters. Working at the intersection of both types of
practices can be beneficial for better understanding the work of
teaching and consequently student learning, as suggested by
scholars who have worked in either domain (e.g., Brophy, 1986;
Grossman & McDonald, 2008).

http://dx.doi.org/10.1016/j.tate.2014.03.001
0742-051X/Ó 2014 Elsevier Ltd. All rights reserved.
 C.Y. Charalambous et al. / Teaching and Teacher Education 41 (2014) 22e33
Recent scholarly discussions about the importance of inte-
grating both types of practices in teacher evaluation and classroom
observation systems (e.g., Hamre et al., 2013) seem to largely ignore
what teachers themselves believe about these practices: To what
extent do they accept these practices as criteria for evaluation
purposes? Do they rate them as distinctly important as they have
been treated for years in the literature, given the emphasis placed
on either type of practice? If so, do they endorse one type of
practice over the other? Examining teachers’ perceptions around
these issues is pivotal, given that any attempt to develop teacher
evaluation and/or classroom observation systems should also take
into consideration the perceptions of their ultimate recipients:
teachers and students (Goe, Bell, & Little, 2008; Ovando, 2001;
Stronge & Tucker, 1999; Tuytens & Devos, 2009). Besides, as
Muñoz, Scoskie, and French (2013) remind us, since teachers and
students “are the closest to the teaching-and-learning action,” we
need to “incorporate teachers and student voices in the important
debate around teacher effectiveness and evaluation” (p. 228).
Making a step toward documenting teachers’ voices, this
manuscript outlines a study that examined teachers’ perceptions
about using generic and content-specific teaching practices as
criteria for teacher evaluation. The remainder of the manuscript is
organized into six sections. The theoretical perspectives informing
the study are presented first, followed by the research questions,
and a brief outline of the context of the study. Next, information on
the study sample and data collection and analysis is presented. In
the last two sections, we outline the study findings and, in dis-
cussing them, provide implications for teacher evaluation and
education.
2. Theoretical perspectives
2.1. Focusing on teaching practices as criteria for teacher evaluation
Several criteria have been considered to evaluate teacher qual-
ity, including teacher qualifications, their personal attributes, their
pedagogical skills and practice, and their effectiveness (Hanushek,
2002; Strong, 2011). From the plethora of these criteria, in this
study we focus on those related to teachers’ practice; we do so for
two reasons.
When it comes to teacher evaluation, teachers themselves have
been found to prefer criteria that focus on their daily practice rather
than on other peripheral aspects of their work, such as the extent to
which they make use of available resources or the degree to which
they keep all stakeholders satisfied (cf. Cheng & Tsui, 1999). For
example, a study by Kyriakides, Demetriou, and Charalambous
(2006) has shown that Cypriot primary teachers considered the
working-process model (cf. Cheng & Tsui, 1999) as the most
appropriate for formative and summative evaluation purposes.
Unlike other models, this model focuses on observable teaching
behaviors, namely on what teachers do in the classroom. Similarly,
a recent study examining teachers’ perceptions about possessing
different competencies (Panti c & Wubbels, 2010) found that
teachers endorse teaching skills (e.g., designing and implementing
lessons) more than general educational values or an awareness of
the educational system.
Empirical evidence also corroborates our focus on observable
teaching behaviors as criteria for teacher evaluation. In particular,
studies exploring teacher-level factors suggest that what matters
most for student learning is what teachers do in the classroom
(Brophy & Good, 1986; Milanowski, 2004; Muijs & Reynolds, 2000;
Strong, 2011; Stronge, Ward, & Grant, 2012) rather than different
teacher attributes, such as their beliefs and job satisfaction (cf.
Borich, 1992; Creemers & Kyriakides, 2008; Hanushek, 1986) or
their background qualifications (Palardy & Rumberger, 2008). This
23
argument is also supported by recent meta-analyses and experi-
mental studies.
Based on more than 800 meta-analyses, Hattie’s (2009) meta-
analysis showed several teaching practices, such as using
advanced organizers, employing concept mapping, presenting
worked examples, and providing targeted and timely feedback to
have medium to large effect sizes (see p. 162). Similarly, a meta-
analysis of 167 studies focusing on teaching practices, such as
teacher assessment, questioning techniques, and providing ori-
enting/structuring information (Kyriakides, Christoforou, &
Charalambous, 2013) showed these practices to be moderately
associated with student achievement. An experimental study using
teacher random assignment to students (Kane, McCaffrey, Miller, &
Staiger, 2013) also empirically corroborates the emphasis placed on
teaching practices by showing these practices to significantly pre-
dict student learning. Therefore, it is not a coincidence that in his
synthesis of several studies exploring teacher quality, Strong (2011)
concludes that “learning what teachers actually do when they
conduct their lessons is perhaps the most important key to un-
derstanding the effects that teachers have on student learning” (p.
46).
A closer examination of the studies reviewed above reveals a
common denominator. With the exception of a meta-analysis
examined below, most studies consider only one type of prac-
tices. The scarcity of studies that focus on both generic and content-
specific practices underscores the importance of working at their
intersection, a point we elaborate next.
2.2. Focusing on generic and content-specific teaching practices
Attending to both types of practices in tandem when it comes to
exploring teaching quality seems critical, especially if one considers
how complex teaching is (Cohen, 2011; Leinhardt, 1993). Apart
from this common-sense argument that combining perspectives
can help better understand a complex phenomenon such as
teaching, theoretical and empirical considerations underline the
importance of integrating both types of practices in investigating
teaching and its effects.
From a theoretical standpoint, regardless of the subject matter
one teaches, several teaching practices can be important for sup-
porting student learning, as both older (e.g., Brophy & Good, 1986)
and more recent (e.g., Creemers & Kyriakides, 2008; Pianta, Belsky,
Vandergrift, Houts, & Morrison, 2008; Stronge et al., 2012) studies
have shown. Such practices include posing good questions, man-
aging classroom time, establishing a positive classroom climate,
and assessing student learningdto name a few. Although conse-
quential for enhancing student learning, the effective enactment of
these generic practices is not sufficient, because teaching particular
subject matters imposes additional disciplinary entailments on
teachers, as emphasized in Schwab’s (1978) and Shulman’s (1986)
seminal works. As a result, researchers are now increasingly
attending to practices whose effective performance can be partic-
ularly significant for supporting student learning in a given disci-
pline (cf. Hill & Grossman, 2013). Such practices include, for
instance, using representations in mathematics (e.g., Mitchell,
Charalambous, & Hill, 2014), demonstrating how to perform a
motor skill by providing selected learning cues in Physical Educa-
tion (e.g., Chen, Hendricks, & Archibald, 2011) or capitalizing on
texts during a language-arts lesson (cf. PLATO in MET, 2010).
This dual attention to both types of practicesdyet largely in
paralleldis also reflected in the plethora of generic or content-
specific theoretical frameworks and instruments that have been
developed to capture the extent to and the manner in which
teachers engage in such practices (cf. McClellan, Donoghue, & Park,
2013). Generic frameworks and instruments include, for example,
24 C.Y. Charalambous et al. / Teaching and Teacher Education 41 (2014) 22e33
the Framework for Teaching (Danielson, 2007), the Dynamic Model of
Educational Effectiveness (Kyriakides & Creemers, 2008), the Class-
room Assessment Scoring System (Hamre et al., 2013), and the Rapid
Assessment of Teacher Effectiveness instrument (Strong, 2011). In
mathematics, which is our focus, content-specific frameworks/
observation instruments include the Mathematical Quality of In-
struction (e.g., Hill, 2010), the UTeach Observation Protocol (cf. Kane
& Staiger, 2012), the Mathematics-Scan (Berry, Rimm-Kaufman,
Ottmar, Walkowiak, & Merritt, 2011), the Reformed Teaching
Observation Protocol (Sawada et al., 2002), and The Inside the
Classroom Observation and Analytic Protocol (Horizon Research,
2000)dwith the latter two focusing on both mathematics and
science.
From an empirical standpoint, the importance of considering
both types of practices is corroborated by research findings point-
ing to the benefits accrued in explaining student learning when
considering these practices in tandem. For example, although not
focusing squarely on content-specific teaching practices, Seidel and
Shavelson’s (2007) meta-analysis of 112 studies revealed that
involving students in content-specific activities (such as solving
problems in mathematics or employing specific reading and
writing strategies in language-arts) had larger effects on student
learning compared to the effects of other generic factors examined
in this meta-analysis, such as time-on-task or differentiated in-
struction. This finding was robust, regardless of the subject matter
taught (i.e., reading, mathematics, science), the stage of schooling
considered (i.e., elementary vs. secondary), or the type of student
learning outcomes examined (cognitive, motivational, learning
processes)dthus pointing to the potential pitfalls of ignoring one
set of practices over the other. A recent secondary analysis of PISA
2009 data (Charalambous & Kyriakides, 2013) also showed that
more variance in student learning can be explained when consid-
ering both types of practices in tandem as opposed to exploring
either generic or content-specific practices in isolation. Interest-
ingly, in some countries (e.g., Japan) generic practices had a larger
share in explaining differences in student performance, while in
other counties (e.g., France) the reverse pattern was true; this
finding again suggests that focusing only on one type of practices
can yield a partial picture of the process of teaching and learning.
Although integrating both types of practices while attempting to
understand teaching quality and its effects seems inevitable,
missing from the literature seem to be studies that give voice to
teachers and examine what teachers think about these practices.
3. Research questions
Focusing on two aspects of teachers’ perceptionsdthe impor-
tance of certain practices for teacher evaluation purposes and the
extent to which they are considered implementable in
instructiondwe ask:
1. To what extent are generic and content-specific teaching prac-
tices endorsed by teachers as criteria for their evaluation? If
considered important for evaluation purposes, are these prac-
tices deemed equally feasible to implement in instruction?
2. Do teachers rate generic and content-specific teaching practices
as distinctly important/feasible to implement?

If so, which practices are rated as more important/feasible to
implement?

If not, which generic/content-specific practices are rated as
the most/the least important/feasible to implement?
Exploring teachers’ perceptions regarding the importance and
feasibility of enacting such practices can have significant implica-
tions for teacher training and evaluation, a point to which we
return toward the end of this manuscript. Obviously, examining
whether teachers do not simply endorse but also incorporate
generic and content-specific practices in their work is necessary.
We nonetheless believe that even investigating teachers’ percep-
tions comprises an important first step when attempting to revamp
teacher evaluation in a given educational system such was the case
with the system considered in this study.
4. Teacher evaluation in Cyprus
Cyprus is characterized by a centralized educational system,
with the Ministry of Education and Culture being responsible for
educational policy-making and the enforcement of educational
laws. The teacher evaluation system in Cyprus is based on a 1976
law (Pashiardis, Savvides, & Tsiakkiros, 2005). According to this law,
teachers are evaluated annually by the school principal based on
four criteria: teacher professional knowledge, teaching skills and
classroom organization, participation in school-activities, and
community service. Teachers are also evaluated by superintendents
who typically observe them at most for four 40-min lesson periods
per year; teachers are notified about these visits a couple of days in
advance. For teachers with fewer than 12 years of experience, only a
descriptive written report is provided; teachers with more than 12
years of experience are also assigned a quantitative evaluation
based on a forty-point scale (10 points for each of the four afore-
mentioned criteria). Based on these evaluations, a teacher can apply
for promotion; teachers with the highest evaluations are inter-
viewed by the Education Service Commission, a five-person com-
mittee directly appointed by the country president. If the decision
is positive, teachers are promoted to assistant principals; a similar
approach is pursued for promoting assistant principals to the post
of school principal.
The evaluation system as described above has been in operation
for nearly 40 years, with almost no substantial alterations. Over this
period, several concerns have been voiced about its functionality,
reliability, and capacity to identify and promote the most effective
teachers (Brauckmann & Pashiardis, 2010). For example, the criteria
for evaluating teacher performance are vague; superintendents do
not have a common and known framework for observation and
grading, which often results in allegations about biased rat-
ingsdespecially when it comes to criteria not related to teacher
observable behaviors, such as their knowledge or their community
service. Additionally, the greatest majority of teachers are being
evaluated as “exceptional”; principals’ reports also do not
discriminate between teachers; and teachers’ perceptions about
their own performance are neither heard nor seem to be welcome.
Apparently, this situation is not endemic to the educational system
studied herein. For instance, a survey of over 15,000 teachers in
twelve districts in the U.S.A (Weisberg, Sexton, Mulhern, & Keeling,
2009) has yielded a comparable picture: most teachers were found
to be rated as proficient or highly proficient with only less than 1
percent of them being rated as unsatisfactory.
More recently, attempts to revamp such ineffective teacher
evaluation systems both in Cyprus (Kyriakides & Demetriou, 2007)
and elsewhere (e.g., McGuinn, 2012; Papay, 2012; Stamelos,
Vassilopoulos, & Bartzakli, 2012) have been invigorated with
several ideas put forward, including adopting more concrete
criteria for teacher evaluation. As a consequence, recent years have
seen concerted efforts to develop classroom observation systems
that examine different teaching practices (McClellan et al., 2013). In
this context, investigating teachers’ perceptions of the extent to
which they endorse generic and content-specific practices as
criteria for their evaluation appears to be both timely and imper-
ative both locally and internationally. The next section details the
methods of the study undertaken toward this end.
 C.Y. Charalambous et al. / Teaching and Teacher Education 41 (2014) 22e33 25

Table 1
Description of the eight classroom factors of the dynamic model of educational effectiveness.a

Factor Description (number of statements included in the survey) Sample statements

Classroom Refers to the use of appropriate techniques to collect data on student knowledge The teacher provides individualized feedback
assessment and skills, the identification of student needs, and the use of these results to to students.
inform students and parents (3).
Orientation Relates to communicating the objectives for which a specific task/lesson/series The teacher explains the purposes and objectives
of lessons take(s) place, challenging students to identify the reason(s) for which of the lesson.
an activity takes place in the lesson, as well as how a series of lessons are
linked together (3).
Structuring Includes practices such as starting the lesson with overviews and/or reviewing The teacher poses questions to link the lesson
the lesson objectives, outlining the content to be taught, signaling transitions of the day to previous lessons.
between lesson parts, and calling attention to/reviewing main ideas (3).
Teaching modeling Includes practices like encouraging students to use problem solving strategies The teacher provides students with strategies to
presented by the teacher or their classmates, asking students to develop such cope with difficulties in solving problems.
strategies and promoting the idea of modeling (3).
Classroom learning Pertains to establishing on-task behavior through appropriate classroom The teacher gives all students the opportunity to
environment interactions (i.e., teacherestudent and studentestudent interactions), dealing participate in the lesson and express their ideas.
with classroom disorder, and setting rules to establish student competition (3).
Time management Relates to maximizing the instructional time, by starting and finishing lessons on The teacher remains on-task for sufficient time.
time, minimizing the transition time between different activities, and ensuring
that the class remains on-task for sufficient time (3).
Questioning Consists of practices which relate to posing different types of questions When the teacher asks a question, he/she allots
(i.e., process and product) at appropriate difficulty levels, giving students time sufficient time for students to respond.
to respond, identifying student misconceptions, and dealing with student
responses (4).
Applications Refers to using seatwork or small-group tasks to provide students with needed The teacher provides students with opportunities
practice and application opportunities; it also includes the use of application to apply concepts/skills acquired either during the
tasks which are more challenging than those that students have been taught (2). lesson or in previous lessons.
a
Based on Creemers and Kyriakides (2008, 2010).

5. Methods aiming to measure this instructional quality as a separate and in-

5.1. Instrumentation and data collection
For the purposes of this study, we focus on one generic frame-
work (Dynamic Model of Educational Effectiveness, DMEE) and one
content-specific framework (the Mathematical Quality of Instruction,
MQI). We opted for these two frameworks because both have been
influential in their own respective domains and because they have
been used in large-scale studies (cf. Kane & Staiger, 2012; Scheerens,
2013). Additionally, in the past we worked with either framework
and we are currently exploring the potential of integrating both in
investigating the quality of instruction in different subject matters.
The DMEE consists of four levels of factors, situated at the stu-
dent, classroom, school, and system level. For the purposes of this
study, we focused on the eight factors at the classroom level, which
refer to observable behaviors (Creemers & Kyriakides, 2008) and
are briefly described in the first two columns of Table 1. Some of
these factors (e.g., structuring and classroom assessment) are
associated with direct teaching and mastery learning and are in line
with the main findings of processeproduct studies. Other factors
(e.g., orientation and teaching modeling) are more consonant with
constructivist theories of teaching and learning (Creemers &
Kyriakides, 2008). Several studies conducted both at national
(Kyriakides & Creemers, 2008; Kyriakides & Tsangaridou, 2008) and
international level (Panayiotou et al., 2013) for different subject
matters have provided empirical support to the DMEE, showing
instructional quality, as measured by these factors, to predict stu-
dent learning. Teachers were also found to be clustered into
different stages according to the teaching practices they imple-
mented during instruction, with teachers clustered at lower stages
being less effective in promoting student learning compared to
teachers situated at higher stages (Antoniou & Kyriakides, 2013;
Kyriakides, Creemers, & Antoniou, 2009).
Recognizing that existing frameworks and associated classroom
observation systems fail to capture the mathematical quality of in-
struction (cf. Learning Mathematics for Teaching Project, 2011) and
dependent construct, the developers of MQI have advanced a
framework that focuses squarely on teaching practices that are more
pertinent to mathematics instruction. Based on the “instructional
triangle,” which illustrates the various types of interactions occurring
during instruction (Cohen, Raudenbush, & Ball, 2003), the MQI
consists of four main dimensions, which are briefly described in the
first two columns of Table 2. Prior validation work has shown the
mathematical quality of instruction, as tapped by the MQI, to relate
to both teachers’ knowledge for teaching the subject (Hill et al.,
2008) and to student learning (Hill, Kapitula, & Umland, 2011). One
of the largest studies exploring instructional practice and student
learning (Kane & Staiger, 2012) has also shown the MQI measures to
relate positively and significantly to student gains, measured both by
state tests and a test gauging student conceptual understanding.
To capture teachers’ perceptions of generic and content-specific
teaching practices, a two-part survey was developed along the
basic premises of DMEE and MQI. To develop the first part, we first
reviewed both frameworks, identifying their main factors/di-
mensions, and developing statements to capture each. For each
factor/dimension, we generated a set of statements pertaining to
observable teaching practices; these statements were then
content-validated by project members of each framework. This
expert panel was also asked to identify the statements they
considered the most representative of each factor/dimension.
Based on their evaluation, we selected 24 statements for DMEE (3
for each of the first six factors of the model, 4 for the seventh factor,
and 2 for the last factor). An equal number of statements was also
selected for MQI: 11 for Richness, 6 for Working with students and
mathematics, 3 for Errors and imprecision, and 4 for Student partic-
ipation in meaning making and reasoning.1 Sample statements for
each factor/dimension appear in the last column of Tables 1 and 2. A
1
The unequal distribution of the MQI statements was due to the fact that the first
two MQI dimensions include a wider variety of teaching practices compared to the
latter two.
26 C.Y. Charalambous et al. / Teaching and Teacher Education 41 (2014) 22e33

Table 2
Description of the four main dimensions of the mathematical quality in instruction.a

Dimension Description (number of statements included in the survey) Sample statements

Richness of the mathematics
Working with students and
mathematics
Errors and imprecision
Student participation in
meaning-making and
reasoning
a
Based on Hill (2010) and Learning Mathematics for Teaching (2011).
Captures the depth of the mathematics offered to students. Rich The teacher encourages the use of multiple procedures
mathematics can be established by either focusing on the meaning and/or solution methods for a single problem.
of facts and procedures (e.g., through linking representations or
providing explanations) or by focusing on key mathematical
practices (e.g., working on multiple solutions, developing
generalizations, using appropriate mathematical language) (11).
Is associated with teachers’ ability to appropriately interpret and The teacher capitalizes on student mathematical
respond to students’ mathematical ideas and errors, as well as errors/difficulties to remediate their misconceptions.
their capacity to use these ideas and errors in instruction (6).
Captures teachers’ major mathematical errors or imprecision in Teacher’s instruction is clean of errors in mathematical
language and notation, as well as the lack of clarity/precision in language and mathematical notation.
the teacher’s presentation of the content (3).
Focuses on student involvement with the mathematical content The teacher encourages students to identify patterns
through activities such as student provision of mathematical and state generalizations based on them.
explanations, posing of mathematically motivated questions,
engagement in reasoning, pattern-noticing and examination
of claims and counter-claims (4).

brief description of all the statements used in the survey appears in
Appendix A.
Using a five-point Likert scale (1: strongly disagree and 5: strongly
agree), the teachers were asked to designate their level of agreement
with each statement twice: once with respect to importance (whether
they considered these criteria important for teacher evaluation pur-
poses) and a second time with respect to feasibility (whether they
deemed the practices implementable in their instruction). Any dis-
crepancies present in teachers’ perceptions regarding these two as-
pects could point to implications for teacher education, a point we
consider later on. A caveat is, however, in order here. The employment
of a Likert-scale survey for capturing teachers’ perspectives was not
without limitations, key among them being the following two: first,
the teacher participants were able to express their views only about
the practices outlined in the survey; second, apart from stating their
level of agreement with the given statements, teachers did not have a
chance to further elaborate their selections. We reconsider these
limitations in discussing the study findings.
The second part of the survey was intended to collect teacher
demographic information. This included teachers’ gender, position
at the school (teacher, assistant principal, and principal), higher
degrees held, years of teaching experience, and years of teaching
mathematics.
5.2. Sample
The survey was administered to 863 Cypriot primary teachers
who had at least one year of experience in teaching mathematics.
This was considered necessary, because without any previous
experience, teachers would not be able to ground their evaluations
on their own practice. Before administering the survey, the purpose
of the study was explained to teachers; it was also emphasized that
their participation was voluntary. Two weeks after the initial
administration of the survey, teachers were kindly reminded to
return their surveys should they opted to do so. This resulted in 589
completed surveys being returned (response rate: 68.25%).
The sample of teachers completing the surveys was represen-
tative of the Cypriot primary school teacher population in terms of
gender2 and the position held at school,3 but not in terms of
teachers’ years of experience, since the participants had fewer years
of experience compared to the population (x ¼ 13:03, SD ¼ 7.24,
m ¼ 16.26, s ¼ 6.26, t2 ¼ 754.51, df ¼ 4403, p < 0.001). All the
participants had at least a bachelor’s degree, with 58.15% of the
teachers holding a Masters’ Degree and 2.92% also holding a
doctorate. Finally, teachers’ years of experience in teaching math-
ematics ranged from 1 to 32 years (x ¼ 11:12, SD ¼ 7.06).4
5.3. Data analyses
Descriptive statistics (means and standard deviations) and
inferential statistics (t-test for paired samples, ManneWhitney U)
were employed to answer the first research question. To address
the second research question, we first performed two distinct
exploratory factor analyses, one for importance, and another for
feasibility. These analyses helped examine whether according to
their ratings, teachers differentiated between generic and content-
specific practices. Using the grouping of the statements yielded
from these two analyses, we then employed Item Response Theory
(Embretson & Reise, 2000), and particularly the Extended Logistic
Model of Rasch (Andrich, 1988), to develop two different scales, one
for importance and another for feasibility. To examine the psycho-
metric properties of these scales, we used a set of criteria, including
whether the outfit and the infit mean square values of the emerging
scales were close to 1; the normalized infit-t and outfit-t values had
a mean of zero; the separability for items (i.e., survey statements)
and persons (teachers) was higher than 0.75; each item had an infit
within the range of 0.77e1.30; and the person estimates were well
targeted against the item fit estimates (Adams & Khoo, 1996; Bond
& Fox, 2001).
Developing these scales helped examine the relative impor-
tance/feasibility of those statements based on teachers’ percep-
tions. However, caution needs to be exercised in interpreting this
relative positioning of the statements. Given our interest in rank
ordering the survey practices with respect to either importance or
feasibility, it was unavoidable that some statements would be
ranked first, some would be clustered in the middle, and some
would be ranked last. In this sense, the ranking of the statements
presented below should be seen in relative terms, both because the

2
Femalessample(s) ¼ 80.81%, femalespopulation(p) ¼ 81.14%; x2 ¼ 0.001, df ¼ 1,
p ¼ 0.99. 4
The Ministry of Education and Culture had no official data on the distribution of
3
Teachers(s) ¼ 85.12%, teachers(p) ¼ 79.27%; assistant principals(s) ¼ 12.63%, degrees or years of teaching mathematics within the teacher population. Hence, we
assistant principals(p) ¼ 14.12%; principals(s) ¼ 2.25%, principals(p) ¼ 6.61%; do not know whether the sample was representative in terms of these character-
2
x ¼ 3.46, df ¼ 2, p ¼ 0.17. istics, although unofficial information suggests that it might have been.
 C.Y. Charalambous et al. / Teaching and Teacher Education 41 (2014) 22e33 27

differences between any given set of statements might not be huge Table 3
and because sampling a different set of practices might yield Means, standard deviations, and comparisons of the levels of importance and
feasibility of the 48 teaching practices.
different results.
Teaching Importance Feasibility Comparison
6. Results Practicea
x SD x SD tb df

1 4.66 0.54 3.99 0.80 20.85 582

The study findings are organized along the two research 2 4.51 0.69 4.08 0.82 13.22 582
questions. We first explore whether teachers endorsed the 3 4.49 0.73 3.86 0.83 16.61 579
teaching practices included in the survey as criteria for their 4 4.64 0.61 4.29 0.73 12.30 581
5 4.53 0.67 4.41 0.71 4.10 585
evaluation. We also search for differences between the level of
6 4.37 0.69 3.87 0.75 15.27 580
importance attributed to these criteria and the level of perceived 7 4.28 0.84 4.12 0.86 4.59 582
feasibility. We then investigate whether the study participants 8 4.65 0.56 3.81 0.78 25.33 577
rated the two types of practices as distinctly important/feasible to 9 4.36 0.78 3.81 0.82 13.97 578
implement and if so, which practices they considered more 10 4.29 0.75 3.78 0.81 14.35 576
11 4.52 0.65 3.26 0.91 30.70 578
important/implementable.
12 4.27 0.75 3.51 0.90 20.13 580
13 4.60 0.63 4.27 0.72 10.89 579
6.1. Investigating teachers’ endorsement of generic and content- 14 4.59 0.63 4.29 0.75 10.65 581
specific practices 15 4.59 0.62 4.12 0.86 13.81 582
16 4.67 0.57 4.44 0.66 9.03 580
17 4.10 0.89 3.80 0.86 7.51 582
Table 3 presents descriptive statistics of teachers’ endorsement 18 4.53 0.64 4.13 0.77 13.43 582
of each practice, be it for importance or feasibility. Given that 19 4.23 0.85 4.00 0.84 6.93 582
teachers were asked to state their level of endorsement using a 20 4.57 0.58 4.22 0.74 12.58 584
five-point Likert scale, the following three observations arise from 21 4.33 0.85 3.55 0.92 17.65 582
22 4.06 0.82 3.54 0.84 13.90 584
this table. First, for both importance and feasibility, the means for
23 4.55 0.64 3.80 0.86 19.41 586
each practice were at least higher than the median of the scale 24 4.58 0.64 4.17 0.79 13.07 583
(i.e., 3). This implies that the study participants deemed this set of 25 4.69 0.56 3.92 0.93 20.21 580
practices as important for their evaluation, and, by and large, 26 4.24 0.76 3.55 0.85 19.71 584
27 4.45 0.69 3.83 0.81 18.09 584
considered them feasible to implement in their instruction. In
28 4.50 0.68 3.79 0.91 19.68 583
particular, for importance, with only one exception (Teaching 29 4.64 0.57 4.41 0.74 7.59 585
Practice 43), all other practices had a mean score higher than 4 30 4.48 0.68 3.96 0.84 15.36 584
(i.e., means ranged from x ¼ 4:03 to x ¼ 4:69)dwhich suggests a 31 4.26 0.81 3.17 1.03 23.88 580
high level of endorsement of all teaching practices included in the 32 4.48 0.70 3.94 0.85 16.41 585
33 4.46 0.66 3.90 0.88 17.18 583
survey. For feasibility, the corresponding means were lower (from
34 4.64 0.65 4.34 0.75 10.17 584
x ¼ 3:17 to x ¼ 4:44). The use of t-test for paired samples (see 35 4.03 0.94 3.61 0.97 11.06 581
Columns 6e7) showed that, with no exception, teachers deemed 36 4.56 0.66 4.23 0.79 11.18 585
the 48 practices more important than implementable in their 37 4.54 0.63 4.04 0.78 16.07 584
instruction. 38 4.46 0.68 3.84 0.84 18.32 583
39 4.09 0.89 3.68 0.87 11.49 577
Second, the employment of ManneWhitney U test indicated 40 4.40 0.71 3.97 0.88 13.12 581
that there were not significant differences between generic and 41 4.57 0.64 4.09 0.80 15.51 583
content-specific teaching practices for either importance (Manne 42 4.45 0.68 4.00 0.81 15.16 582
Whitney U(46) ¼ 249, z ¼ 0.81, p ¼ 0.42) or feasibility (Manne 43 3.66 1.00 3.18 0.97 12.09 581
44 4.45 0.71 3.95 0.87 14.14 578
Whitney U(46) ¼ 224, z ¼ 1.32, p ¼ 0.19). This implies that for this
45 4.51 0.67 3.97 0.86 15.77 583
given set of practices, teachers did not endorse one type over the 46 4.49 0.66 4.04 0.82 14.85 580
other for teacher evaluation purposes. 47 4.47 0.67 3.92 0.84 16.93 583
Third, we observe that even in terms of individual items, both 48 4.56 0.63 4.09 0.83 14.37 583
generic and content-specific practices are included in the top and a
Teaching practices as they appeared on the survey (odd numbers: generic
bottom five items of teachers’ rankings, both for importance and teaching practices; even numbers: content-specific teaching practices).
feasibility. For example, a content-specific practice (i.e., 16) was
b
All differences were statistically significant at level a ¼ 0.001.
considered the most feasible and was endorsed as the second most
important for evaluation purposes; stipulating that all students
should be afforded opportunities to participate in the lesson, a
teachers did not rate these practices as distinctly important or
generic practice (i.e., 25) was the most endorsed in terms of
feasible, since only one-factor solution emerged for either impor-
importance. This pattern suggests that teachers might not have
tance or feasibility. This was suggested by the fact that the ratio of
differentiated between the two sets of practices in their ratings, a
the first to the second eigenvalue for importance was 9.89 and for
point to which we turn next.
feasibility was 7.29 (explaining 38.5% and 36.3% of the total variance
correspondingly).5 Ratios of this magnitude support a unidimen-
6.2. Are generic and content-specific practices rated as distinctly
sional rather than a multidimensional construct (cf. Kline, 1994).
important/feasible?
Although the emergence of a single factor for either aspect exam-
ined could be attributed to social desirability (Edwards, 1957), we
Besides investigating the level of importance and feasibility of
the 48 teaching practices, a key goal of this study was to examine
whether teachers rated these two types of practices as distinctly
important. Toward this end, we first submitted the 48 statements 5
Even when considering multiple-factor solutions with two, three or four factors
into two exploratory factor analyses, one for importance, and (for either importance or feasibility), all the emerging factors included a mixture of
another for feasibility. Exploratory factor analyses revealed that the generic and content-specific teaching practices.
28 C.Y. Charalambous et al. / Teaching and Teacher Education 41 (2014) 22e33

argue that this was not the case. This is because, in addition to dimension of MQI. This dimension includes statements pertaining
importance, we also examined feasibility, and we have no reason to to the significance of involving students in cognitively demanding
believe that teachers endorsed the feasibility statements to simply activities, such as identifying patterns, engaging in reasoning, or
please the researchers; besides, as already discussed, the feasibility considering the underlying meaning of given mathematical oper-
statements were endorsed significantly less than the importance ations. This quadrant also includes most of the practices of time
statements. management, teaching modeling, and the classroom as a learning
Given this unidimensionality, to further search for patterns as to environment from DMEE. For example, clustered in this quadrant
which teaching practices were deemed the most or least important/ were teaching practices such as the teacher spends as much time as
feasible, we then employed the Extended Logistic Model of Rasch needed on a given activity, models strategies for solving problems,
(Andrich, 1988). Using the 48 practices as scale items, we developed and encourages student collaboration. Regardless of being generic
two different scales, one for importance and another for feasibility. or content-specific, the practices clustered in this quadrant give
After dropping Teaching Practice 43, whose item infit t lied outside students a prominent role in the process of teaching and learning,
of the acceptable range of 0.77e1.30 logits, the emerging scales for and ascribe teachers a more facilitative role.
both importance and feasibility had good psychometric properties, At the antipode of Quadrant A lies Quadrant C, which encom-
as illustrated in Table 4. Specifically, the separability values for both passes all practices considered both more important and feasible.
items and persons were at least 0.84; the infit and outfit mean From MQI, this quadrant includes (a) all the practices related to
square values of the emerging scales were very close to 1 (ranging providing instruction clean of mathematical errors and linguistic or
from 1.00 to 1.03); and the normalized infit-t and outfit-t values notational imprecision, (b) a little more than half of the statements
had means close to zero (with the highest mean in absolute value (6 out of 11) of Richness, especially those related to providing
being equal to 0.10). Finally, the person estimates were well tar- appropriate mathematical explanations and using mathematical
geted against the item estimatesdespecially for the feasibility language precisely, and (c) half of the statements of Working with
scaledwith some limitations in the upper and lower ends of the students and mathematics, especially those concerned with capi-
two scales. talizing on student errors or contributions to support student
Provided that the two scales had good psychometric properties, learning. From DMEE, this quadrant comprises most of the practices
we then cross-tabulated the item difficulty levels of the impor- of student assessment and those related to structuring; it also en-
tance scale with those of feasibility scale, and taking the mean of compasses half of the statements of questioning. Sample state-
these item difficulty levels, we created the four quadrants shown ments include that teacher assesses student learning
in Fig. 1. Going counter-clockwise, Quadrant A represents practices systematically, tries to understand the degree to which students
deemed both less important and feasible, Quadrant B shows have understood what they have been taught, asks process rather
practices considered less feasible but more important, Quadrant C than product questions, connects the lesson of the day with pre-
illustrates practices thought to be both more important and vious lessons, and rephrases a question in simpler terms when
feasible, and Quadrant D corresponds to practices evaluated as less students encounter difficulties to respond. A common denominator
important but more feasible. Examining how the practices in all these statements, both those pertaining to generic practices
(generic or content-specific) were clustered in each quadrant and those related to content-specific practices, is that they ascribe a
helped explore whether teachers’ perceptions coincided with more central role to teachers: the teacher is the main actor in the
scholarly views as to what comprises important elements of entire teaching and learning process and engages in activities
teaching mathematics (cf. National Council of Teachers of ranging from providing comprehensible explanations, to capital-
Mathematics [NCTM], 2000). izing on student productions, to asking appropriate questions and
Given that odd numbers correspond to generic teaching prac- assessing student learning. Although students are definitely not
tices and even numbers correspond to content-specific practices, absent from such activities, their role appears to be less
Fig. 1 shows that all quadrants include a mixture of both types of consequential.
practices. This again corroborates the idea that teachers did not rate Quadrant B includes a conglomerate of practicesdboth
these practices as distinctly important or feasible. A closer exami- generic and content-specificdthat were considered important
nation of Fig. 1 reveals that Quadrant A encompasses all practices but less applicable in daily practice. Interestingly, these practices
related to the Student-participation-and-meaning-making align with what several scholars (e.g., Creemers & Kyriakides,
2008; Kilpatrick, Swafford, & Findell, 2001; NCTM, 2000) re-
gard as elements of successful (mathematics) lessons: building
connections among different mathematical ideas, solving prob-
Table 4 lems in multiple ways, identifying the source of student mis-
Psychometric properties of the two scales.
conceptions, providing individualized student support, and
Statistical indicators Importance Feasibility allotting sufficient waiting time before calling on a student to
Means (items) 0.00 0.00 answer a question. What we found worrisome is that the study
(subjects) 1.49 0.02 participants acknowledged the importance of these practices, but
Standard deviations (items) 0.55 0.66 considered them relatively hard to implement. This discrepancy
(subjects) 1.46 1.21 raises the question as to what might hinder teachers from
Separabilitya (items) 0.84 0.92
(subjects) 0.94 0.95
materializing these practices in their instruction, a point to
Mean Infit mean square (items) 1.00 1.00 which we return below.
(subjects) 1.03 1.00 Quadrant D encompasses most of the practices related to
Mean Outfit mean square (items) 1.02 1.01 orientation from DMEE (e.g., teachers outline the lesson objectives
(subjects) 1.02 1.01
or explain how a lesson is related to previous or next lessons within
Infit t (items) 0.03 0.08
(subjects) 0.02 0.10 a unit) and content-specific statements, such as asking students to
Outfit t (items) 0.08 0.03 describe the steps of a given mathematical procedure. That these
(subjects) 0.02 0.01 practices were considered applicable but less important for evalu-
a
Separability (reliability) represents the percentage of observed variance that is ation purposes raises the question as to whether teachers really
explained. implement them in their daily practice, given that research has
 C.Y. Charalambous et al. / Teaching and Teacher Education 41 (2014) 22e33
Fig. 1. The four quadrants of teachers’ endorsement of generic and content-specific practices in terms of their importance and feasibility.
shown at least some of these practices to be relatively hard to enact
(see Kyriakides et al., 2009).
7. Discussion and conclusions
Reforming teacher evaluation appears to be a high-priority issue
for several educational systems (cf. Klenowksi, 2012; McGuinn,
2012; Papay, 2012; Steiner, 2010; Strong, 2011). From this respect,
although conducted in a specific educational system, this study
yields three important lessons not necessarily confined to its local
context. First, that examining teachers’ perceptions about the
importance of certain teaching practices as criteria for teacher
evaluation might not be sufficient, for important insights can also
be gleaned by investigating teachers’ views about whether these
practices can be implemented in instruction. Second, that unlike
the scholarly community which, as discussed above, seems to have
endorsed either generic or content-specific practices in examining
instructional quality, teachers might not rate these practices
differently. Third, that for teachers, what matters most might not be
the type of practices per se, but the centrality of their role during
enacting these practices as opposed to that of students. In this
section, we elaborate upon these ideas and draw implications for
teacher evaluation and education.
7.1. Considering teaching practices as criteria for teacher evaluation
The study participants were particularly supportive of nearly all
the teaching practices examined in the study. With only one
exception of a practice related to the difficulty of the tasks
assigned during the lesson (Practice 43), all other 47 practices
were highly supported by the teachers, given that their mean level
of acceptance was higher than four on a five-point Likert scale.
This result aligns with prior research findings in Cyprus and in
29
other countries suggesting that teachers are in favor of using
observable teacher skills and behaviors as criteria for their eval-
uation (Kyriakides et al., 2006; Panti
c & Wubbels, 2010). In Cyprus,
where this study was conducted, this is not surprising, given that
the teacher evaluation system has been harshly criticized as being
subject to superintendents’ and principals’ biases and has often
been alleged to promote “friends and acquaintances” (cf.
Brauckmann & Pashiardis, 2010, p. 347). By endorsing these
teaching practices, the study participants might, in essence, have
been showing a preference toward more observable criteria as
compared to other ambiguous and often contestable criteria (e.g.,
teacher knowledge or community service). But does this piece of
evidence suffice when trying to develop a new or reform an
existing teacher evaluation system?
A cursory interpretation of the abovementioned finding could
be that, given the political dimension of change (cf. Donaldson &
Papay, 2012; Stronge & Tucker, 1999), examining teachers’ per-
ceptions about the importance of these practices is indeed suffi-
cient. This might be true, especially when trying to establish a
teacher evaluation system through consensus among different
stakeholdersdbesides, in this way, as one could claim, teachers’
voices are heard. But are they really so? Both the study limitations
and the study findings suggest that this might not be the case. To
start, the study participants were presented with a preselected set
of criteria and were not given the opportunity to identify any other
criteria they would endorse for their evaluation. Second, even for
this set of criteria, teachers were only asked to state their level of
agreement, without being prompted to elaborate their thinking.
There is yet a third reason for concern, which emerges from
examining teachers’ perceptions regarding the feasibility of
employing these practices in their work: with no exception, all 48
teaching practices were deemed more important than imple-
mentable; these differences were significant at level a ¼ 0.001,
30 C.Y. Charalambous et al. / Teaching and Teacher Education 41 (2014) 22e33
which suggests that they would hold even after correcting for the
fact that multiple comparisons were undertaken.
One could interpret this finding to imply that the study partic-
ipants were simply paying lip service to the practices under
consideration without necessarily enacting them in their instruc-
tion. While this is definitely an open issue that calls for future
research, two other explanations are also possible and important to
entertain in considering this discrepancy, especially when one
strives to acknowledge and respect teachers’ voices. First, perceived
or real constraints might prevent teachers from realizing these
practices in their teaching. Such constraints include the pressure to
cover the curriculum, classroom disciplinary problemsdespecially
when it comes to giving students more autonomy and a more
central role in the teaching and learning processdor having to
work in mixed-ability classes with students of different needs. This
explanation is reinforced by an arsenal of studies conducted during
the 1990s (e.g., Ball, 1990; Peterson, 1990; Putnam, 1992), which
document teachers’ difficulties in implementing different teaching
practices in their instruction because of constraints like those
identified above. Another explanation is also plausible. Although
teachers might realize the importance of these practices for student
learningdand consequently for teacher evaluation purposesdthey
might not be adequately prepared to incorporate them in their
teaching.
Whatever explanation might be driving this discrepancy, certain
implications for teacher evaluation and teacher education can be
drawn. With regards to teacher evaluation, this finding underlines
that, besides exploring teacher endorsement of certain criteria for
evaluation purposes, any attempts to develop a new or revamp an
existing evaluation system also need to identify teachers’ concerns
or anticipated difficulties when asked to implement these criteria/
practices in their teaching. Doing so can serve as another way of
empowering teachers and giving them voice, for their opinions and
concerns are heard. At the same time, exploring teachers’ percep-
tions can also help identify perceived or real challenges that might
impinge on teachers’ attempts to modify their teaching to render it
more aligned with the proposed evaluation criteria. This is partic-
ularly important not only for safeguarding that a teacher evaluation
system will initially be endorsed by different stakeholders, but
more critically, for ensuring that this system will eventually be
implemented and will have potential for improving teaching
quality. In developing such systems, studies adopting other
methods can obviously help conceptualize the survey findings of
the current study: in-depth interviews with selected samples of
teachers can probe into what teachers themselves consider the
main reasons for evaluating certain practices less feasible than
important. Ethnographic studies following teachers in their class-
rooms for a relatively long period of time could also help under-
stand whether certain constraints prevent teachers from
incorporating these practices in their teaching.
With regards to implications for teacher educationdand to
the extent that the hypothesis advanced above about the diffi-
culties of implementing certain practices in instruction
holdsdteacher training and professional development seminars
could be designed to help teachers identify and discuss ways to
effectively implement such practices in their teaching. Such
seminars should not be one-time shot events. Instead, teachers
need systematic and recurrent opportunities to discuss these
practices with their peers, guided facilitation to develop strate-
gies for implementing them in their teaching, encouragement to
experiment with materializing these practices in their instruc-
tion, but alsodand perhaps more criticallydopportunities to
reflect upon their experimentations and receive feedback from
peers and teacher educators. Such focused and sustained train-
ings can encourage teachers to not only implement these
practices in their work, but to also gradually refine them to
further support student learning (cf. Silver, Ghousseini, Gosen,
Charalambous, & Strawhun, 2005).
A note is in order here. Teachers’ perceived or real challenges
should not be the ultimate criterion for incorporating (or not)
certain teaching practices in a given evaluation system. Listening to
teachers’ voices does not imply ignoring other equally criticaldto
say the leastdvoices, such as those coming from different meta-
analyses (see Section 2), and which pinpoint to practices that are
consequential for student learning. The real bet is not to develop a
teacher evaluation system that is simply accepted by different
stakeholders; rather, it is to develop a widely accepted system that
also has the potential to improve teaching quality. In developing
such a system, teacher initial and ongoing education can have a role
by supporting teachers in both appraising and implementing
certain practices in their work. It is for this reason that we and
others (e.g., Hill & Grossman, 2013; Papay, 2012) see teacher eval-
uation and teacher education going hand-in-hand: teacher educa-
tion can prepare and support teachers to accept and enact
ambitious teaching practices in their work; at the same time,
teacher evaluation can act as a lever and provide tools for
improving instructional practice.
7.2. Rating generic and content-specific practices as equally
important/implementable
Unlike the research community which has for years worked on
either generic or content-specific practicesdthus endorsing one set
of practices over the otherdthe study participants did not rate the
two types of practices as distinctly important for teacher evalua-
tion; nor did they consider them distinctly different to implement.
This was suggested by the two exploratory factor analyses, which
both showed the teaching practices under consideration to form a
single factor; the results of the ManneWhitney U test that showed
no statistically significant differences in teachers’ ratings; and the
quadrants shown in Fig. 1, all of which included generic and
content-specific practices.
Cognizant of the study limitations, we hasten to note that
replication studies with other teacher populations in different
educational settings are needed before making strong claims
regarding teachers’ endorsement of both types of practices; in-
depth examinations of teacher perceptions through open-ended
written prompts or interviews are also warranted. Yet, the study
findings seem to suggest that dichotomies between different per-
spectives, often imposed by researchers when attempting to
investigate complex phenomena, such as teaching, might not be
reflected in teachers’ views. Daily immersed in the complex work of
teaching, teachers might rate generic and content-specific teaching
practices as equally vital parts of their teaching toolkit. From this
respect, and given that this “voice” comes from those who are at the
forefront of teaching, this finding resonates with and reinforces
recent scholarly calls to integrate different perspectives in studying
teaching (Grossman & McDonald, 2008; Hamre et al., 2013). For as
Douglas (2009) has emphasized, to sharpen our focus in measuring
and understanding instruction, we need frameworks “capable of
supporting the full range of dimensions that influence teaching and
learning” (p. 519).
Provided that “voices” like those heard in the present study are
replicated in studies recruiting other teacher populations in both
centralized and de-centralized educational systems, as well as
different stakeholders (e.g., superintendents, administrators, and
teacher educators), from a practical standpoint, the study findings
suggest that both types of practices could be integrated in teacher
evaluation systems. However, this seems easier said than done:
How many such practices should/could be included in such
 C.Y. Charalambous et al. / Teaching and Teacher Education 41 (2014) 22e33
systems so that valid information on teachers’ performance could
be collected? How many lessons ought to be observed per teacher
so that teachers are appropriately evaluated based on these
practices? Are superintendents and/or school principals suffi-
ciently prepared to assess teachers using these criteria? If not,
what type of training should they receive to effectively assess
teachers along these practices, and more critically, be able to
provide constructive feedback for improvement? As the interna-
tional research community increasingly realizes (cf. Correnti &
Martinez, 2012; Hill, Charalambous, & Kraft, 2012), these issues
are not trivial, and addressing them requires concerted efforts and
deep thinking on the part of those responsible for measuring
instructional quality.
7.3. Investigating nuances in teachers’ endorsement of the explored
practices
Given that the teacher participants did not rate the two sets of
practices as distinctly important or implementable, a logical
question arises: If the type of practice was not an issue for these
teachers, what criteria could have driven their appraisals? While
we acknowledge that the ranking of the practices in Fig. 1 is
relative, the clustering of these practices in the four quadrants in
this figuredand particularly in Quadrants A and Cdis revealing:
whereas scholarly discussions might be focusing on whether these
practices are applicable across or within specific subject matters,
practitioners might be more concerned with how central their role
is in teaching as opposed to that of students. This argument is
reinforced by the fact that the greatest majority of the practices
included in Quadrant A assign students a central role in instruc-
tion. These practices are also more aligned with what other re-
searchers (cf. Antoniou & Kyriakides, 2013; Brekelmans, Sleegers,
& Fraser, 2000) have linked to constructivist approaches, and
particularly to students’ active involvement in instruction. In
contrast, practices grouped in Quadrant C give teachers a more
prominent role and seem to keep students at the background;
hence, these practices appear to be more aligned with direct/
active teaching.
In contrast to teachers’ perceptions, prior research has
underlined the importance of the practices grouped in Quadrant
A for student learning. For example, capitalizing on the eight
factors of DMEE at the classroom level, Kyriakides and colleagues
(Antoniou & Kyriakides, 2013; Kyriakides et al., 2009), showed
that teachers could be clustered into five distinct stages, ac-
cording to whether they incorporated particular practices in their
teaching. More importantly, teachers who incorporated more
practices of the upper stages (e.g., teaching modeling, classroom
as a learning environment)dthose clustered in Quadrant
Adwere also more effective in terms of promoting student
learning than their counterparts who mainly employed practices
of lower stages (e.g., structuring)dthose grouped in Quadrant C.
Similarly, as Doyle (1983) argued more than three decades ago,
and as empirical evidence has since attested (Baumert et al.,
2010; Boaler & Staples, 2008; Stein & Lane, 1996), engaging
students in cognitively demanding tasks and in activities that
require mathematical reasoning and thinkingdwhich resonates
with the practices of Quadrant Adcan significantly boost student
learning.
This discrepancy between teachers’ perceptions and empirical
evidence accrued over the past three decades has implications in
two different directions. On the one hand, teachers could be
sensitized to the gap between what they believe is important and
feasible to implement in their instruction and what research has
repeatedly shown as critical for enhancing student learning (i.e.,
sensitizing teachers to the voice(s) of the research community).
31
Developing a clear(er) understanding of the contribution of these
practices to student learning might, in fact, serve as a lever moti-
vating teachers to at least experiment with these practices in their
instruction. Of course, as discussed above, this is not sufficient
because, besides good intentions for experimenting with such
practices, teachers also need sustained and carefully structured
support to implement these practices in their lessons. On the other
hand, as already highlighted, researchers, superintendents, ad-
ministrators, and teacher educators alike need to be sensitized to
teachers’ voice(s): Why do teachers consider these practices hard to
implement and/or less important than others? Are they simply
ignoring what research might be suggesting or do perceived or real
classroom realities prevent them from materializing these practices
in their work? Whatever the answer is, establishing channels of
communication that allow both types of voices to be heard seems
to be productive.
Professional development in the form of video clubs (van Es &
Sherin, 2008) could help establish such channels of communi-
cation, by engaging teachers in discussing videotaped lessons of
their own teaching. Through guided discussions, researchers can
first investigate whether teachers notice the frequency with
which certain practices occur in their teaching and to then
explore how teachers interpret the presence or absence of these
practices in their work. At the same time, teachers can be
informed about what research suggests in terms of the impor-
tance of incorporating different practices in their work. These
interchanges could help both teachers and researchers create a
forum for discussing how such practices can be incorporated in
teachers’ daily work and how certain contextual factors can be
eliminated or weakened to enable better implementation of
these practices in instruction.
7.4. Future directions
Although directions for future studies have already been
offered throughout this section, there are additional venues for
future research. In particular, we wonder, how aligned are the
perceptions of different stakeholders (e.g., superintendents, ad-
ministrators, teacher educators) with those of teachers? Do these
different stakeholders focus more on the centrality of teachers’
role in instruction, or are their views more aligned with the
scholarly perceptions discussed above? Finally, it would be inter-
esting to explore whether teachers’ perceptions change after
participating in professional development programs as previously
outlined.
7.5. In lieu of conclusion
We initiated this investigation by emphasizing the impor-
tance of giving teachers voice to express their opinions regarding
a set of teaching practices. In an era dominated by heated de-
bates about the need to hold teachers accountable for student
learning and the ways to do so (Baker et al., 2010; Kennedy, 2010;
Papay, 2012), it seems at least unfair to deprive teachers from the
opportunity to express their ideas about issues that might have
consequential implications for their career. In line with other
studies that argue for affording teachers a platform for express-
ing their ideas and perceptions, this study made a step in
creating such a forum for teachersdalthough, admittedly, within
certain confines. We hope that studies assigning teachers a more
prominent role in discussions around teacher evaluation and
classroom observation systems will be more frequent in the years
to come.
32 C.Y. Charalambous et al. / Teaching and Teacher Education 41 (2014) 22e33
Appendix A. Brief description of the survey statements
# Generic teaching practices
1 Teacher (T) evaluates student progress systematically
3 T ensures that instructional time is used appropriately
5 T asks questions to connect the lesson of the day to other lessons
7 T explains the goals of the lesson
9 T encourages student collaboration
11 T provides individualized feedback to students
13 T poses questions to examine student learning
15 T attends to the processdnot only the final productdin student answers
17 T encourages competition among students
19 T explains how the lesson connects to prior/next lessons
21 T ensures that the lesson starts and finishes on time
23 T gives student time to respond to his/her questions
25 T encourages all students to participate in the lesson
27 T models strategies for overcoming difficulties
29 T restates a question in simpler terms to scaffold student thinking
31 T remains on an activity for sufficient time
33 T gives students opportunities to apply what was learned in the lesson
35 T explains how the lesson activities serve the goals of the lesson
37 T poses questions to assess student learning
39 T explains how each activity connects to its prior activities
41 T asks process rather than only product questions
43 T assigns more difficult tasks than those considered in the lesson
45 T poses questions to identify student difficulties
47 T builds on student thinking to discuss a problem solving strategy
References
Adams, R. J., & Khoo, S. T. (1996). Quest-the interactive test analysis system. Australian
Council of Educational Research.
Andrich, D. (1988). A general form of Rasch’s extended logistic model for partial
credit scoring. Applied Measurement in Education, 1(4), 363e378.
Antoniou, P., & Kyriakides, L. (2013). A dynamic integrated approach to teacher
professional development: impact and sustainability of the effects on
improving teacher behavior and student outcomes. Teaching and Teacher Edu-
cation, 29(1), 1e12.
Baker, E. L., Barton, P. E., Darling-Hammond, L., Haertel, E., Ladd, H. F., Linn, R. L.,
et al. (2010). Problems with the use of student test scores to evaluate teachers (EPI
Briefing Paper 278). Washington, DC: Economic Policy Institute.
Ball, D. L. (1990). Reflections and deflections of policy: the case of Carol Turner.
Educational Evaluation and Policy Analysis, 12(3), 247e259.
Ball, D. L., Sleep, L., Boerst, T., & Bass, H. (2009). Combining the development of
practice and the practice of development in teacher education. Elementary
School Journal, 109(5), 458e474.
Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, Jordan,, A., et al. (2010).
Teachers’ mathematical knowledge, cognitive activation in the classroom, and
student progress. American Educational Research Journal, 47(1), 133e180.
Berry, R. Q., III, Rimm-Kaufman, S. E., Ottmar, E. M., Walkowiak, T. A., & Merritt, E.
(2011). The mathematics scan (M-Scan): A measure of mathematics instructional
quality. Unpublished measure. Retrieved 20.01.13 from http://www.vsup.org/M-
Scan%20Workshop%20Article%202%2009272011.pdf.
Boaler, J., & Staples, M. (2008). Creating mathematical futures through an equitable
teaching approach: the case of Railside school. Teachers College Record, 110(3),
608e645.
Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model: Fundamental measurement
in the human sciences. Mahwah, NJ: Lawrence Erlbaum Associates.
Borich, G. D. (1992). Effective teaching methods (2nd ed.). New York: MacMillan.
Brauckmann, S., & Pashiardis, P. (2010). The clash of evaluations: in search of the
missing link between school accountability and school improvement: experi-
ences from Cyprus. International Journal of Educational Management, 24(4),
330e350.
Brekelmans, M., Sleegers, P., & Fraser, B. (2000). Teaching for active learning. In
P. R. J. Simons, J. L. van der Linden, & T. Duffy (Eds.), New learning (pp. 227e242).
Dordrecht: Kluwer Academic Publishers.
Brophy, J. (1979). Teacher behavior and its effects. Journal of Educational Psychology,
71, 733e750.
Brophy, J. (1986). Teaching and learning mathematics: where research should be
going? Journal for Research in Mathematics Education, 17(5), 323e346.
Brophy, J., & Good, T. L. (1986). Teacher behaviour and student achievement. In
M. C. Wittrock (Ed.), Handbook of research on teaching (pp. 328e375). New York:
MacMillan.
Charalambous, C. Y., & Kyriakides, L. (2013, August). Exploring generic and domain-
specific factors of effective teaching: Secondary PISA analyses. Paper presented
at the 15th EARLI Conference. Germany, Munich.
Chen, W., Hendricks, K., & Archibald, K. (2011). Assessing pre-service teachers’
quality teaching practices. Educational Research and Evaluation, 17, 13e32.
# Content-specific teaching practices
2 T builds clear links and connections between representations
4 T makes no errors in mathematical language and notation
6 T encourages students to identify patterns and generalizations
8 T identifies the source of student errors/misconceptions
10 T encourages the building of connections through specific examples
12 T encourages students to compare different solution methods
14 T uses mathematical language fluently
16 T ensures that the lesson activities are connected to the mathematical content
18 T presents the mathematical content/lesson activities with clarity
20 T selects appropriate examples, numbers, and representations
22 T distinguishes between correct and incorrect bits of thinking in
students’ mathematical productions
24 T encourages students to use appropriate mathematical language
26 T asks students to identify connections between mathematical ideas
28 T encourages students to use multiple solution approaches
30 T capitalizes on students’ mathematical productions
32 T builds clear links between different mathematical concepts
34 T ensures that she/he does not make any mathematical errors
36 T clearly outlines the steps of a mathematical procedure
38 T encourages students to explain the meaning of different operations
40 T explains how the problem features lead to particular solution paths
42 T builds connections between representations and mathematical concepts
44 T asks students to describe the steps of a mathematical procedure
46 T uses mathematically appropriate/student comprehensible explanations
48 T builds on student errors to help them overcome misconceptions
Cheng, Y. C., & Tsui, K. T. (1999). Multimodels of teacher effectiveness: implications
for research. Journal of Educational Research, 92(3), 141e150.
Cohen, D. K. (2011). Teaching and its predicaments. Cambridge, MA: Harvard Uni-
versity Press.
Cohen, D. K., Raudenbush, S. W., & Ball, D. L. (2003). Resources, instruction, and
research. Educational Evaluation and Policy Analysis, 25(2), 119e142.
Correnti, R., & Martinez, J. F. (2012). Conceptual, methodological, and policy issues
in the study of teaching: implications for improving instructional practice at
scale. Educational Assessment, 17(2e3), 51e61.
Creemers, B. P. M., & Kyriakides, L. (2008). The dynamics of educational effectiveness:
A contribution to policy, practice and theory in contemporary schools. London:
Routledge.
Creemers, B. P. M., & Kyriakides, L. (2010). Explaining stability and change in school
effectiveness by looking at changes in the functioning of school factors. School
Effectiveness and School Improvement, 21(4), 409e427.
Danielson, C. (2007). Enhancing professional practice: A framework for teaching (2nd
ed.). Alexandria, VA: Association for Supervision and Curriculum Development.
Donaldson, M. L., & Papay, J. P. (2012). Reforming teacher evaluation: One district’s
story. Retrieved 05.07.13 from http://www.americanprogress.org/issues/
education/report/2012/12/13/47662/reforming-teacher-evaluation-one-
districts-story/.
Douglas, K. (2009). Sharpening our focus in measuring classroom instruction.
Educational Researcher, 38(7), 518e521.
Doyle, W. (1983). Academic work. Review of Educational Research, 53(2), 159e199.
Edwards, A. L. (1957). The social desirability variable in personality assessment and
research. Ft Worth, TX, US: Dryden Press.
Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mah-
wah, NJ: Lawrence Erlbaum Associates, Inc.
van Es, E. A., & Sherin, M. G. (2008). Mathematics teachers’ “learning to notice” in
the context of a video club. Teaching and Teacher Education, 24, 244e276.
Goe, L., Bell, C., & Little, O. (2008). Approaches to evaluating teacher effectiveness: A
research synthesis. Washington, DC: National Comprehensive Center for Teacher
Quality.
Grossman, P., Hammerness, K., & McDonald, M. (2009). Redefining teaching, re-
imagining teacher education. Teachers and Teaching: Theory and Practice, 15(2),
273e289.
Grossman, P., & McDonald, M. (2008). Back to the future: directions for research in
teaching and teacher education. American Educational Research Journal, 45, 184e
205.
Hamre, B. K., Pianta, R. C., Downer, J. T., DeCoster, J., Mashburn, A. J., Jones, S. M.,
et al. (2013). Teaching through interactions: testing a developmental frame-
work of teacher effectiveness in over 4,000 classrooms. The Elementary School
Journal, 113(4), 461e487.
Hanushek, E. A. (1986). The economics of schooling: production and efficiency in
public schools. Journal of Economic Literature, 24, 1141e1177.
Hanushek, E. A. (2002). Teacher quality. In L. T. Izumi, & W. M. Evers (Eds.), Teacher
quality (pp. 2e12). Stanford, CA: Hoover Institution Press.
Hanushek, E. A., & Rivkin, S. G. (2010). Generalizations about using value-added
measures of teacher quality. American Economic Review: Papers and Pro-
ceedings, 100, 267e271.
 C.Y. Charalambous et al. / Teaching and Teacher Education 41 (2014) 22e33
Hattie, J. (2009). Visible learning: a synthesis of over 800 meta-analyses relating to
achievement. New York: Routledge.
Hill, H. C. (2010, May). The mathematical quality of instruction: Learning mathematics
for teaching. Paper presented at the 2010 annual meeting of the American
Educational Research Association, Denver, CO.
Hill, H. C., Blunk, M., Charalambous, C. Y., Lewis, J., Phelps, G. C., Sleep, L., et al.
(2008). Mathematical knowledge for teaching and the mathematical
quality of instruction: an exploratory study. Cognition and Instruction, 26,
430e511.
Hill, H. C., Charalambous, C. Y., & Kraft, M. (2012). When rater reliability is not
enough: teacher observation systems and a case for the G-study. Educational
Researcher, 41(2), 56e64.
Hill, H. C., & Grossman, P. (2013). Learning from teacher observations: challenges
and opportunities posed by new teacher evaluation systems. Harvard Educa-
tional Review, 83(2), 371e384.
Hill, H. C., Kapitula, L. R., & Umland, K. L. (2011). A validity argument approach to
evaluating value-added scores. American Educational Research Journal, 48(3),
794e831.
Horizon Research, I. (2000). Inside the classroom observation and analytic protocol.
Chapel Hill, NC: Horizon Research, Inc.
Kane, T. J., McCaffrey, D. F., Miller, T., & Staiger, D. O. (2013). Have we identified
effective teachers? Validating measures of effective teaching using random
assignment. Seattle: Bill & Melinda Gates Foundation. Retrieved 29.07.13, from
http://www.metproject.org/reports.php.
Kane, T. J., & Staiger, D. O. (2012). Gathering feedback for teaching: Combining high-
quality observations with student surveys and achievement gains. Seattle: Bill &
Melinda Gates Foundation. Retrieved 30.11.12, from http://www.metproject.
org/reports.php.
Kennedy, M. (2010). Teacher assessment and the quest for teaching quality. San
Francisco, CA: John Wiley & Sons.
Kilpatrick, J., Swafford, J., & Findell, B. (Eds.). (2001). Adding it up: Helping children
learn mathematics. Washington, DC: National Academy Press.
Klenowksi, V. (2012). Raising the stakes: the challenges for teacher assessment.
Australian Educational Researcher, 39(2), 173e192.
Kline, P. (1994). An easy guide to factor analysis. London: Routledge.
Konstantopoulos, S., & Chung, V. (2011). The persistence of teacher effects in
elementary grades. American Educational Research Journal, 48(2), 361e386.
Kyriakides, L., Christoforou, C., & Charalambous, C. Y. (2013). What matters for
student learning outcomes: a meta-analysis of studies exploring factors of
effective teaching. Teaching and Teacher Education, 36, 143e152.
Kyriakides, L., & Creemers, B. P. M. (2008). Using a multidimensional approach to
measure the impact of classroom-level factors upon student achievement: a
study testing the validity of the dynamic model. School Effectiveness and School
Improvement, 19(2), 183e205.
Kyriakides, L., Creemers, B. P. M., & Antoniou, P. (2009). Teacher behaviour and
student outcomes: suggestions for research on teacher training and profes-
sional development. Teaching and Teacher Education, 25(1), 12e23.
Kyriakides, L., & Demetriou, D. (2007). Introducing a teacher evaluation system
based on teacher effectiveness research: an investigation of stakeholders’ per-
ceptions. Journal of Personnel Evaluation in Education, 20(1e2), 43e64.
Kyriakides, L., Demetriou, D., & Charalambous, C. (2006). Generating criteria for
evaluating teachers through teacher effectiveness research. Educational
Research, 48(1), 1e20.
Kyriakides, L., & Tsangaridou, N. (2008). Towards the development of generic and
differentiated models of educational effectiveness: a study on school and
teacher effectiveness in physical education. British Educational Research Journal,
34(6), 807e838.
Lampert, M. (2010). Learning teaching in, from, and for practice: what do we mean?
Journal of Teacher Education, 61(1e2), 21e34.
Learning Mathematics for Teaching Project. (2011). Measuring the mathematical
quality of instruction. Journal of Mathematics Teacher Education, 14, 25e47.
Leinhardt, G. (1993). On teaching. In R. Glaser (Ed.), Advances in instructional psy-
chology (Vol. 4); (pp. 1e54). Hillsdale, NJ: Lawrence Erlbaum Associates.
McClellan, C., Donoghue, J., & Park, Y. S. (2013, April). Commonality and uniqueness in
teaching practice observation. Paper presented at the annual meeting of the
National Council of Measurement in Education, San Francisco, CA.
McGuinn, P. (2012). The state of teacher evaluation reform: State education agency
capacity and the implementation of new teacher-evaluation systems. Retrieved
30.10.13, from http://www.americanprogress.org/issues/education/report/2012/
11/13/44494/the-state-of-teacher-evaluation-reform/.
Measures of Effective Teaching Project. (2010). The PLATO protocol for classroom
observations. Retrieved 10.04.12, from http://metproject.org/resources/PLATO_
10_29_10.pdf.
Milanowski, A. (2004). The relation between teacher performance evaluation scores
and student achievement: evidence from Cincinnati. Peabody Journal of Edu-
cation, 79(4), 33e53.
Mitchell, R. N., Charalambous, C. Y., & Hill, H. C. (2014). Examining the task and
knowledge demands needed to teach with representations. Journal of Mathe-
matics Teacher Education, 17, 37e60.
Muijs, D., & Reynolds, D. (2000). School effectiveness and teacher effectiveness in
mathematics: some preliminary findings from the evaluation of the mathe-
matics enhancement programme (primary). School Effectiveness and School
Improvement, 11, 273e303.
33
Muñoz, M. A., Scoskie, J. R., & French, D. L. (2013). Investigating the “black box” of
effective teaching: the relationship between teachers’ perception and student
achievement in a large urban district. Educational Assessment, Evaluation and
Accountability, 25(3), 205e230.
National Council of Teachers of Mathematics. (2000). Principles and standards of
school mathematics. Reston, VA: Author.
Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects?
Educational Evaluation and Policy Analysis, 26(3), 237e257.
Ovando, M. (2001). Teachers’ perceptions of a learner-centered teacher evaluation
system. Journal of Personnel Evaluation in Education, 15(3), 213e231.
Palardy, G. J., & Rumberger, R. W. (2008). Teacher effectiveness in first grade: the
importance of background qualifications, attitudes, and instructional practices
for student learning. Educational Evaluation and Policy Analysis, 30, 111e140.
Panayiotou, A., Kyriakides, L., Creemers, B. P. M., McMahon, L., Vanlaar, G.,
Pfeifer, M., et al. (2013, April). Teacher behavior and student outcomes: Results of a
European study. Paper presented at the annual meeting of the American Edu-
cation Research Association, San Francisco, CA.
Pantic, N., & Wubbels, T. (2010). Teacher competences as a basis for teacher edu-
cation e views of Serbian teachers and teacher educators. Teaching and Teacher
Education, 26(3), 694e703.
Papay, J. P. (2012). Refocusing the debate: assessing the purposes and tools of
teacher evaluation. Harvard Educational Review, 82(1), 123e141.
Pashiardis, P., Savvides, I., & Tsiakkiros, A. (2005). The evaluation of teachers’ work
in Cyprus. In P. Pashiardis, J. Savvides, & A. Tsiakkiros (Eds.), The evaluation of
teachers’ work: From theory to praxis (pp. 27e38). Athens: Ellin Publishing
House.
Peterson, P. L. (1990). Doing more in the same amount of time: Cathy Swift.
Educational Evaluation and Policy Analysis, 12(3), 261e280.
Pianta, R. C., Belsky, J., Vandergrift, N., Houts, R., & Morrison, F. J. (2008). Classroom
effects on children’s achievement trajectories in elementary school. American
Educational Research Journal, 45(2), 365e397.
Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2006). Classroom assessment scoring
system: Preschool (pre-k) version (Vol. Manual). Charlottesville, VA: Center for
Advanced Study of Teaching and Learning.
Putnam, R. T. (1992). Teaching the ‘hows’ of mathematics for everyday life: a case
study of a fifth-grade teacher. The Elementary School Journal, 93(2), 63e176.
Rivers, J. C., & Sanders, W. L. (2002). Teacher quality and equity in educational
opportunity: findings and policy implications. In L. T. Izumi, & W. M. Evers
(Eds.), Teacher quality (pp. 13e23). Stanford, CA: Hoover Press.
Rockoff, J. E. (2004). The impact of individual teachers on student achievement:
evidence from panel data. American Economic Review, 94(2), 247e252.
Sawada, D., Piburn, M. D., Judson, E., Turley, J., Falconer, K., Benford, R., et al. (2002).
Measuring reform practices in science and mathematics classrooms: the
reformed teaching observation protocol. School Science and Mathematics, 102(6),
245e253.
Scheerens, J. (2013). The use of theory in school effectiveness research revisited.
School Effectiveness and School Improvement, 24(1), 1e38.
Schwab, J. J. (1978). Education and the structure of the disciplines. In I. Westbury, &
N. J. Wilfolk (Eds.), Science, curriculum, and liberal education: Selected essays (pp.
229e272). Chicago: The University of Chicago Press.
Seidel, T., & Shavelson, R. J. (2007). Teaching effectiveness research in the past
decade: the role of theory and research design in disentangling meta-analysis
results. Review of Educational Research, 77(4), 454e499.
Shulman, L. S. (1986). Those who understand: knowledge growth in teaching.
Educational Researcher, 15(2), 4e31.
Silver, E. A., Ghousseini, H., Gosen, D., Charalambous, C., & Strawhun, B. (2005).
Moving from rhetoric to praxis: issues faced by teachers in having students
consider multiple solutions for problems in the mathematics classroom. Journal
of Mathematics Behavior, 24, 287e301.
Stamelos, G., Vassilopoulos, A., & Bartzakli, M. (2012). Understanding the difficulties
of implementation of a teachers’ evaluation system in Greek primary education:
from national past to European influences. European Educational Research
Journal, 11(4), 545e557.
Stein, M. K., & Lane, S. (1996). Instructional tasks and the development of student
capacity to think and reason: an analysis of the relationship between teaching
and learning in a reform mathematics project. Educational Research and Eval-
uation, 2(1), 50e80.
Steiner, L. (2010). Using competency-based evaluation to drive teacher excellence:
Lessons from Singapore. Chapel Hill, NC: Public Impact.
Strong, M. (2011). The highly qualified teacher: What is teacher quality and how do we
measure it? New York, NY: Teachers College Press.
Stronge, J. H., & Tucker, P. D. (1999). The politics of teacher evaluation: a case study
of new system design and implementation. Journal of Personnel Evaluation in
Education, 13(4), 339e359.
Stronge, J. H., Ward, T. J., & Grant, L. W. (2012). What makes good teachers good? A
cross-case analysis of the connection between teacher effectiveness and stu-
dent achievement. Journal of Teacher Education, 62(4), 339e355.
Tuytens, M., & Devos, G. (2009). Teachers’ perceptions of the new teacher evalua-
tion policy: a validity study of the Policy Characteristics Scale. Teaching and
Teacher Education, 25(6), 924e930.
Weisberg, D., Sexton, S., Mulhern, J., & Keeling, D. (2009). The widget effect: Our
national failure to acknowledge and act on differences in teacher effectiveness.
Brooklyn, NY: New Teacher Project.