Asia Pacific Journal of Education

ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/cape20

Assessment reform in Indonesia: contextual

barriers and opportunities for implementation

Mohammad Arsyad Arrafii

To cite this article: Mohammad Arsyad Arrafii (2021): Assessment reform in Indonesia:
contextual barriers and opportunities for implementation, Asia Pacific Journal of Education, DOI:
10.1080/02188791.2021.1898931

To link to this article: https://doi.org/10.1080/02188791.2021.1898931

Published online: 22 Mar 2021.

Submit your article to this journal

Article views: 97

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=cape20
ASIA PACIFIC JOURNAL OF EDUCATION
https://doi.org/10.1080/02188791.2021.1898931

Assessment reform in Indonesia: contextual barriers and

opportunities for implementation
Mohammad Arsyad Arrafii
Faculty of Culture, Management, and Business, Mandalika University of Education (Formerly Mataram Teacher
Training and Education Institute), Mataram, Indonesia

ABSTRACT ARTICLE HISTORY

This paper seeks to understand the contextual influences that were con­ Received 4 May 2020
sidered important for shaping teachers’ use of Assessment for Learning Accepted 28 February 2021
(AfL). Using the micro, meso and macro-level contextual framework, the KEYWORDS
content analysis of teachers’ propositions drawn from semi-structured AfL reform; learner agency;
interviews and three group discussions unveiled both internal and exter­ contextual influences; EFL
nal-to-school factors. Internally, the factors were associated with the teachers
students, teachers, resources at school and admission issues. A lack of
ongoing involvement and support from the government and parents was
mentioned as an external determinant. Besides considering these influ­
ences as barriers to innovation, teachers reported some potential oppor­
tunities that enabled them to perform assessment innovation, but
evidence for the barriers outweighed the opportunities, which led to the
perception that the reform was difficult to carry out. This implies that
teachers’ approach to the AfL initiatives would rely heavily on the shallow
approach of implementation. The study recommends that the incremen­
tal approach to reform implementation and sustainable professional
development would need to be promoted to address the implementation
barriers.

Introduction
Traditional views of learning assign a greater responsibility to teachers for designing, implementing
and assessing learning and instruction, while ostensibly limiting the active participation of learners.
The theorization of AfL clearly emphasizes the sharing of roles and labour among multiple agents
(student, teacher and peer) (Wiliam & Thompson, 2007) and holds that teaching and learning are
collaborative work undertaken by these agents to mitigate potential failure in terms of achieving the
intended outcomes (Black & Wiliam, 2009). It is the teacher’s responsibility to design and facilitate
effective and constructive learning environments through which the learners and their peers engage
and are responsible for their own learning and academic development (Black & Wiliam, 2009).
Student roles in AfL are central; they are the regulators of their own learning and the sources of
learning for others. Students should have a strong sense of autonomy to develop the skills and
capacity for reflective judgement (Earl & Katz, 2008, p. 90; Earl, 2003) to activate these roles. They
need to be active and self-regulated, and engage in communicating, justifyingand defending their
ideas (Maclellan, 2004) during the planning, task production and completion, and post-learning
activities. The curriculum reform (K13) in Indonesia expects teachers to change their assessment

CONTACT Mohammad Arsyad Arrafii arsyad.arrafii@ikipmataram.ac.id Faculty of Culture, Management, and Business,
Mandalika University of Education (Formerly Mataram Teacher Training and Education Institute), Pemuda Street No.59A,
Mataram, 83126 Indonesia
© 2021 National Institute of Education, Singapore
2 M. ARSYAD ARRAFII

practices from a content to an interactive, participative, and competence-based approach to

assessment, involving the active participation of students in the assessment as well as the involve­
ment of the wider school community (MoE, 2016a, 2016b). However, students’ active involvement in
assessment is often neglected in the classroom (Waters, 2009), even if it is strongly emphasized in the
curriculum reform.
The research identified a number of constraints that teachers encounter to promoting AfL
practices in class (e.g., Heitink, Van der Kleij, Veldkamp, Schildkamp, & Kippers, 2016; Herman,
Osmundson, Dai, Ringstaff, & Timms, 2015). These include constraints associated with teachers,
learners, assessment, and context (Heitink et al., 2016). Kozma (2003) and Fulmer, Lee, and Tan
(2015) developed a three-level conceptual framework of contextual factors (micro, meso and macro)
that influences educational practices. However, there is scarce published evidence of the use of this
framework to analyse contextual barriers and opportunities for educational reform implementation,
the report by Fulmer, Tan, and Lee (2019) being an exception. Likewise, far less is known about how
this area of interest is being explored in Indonesia. This paper seeks to clarify, from teachers’
subjective perspective, a range of contextual influences that teachers consider important for shaping
their uses of AfL.

AfL key strategies: the roles of learner and teacher

An initial theory of AfL consisted of five strategies: sharing success criteria with learners, strategic
questioning, comment-only marking, peer and self-assessment, and formative use of summative
tests (Black, Harrison, Lee, Marshal, & Wiliam, 2003; Wiliam, 2000, 2007). Wiliam and Thompson (2007)
synthesized research into AfL and then generated a model of implementation. They integrated
classroom activities into three assessment processes, with the aim of collecting evidence regarding
where the students are in their learning, what they need to achieve, and how to help them
accomplish their goals (Ramaprasad, 1983). They identified five key strategies of AfL, where teachers,
students and peers engage actively in classroom pedagogic activities(See Table 1).
Each strategy emphasizes the importance of interactive classroom dialogue among learning
agents in each stage of a learning episode to cultivate a “moment of contingency” (Black &
Wiliam, 2009, p. 10) in order to better understand pupils’ learning and help them act as the regulators
of the learning process. Sharing learning expectations and defining the success criteria are the initial
stage of classroom dialogue. This could be expanded through eliciting students’ personalized
learning targets and exploring their ideas about the quality criteria of a performance (Marshall &
Drummond, 2006). After the objectives have been set, understood and even agreed by the students
(Sadler, 1989), the teacher can use strategic questioning and feedback to seek evidence of students’
understanding and to close the gaps towards the learning goals. However, not all types of feedback
help close the gaps. Feedback is only meaningful when it triggers learners to think and reflect on
their learning and promotes learner agency (Black & Wiliam, 2018). Therefore, AfL requires teachers
to help learners to develop confidence, learning autonomy, and a sense of ownership and respon­
sibility for their learning. Then, teachers can ask students to act as a learning resource for one another
(peer assessment).
The application of these strategies in class is not without constraints. Contextual influences play
determinant roles in shaping whether teachers will use these strategies in class (Carless, 2011; Fulmer
et al., 2015; Kennedy, 2007; Pryor & Crossouard, 2008).

Internal and external-to-school influences on AfL practices

AfL is considered an innovation that requires careful consideration of internal and external-to-school
factors in order to be effective (Black & Wiliam, 1998; Fulmer et al., 2015; Heitink et al., 2016). These
contextual influences have been conceptualized into micro, meso and macro levels (Fulmer et al.,
2015; Kozma, 2003). Teachers’ practices of AfL vary depending on the density of the constraints in
 ASIA PACIFIC JOURNAL OF EDUCATION 3

operation (Heitink et al., 2016; Pryor & Crossouard, 2008; Van der Kleij, Vermeulen, Schildkamp, &
Eggen, 2015).
The micro-level factors in Kozma (2003) framework relate to the classroom context, and include
personal factors relating to the teacher and student (e.g., assessment literacy, conceptions, beliefs
and attitudes), social factors and classroom organization (e.g., the nature of the teacher–student
interaction in class, and the role division between the teacher and students), and learning resource
factors (e.g., access to learning resources and tools that support practices: technology and teaching
aids). The meso level encompasses factors beyond the classroom that directly influence classroom
practices. These factors include school policies and support, school climate and access to profes­
sional learning. The macro-level influences are those linked to the broader context of the education
system that indirectly influence classroom practices through the mediation of the meso-level factors.
The macro factors can include education policies issued by the district, state and national authorities,
cultural norms and values that are practised in the context.
This contextual framework can be considered comprehensive; however, the links between, and
the interplay of influences among the levels in regard to teachers’ assessment practices are beyond
examination. Departing from Kozma (2003) model, Fulmer et al. (2015) reviewed the literature on
assessment practices and explained the relationship among the micro, meso and macro-level factors.
The review concludes that the contextual factors at each level affect and shape teachers’ assessment
knowledge, beliefs, views, values and practices. Nevertheless, tension often arises between micro
factors (e.g., teachers’ aspirations to carry out a more holistic and authentic approach to assessment)
and factors at the macro-level, e.g. the nationwide testing and accountability pressure from society
(Fulmer et al., 2019). Moreover, the mismatch between government policies and established prac­
tices in school often diminishes the effective use of AfL (Sach, 2015).
A review of the AfL literature indicated a number of prerequisite categories for effective AfL
practices, including teachers, students, assessment, and context (Heitink et al., 2016). Among these,
teacher factors were found to be the main determinant, replicating the prominence of individual
factors in affecting assessment practices (Fulmer et al., 2015). This suggests the crucial role of
teachers as a locomotive of change: the success or failure of AfL practices relies on teachers’ ability
and willingness to change, which are determined by their content and pedagogical knowledge of
the change (Herman et al., 2015) and the availability of the educational resource at schools. In
addition, in a reform context like Indonesia, teachers often misinterpret assessment policies and are
forced to follow a seemingly paradoxical assessment policy, e.g. while teachers are mandated to use
multiple assessment methods in class, the time allocated for conducting the assessment is limited
(Lumadi, 2013). Heitink et al. (2016) argue that to be able to promote effective AfL, teachers need to
acquire expertise and skills in regard to creating an educational situation within which they can elicit
students’ thinking and reasoning, foster academic participation and intensive discussion, interpret
information about student learning and use it accordingly for improvement. Teachers should have
a sound literacy and understanding of the content and presentation of the assessment (e.g., effective
delivery of feedback) (Herman et al., 2015), the connection and alignment between the broad
curriculum and the instructional tasks (Popham, 2011), and the educational technology related to
the assessment (Rudner & Schafer, 2002), and should develop a sense of confidence in their
judgement of student learning (Heitink et al., 2016). Furthermore, teachers need to be open-
minded and demonstrate a strong teaching commitment to applying the underlying principles of
AfL and promoting a greater degree of student involvement and agency in assessment processes
(Heitink et al., 2016). However, although teachers are well equipped with advanced assessment
literacy and skills, their classroom practices are largely shaped by the intensity of the influences of
the three-level contextual factors (Fulmer et al., 2015). For example, teachers who have
a sociocultural philosophy of teaching and assessment will emphasize interactive learning situations
by promoting dialogue and collaboration but may not use such practices when external pressures
exist or when resources are inadequate.
4 M. ARSYAD ARRAFII

Table 1. Agents, aspects and stages of AfL.

where the learner is going
Teacher 1 Clarifying learning intentions and
sharing criteria for success
Understanding and sharing learning
Peer intentions and sharing criteria for
success
Learner Understanding learning intentions and
criteria for success
(Wiliam & Thompson, 2007)
where the learner is now how to get there
2 Engineering classroom discussions, 3 Providing feedback that
questions and tasks that elicit evidence of moves learning
learning forward
4 Activating students as instructional resources for one another
5 Activating students as the owners of their own learning

Given such complex, hierarchical and dimensional influences of contextual factors on teachers’
uses of AfL and that the intensity of the contextual influences on assessment practices may vary
among different educational contexts, a case, contextual exploration of the challenges and oppor­
tunities for AfL practices is a legitimate area for study. This paper seeks to explore, from the
perspective of teachers in Indonesia, the extent to which contextual factors at the three levels
shape teachers’ practices of assessment, guided by the following research question: What kinds of
internal and external-to-school influences do Indonesian teachers consider important for shaping
their use of AfL? It is important to address this issue as part of an effort to support the implementa­
tion of AfL reform in Indonesia through delving into teachers’ own propositions regarding the
contextual challenges and opportunities they experienced with the reform mandates, a topic that
requires much deeper attention. This can also be a kind of evaluation towards the reform imple­
mentation from the teachers’ perspective, which is often neglected in a top-down reform imple­
mentation context. Moreover, the study will help amplify teachers’ voices regarding their concerns
about the reform initiatives.

Methodology
The evidence reported here is part of the findings of a larger mixed-methods study of EFL secondary
school teachers’ conceptions and practices of AfL in Indonesia (Arrafii, 2020b). This paper reports on
teachers’ perspectives about the contextual influences, which is one of the main findings that
emerged in the main mixed-methods study.

Participants
Semi-structured interviews were employed to gather data from 15 teachers working in different
types of secondary schools in Lombok, Indonesia. These participants were selected purposively
from 343 teachers, who participated in the first quantitative phase of the study, based on
a number of teacher and school criteria (e.g., teaching experience, school level, location, and
size) to reveal a variety of perspectives regarding contextual influences on the reform imple­
mentation and to ensure a balance of participation. The interviewees comprised five teachers
working in junior secondary, another five in senior secondary, and another five in vocational
schools (See Table 2).
The semi-structured interviews were followed by three group discussions (GD) to explore the
broader perspective of English teachers’ thinking regarding the challenges and opportunities for the
implementation of AfL practices. Each group consisted of six or seven participants (19 teachers in
total), recruited through a kind of snowball sampling strategy by which key contact teachers invited
one or two other participants. This strategy was used due to the challenge regarding teachers’
commitment to attend the meeting, even if they had firmly accepted an invitation and confirmed
their participation. However, although this strategy ensured attendance, it yielded participants from
relatively similar individual and school backgrounds; the sample was dominated by male teachers
 ASIA PACIFIC JOURNAL OF EDUCATION 5

Table 2. Profile of semi-structured interview participants.

No Name (pseudonym) Sex School level Teaching experience (year) School location
1 Devi Female Junior high school 15 Rural
2 Kasul Male Junior high school 35 Urban
3 Elaine Female Junior high school 33 Urban
4 Pujika Male Junior high school 8 Urban
5 Ritama Female Junior high school 8 Rural
6 Melda Female Senior high school 21 Rural
7 Jupris Male Senior high school 33 Rural
8 Madan Male Senior high school 13 Rural
9 Sabari Male Senior high school 24 Urban
10 Munah Female Senior high school 34 Rural
11 Muhis Male Vocational school 8 Urban
12 Mumun Male Vocational school 12 Rural
13 Wayan Male Vocational school 36 Rural
14 Lia Female Vocational school 3 Urban
15 Fathul Male Vocational school 15 Rural

Table 3. Profile of group discussion members.

Group No Name (pseudonym) Sex Occupational status School level Years of teaching School location
1 1 Dedi Male Civil servant Junior 12 Rural
2 Rani Female Civil servant Junior 11 Rural
3 Dar Male Civil servant Senior 12 Rural
4 Andre Male Civil servant Senior 9 Rural
5 Heru Male Civil servant Senior 9 Rural
6 Yulia Female Volunteer Vocational 9 Rural
2 7 Mari Male Civil servant Junior 9 Rural
8 Idar Female Civil servant Junior 10 Rural
9 Dat Male Volunteer Junior 9 Rural
10 Nias Female Volunteer Senior 12 Urban
11 Zaini Male Civil servant Vocational 26 Rural
12 Rakab Male Civil servant Vocational 9 Rural
13 Subhan Male Civil servant Vocational 8 Rural
3 14 Eful Male Volunteer Junior 9 Rural
15 Nas Male Volunteer Junior 8 Urban
16 Suudi Male Volunteer Junior 22 Rural
17 Tomi Male Volunteer Senior 4 Rural
18 Adila Male Volunteer Senior 13 Rural
19 Rozika Male Civil servant Senior 8 Rural

and those working in rural areas (see Table 3). Like the interview participants, teachers in the group
meeting were also participants in the survey phase of study.

Data analysis method

The process of analysing the interview and group meeting transcripts was inductive and interpretive
using the content analysis approach, which strives “to preserve the richness of textual interpretations”
(Krippendorff, 2013). According to Cohen, Manion, and Morrison (2011, p. 564), there are three essential
features of the content analysis process, including “breaking down text into units of analysis, under­
taking statistical analysis of the units, [and] presenting the analysis in as economical a form as possible”.
Denscombe (2007, p. 237) describes a more detailed procedure for using content analysis:

(a) Choose an appropriate sample of texts or images

(b) Break the text down into smaller component units.
(c) Develop relevant categories for analysing the data.
(d) Code the units in line with the categories.
(e) Count the frequency with which these units occur.
6 M. ARSYAD ARRAFII

(f) Analyse the text in terms of the frequency of the units and their relationship with other units
that occur in the text.

(See also Flick, 2009)

Following such a procedure, the research question guided and framed the initial sweeps through
the transcripts, which were carefully read and annotated to become familiar with the data. The
iterative sweeps of the raw data revealed a large number of initial codes that could be grouped into
a category or a “typology” through examining the relationship among them (Denscombe, 2007,
p. 297). Colour coding was used to differentiate these categories and initial codes associated with
each of them. Then, all of the categories were grouped based on the theoretical codes (Thornberg &
Charmaz, 2014, p. 159).
Furthermore, to make the data analysis more manageable, the analysis of each broad category
was conducted separately through placing and dividing all of the propositions into their subcate­
gories. I counted the number of sub-categories and computed their percentages in order to display
the frequency of the subcategories occurring in the transcripts. For this report, the analysis focuses
on the category of barriers and opportunities. However, the counting and percentage procedures of
the content analysis approach were not applicable or relevant to the group discussion due to its
interactive nature, which allowed one idea to be uttered or expressed several times by different
individuals. Thus, the focus of the analysis of the group discussion was thematic ideas from the
interactive conversation among the group members.
When analysing each transcript, codes and categories within and between transcripts were
compared, and a coding framework was developed during the analysis and interpretation processes,
which was then refined to maintain consistency of the coding system across the transcripts to arrive
at a more accurate synthesis of teachers’ ideas.

Findings
The content analysis identified internal and external-to-school influences on AfL practices. Internally,
these influences can be grouped into student, teacher, resource, and school admission categories.
Externally, a lack of participation by parents and a lack of continuous control from the government
were perceived to be barriers to innovative assessment practices. Opportunity-related propositions
were also identified. However, the barriers outweighed the opportunities.

Internal-to-school influences
Table 4 displays the categories of barriers and the number of teacher propositions associated with
each category. Among the reported barriers, the main concern was resource issues, which accounted
for 45.16% of the propositions. This was followed by concerns relating to student readiness for
innovation, which counted a slightly higher number of propositions (28.49%) than concerns about
teacher readiness (23.66%). Ignorance of external-to-school stakeholders in educational processes
received the smallest number of propositions, less than 3%.

Table 4. Number of propositions related to perceived barriers and opportunities for AfL practices.
Teachers
Category A1 A2 A3 A4 A5 B1 B2 B3 B4 B5 C1 C2 C3 C4 C5 Sum
Resources 8 2 5 5 1 4 3 8 4 4 3 9 4 0 24 84
Student readiness 11 0 3 2 0 5 1 4 0 3 4 2 1 10 7 53
Teacher readiness 1 3 0 1 1 2 0 3 3 0 6 0 1 1 22 44
Stakeholder ignorance 1 0 0 0 0 0 0 0 0 0 0 0 0 0 4 5
Total 21 5 8 8 2 11 4 15 7 7 13 11 6 11 57 186
 ASIA PACIFIC JOURNAL OF EDUCATION 7

Resource issues
Among the perceived influences on AfL practices, resources were considered the main concern of
teachers. This was expressed in 84 propositions, but one respondent accounted for 24 of these
propositions (29%). The resource-related propositions mainly related to classroom resources and
professional training issues. The resource category encompassed positive and negative experiences
with the learning facilities, the well-being of the teachers, the support that teachers received and the
impact of these issues on their aspirations to improve the learning process. However, on balance, the
impact on teachers was more discouraging than encouraging, leading to the view that they did their
job with limited support and resources.
Starting from the downside perspective, the majority of the participants reported that they
worked at a school with limited learning resources available to support learning, e.g. a lack of
textbooks, audio and video teaching aids, and access to the internet. They rated their school
resources as rudimentary, consisting of basic textbooks, and standard portable electronic equip­
ment (e.g., tape and audio cassettes). Two teachers reported that although their schools pro­
vided a computer laboratory, they made little use of the computer as it was not set up for
language learning [Mumun]. Another teacher stated that the computer laboratory in her school
was like “a cemetery” because it had never been used by English teachers to aid learning
[Ritama]. This may be indicative of a lack of agency or initiative on the part of teachers or
poor resource allocation.
Support in terms of professional development for teachers to implement the reform was claimed
to be inadequate. Although all of the interviewees had attended professional training, they
expressed a couple of concerns. The first concern was linked to equality and equity issues. They
stated that opportunities to attend professional development were not equally provided among
teachers, as the statement below exemplifies.

I think professional training for teachers should be conducted more frequently because recently the procedure
[to determine who to attend teacher professional training] is the officials’ privileges. I hope this is not of my
prejudice. Only those who are ‘close’ to education officers were invited to attend the training. We [who do not
have network in the education bureaucracy] have had little access on training information . . . I myself attended
only two teacher trainings, one was conducted in this school [participant’s school] and the other one was in
Mataram. Never have I attended professional training outside Lombok. It is always other teachers who were sent
[to training outside Lombok]. I do not know why [they did not pick teachers in our school] as it is just a few
meters from the state education office [Munah].

This proposition agreed with the findings in the group discussions, which revealed that many
teachers had missed the opportunity to attend professional training. However, some had taken the
initiative to learn about the concepts and application of the reform from self-directed reading and
conversations with colleagues who had attended the training [GD1].
Secondly, the teachers who had attended professional training felt disappointed with the training
organization, as it was assumed to have involved corruption and manipulation. For instance,
teachers were invited to attend a 5-day training course, but the training was intensified and took
only 3 or 4 days, focusing on theoretical rather than practical aspects of the reform and assessment
[GD2]. In addition, teachers in GD3 reported that they had attended a kind of reform socialization
rather than professional development.

I attended the teacher training in [one of the leading schools] in Praya. What we did in the training was sitting
down, listening to a speech, signing attendance, and done. There was no pedagogic [input] and K13 content
training, it is just a kind of K13 socialization, but in an invitation letter, it says [in the letter] that it is a K13 training.
This happened also in [training for] other curricular subjects. We came, signed [the attendance], and went home.
However, the facilitator asked teachers to use K13 assessment in the class. How can we do that? That’s why we
find K13 assessment is complicated as we never attend an appropriate training [GD3]

Teachers hoped for a situation in which teachers’ professional training would be more accoun­
table and not involve any kind of corruption.
8 M. ARSYAD ARRAFII

Thirdly, teachers noticed various levels of understandings of the reform held by curriculum
trainers, which often conflicted with each other, thereby affecting the quality of the training delivery.
According to the teachers, professional training should focus on a specific subject, facilitated by
a subject-specialist trainer.
Other challenges for teachers to apply assessment practice [in K13] is lack of specific assessment and subject-
specific training. It is commonly found that teachers with different educational backgrounds and expertise were
trained together and the facilitators illustrated the concept and practices of assessment based on their own
background. I attended K13 assessment several times, but all instructors were from science-related subjects. We
need English-focused assessment training, with English specialist tutor [Muhis]

This view was confirmed and agreed on by group participants, who asserted that the training
topics were often redundant and irrelevant, yielding poor outcomes. One of the group discussion
members stated that:
In the professional training we attended, they [tutors] gave us the training materials that were mostly associated
with natural science subjects and the trainers were science subject specialists. We have asked the trainers this
issue, but the trainers said, “This is the kind of training materials we received in the training of trainers [TOT] in
Jakarta”. [As a result] We attended the training, but we don’t think we have a better understanding of K13
implementation afterwards. Sometimes, we get bored attending the training because we received the repeated
topics and materials. There seems no improvement in our understanding of K13 implementation [GD1]

Whilst teachers held negative impressions towards internal, government-initiated training, three
teachers expressed positive views and experiences with the training provided by external agencies
and described it as meaningful and inspiring [Devi, Kasul and Fathul]. One teacher in the vocational
school reported positive outcomes from the training organized by an external agent, stating that:
It [training provided by Global Partner] covered many issues, starting from the process of handling students’
learning, from the very beginning process of instruction, designing and preparing lesson plans, implementation
of lesson plans, evaluation of lesson plans, designing tasks and instruction, assessment process, time manage­
ment, handouts and exercise sheets, use of teacher notes, classroom management, testing: oral and written,
feedback to students etcetera. In sum, the training was very detailed and comprehensive. I keep the training files
and handouts at home. It is a very clear process for handling individual students, when and how to deliver
feedback, when to give tests, very clear . . . Except with Global Partner, I never attended such detailed and
inspiring assessment training [Fathul].

Another prominent perceived barrier to innovation related to teachers’ financial welfare and well-
being. While workloads were relatively equal between public and non-public teachers, a huge
disparity in terms of their earnings persisted. Whilst public teachers secured their incomes on
a monthly basis and most of them taught in one school, their non-public counterparts strived to
earn more through seeking additional teaching roles in multiple schools, despite the fact that these
additional workloads did not significantly address the income disparity, as non-public teachers were
paid very low wages. Given such a situation, teachers argued that teachers’ salaries, especially for
those with non-government official status, were a significant problem in Indonesia [Wayan, GD1, and
GD2]. Some informants expressed that to increase the quality of teaching, teachers’ salaries must be
raised, although not all teachers agreed about the positive correlation between a high salary and the
quality of teaching and outcomes [e.g. Fathul].

Student readiness
Student readiness refers to teachers reporting positive and negative perspectives and experiences
with the students’ learning motivation, courage and curiosity, and their level of knowledge and
abilities in English. From a negative point of view, four teachers claimed that most students had a low
level of learning motivation, describing them as lazy and with a poor interest in learning English
[Devi, Melda, Muhis and Lia]. Teachers defined lazy as inadequate level of classroom engagement:
a low rate of attendance, task and homework incompletion; a lack of learning initiative, passivity, and
 ASIA PACIFIC JOURNAL OF EDUCATION 9

feeling discouraged [Devi, Melda, Mumun, and Lia]. Group discussion members agreed that students’
motivation for learning English was relatively low, and grounded strongly on external rather than
internal motivation: “They learn English because they have to [because] English is a compulsory, pre-
requirement subject for graduation and is one of the subjects tested in national examination” [RK-
GD2]. One teacher in a professional school evaluated students’ motivation to pursue education,
stating that: “school [may be perceived] like a jail by students, as if home time is freedom, an escape
from learning tasks” [Muhis]. Teachers also believed that most students had a poor interest in
English, even when native English speakers were invited as tutors to teach in the class [DAR and
AN, GD1].
However, as the teachers acknowledged, a small number of students demonstrated a high level of
learning motivation, showing a commitment to learning as opposed to the negative behaviours
described above. One teacher in GD1 praised some highly motivated individuals for their learning
and described how students strived to attend the class: “Many of my students live far away from
school. One of them leaves for school at 6 am, because he walks to school every day” [YL-GD1].
Student readiness was also attributed to student courage and curiosity in the learning process.
Several teachers described their students as shy, stating that they lacked confidence in demonstrat­
ing academic performance and avoided being active and assertive in the learning process, e.g. active
involvement in dialogue and argumentation around their learning [Devi, Elaine and Pujika]. One
teacher argued that this characteristic stemmed from a fear of making mistakes (a factor associated
with language anxiety), as this, when it happened, was embarrassing for students [Elaine]. For many
students, according to one teacher, being quiet was a safe behaviour to avoid embarrassment
[Muhis]. In addition, limited comprehension of vocabulary items inhibited learners in regard to
demonstrating their speaking performance, which meant that they could not express their thinking
in English in a longer response (GD1).
However, despite these negative experiences, teachers reported evidence of high curiosity and
resilience among students in expressing their ideas and argumentation. One teacher labelled this
kind of student as “talkative”, stating that they “annoyingly kept voluntarily answering questions
although their answers were wrong” [Devi]. Moreover, when asked to give an argument, reason­
ing and explanation in Bahasa Indonesia, a number of students could perform this task well [HR-
GD1]. This finding indicates that it was not cognitive abilities that were the main concern of
teachers; rather, it was the means through which students can convey their thinking that was
inadequate.
For this reason, teachers in the group discussions, especially those who taught at senior second­
ary schools, questioned the effectiveness of English instruction in junior high schools and argued
that although students had studied English for three years in junior secondary, the majority were not
equipped with adequate capabilities, skills and knowledge in English [GD1 and GD2]. Consequently,
senior secondary teachers admitted that they often went back to teaching the very basic learning
topics, such as personal pronouns and to be, to their students.

Teacher readiness
Teacher readiness can be defined as teachers reporting positive and negative perspectives about
their levels of assessment and technological literacy, pedagogic commitment, and reluctance to
change their practices. Almost all of the teachers confessed that they lacked a conceptual and
practical understanding of assessment and some based their assessment practices on the traditional
approaches regardless of the pressure from the curriculum to employ a holistic and authentic
assessment framework [Kasul, Jupris, Madan and Fathul]. Teachers were not confident with their
abilities to develop assessment rubrics and encountered difficulties in developing a valid and reliable
assessment instrument [Devi, Kasul, Wayan and Fathul]. Teachers also admitted that their under­
standing and abilities to run learning processes, in which learning autonomy, self and peer assess­
ment are facilitated, were not developed [Melda and Devi].
10 M. ARSYAD ARRAFII

Regarding the assessment reform, respondents believed that many teachers did not have a clear
understanding about the concept and application of the reform. For example, teachers were still
confused about how to integrate the assessment of spiritual aspects of learning into the language
assessment framework [GD1]. Furthermore, teachers were not literate in terms of using technology
although the curriculum asks teachers to engage in technology-aided instruction. Consequently,
many teachers were dependent on other teachers who were more technologically literate to cope
with the technological issues. As well as teachers, school administrators also struggled with using
computer applications to support teachers in resolving their problems in using technology in assess­
ment [GD2]
Teachers raised concerns about their own pedagogic competence. One teacher in the group
discussion [GD2] proposed that, if students must achieve a certain threshold to be considered
competent, a threshold for teachers is also needed. Although other group members disagreed
with this, the teacher argued that the government should test teachers: those who did not pass
the teaching threshold scores should not be allowed to teach [GD2].
Besides pedagogic competence and assessment literacy, teachers proposed that teachers’ com­
mitment to innovative teaching and assessment was questionable. One claimed that:

It is not the teachers’ competence that needs upgrading, most teachers are already competent, a lot of them are
master degree holders, but we need to upgrade the teachers’ teaching commitment. We have a deficit of
teachers’ integrity . . . indeed do our teachers have commitment to improving the quality of instruction and
learning? It is a serious question [Fathul].

A lack of teaching commitment brought other consequences such as teachers’ ignorance about the
quality of students’ learning. In such a situation, teaching becomes routine, artificial and less meaningful:
Nowadays, I think, we, most teachers, do not prepare and plan our lesson well. We attend the classroom, teach
the lesson, and finish the learning materials; our target is just to finish the learning materials. We do not really
care about student understanding [Fathul].

These admissions show us the extent to which training for teaching is not adequate or well
focused to equip teachers with an understanding of learning development.

Admission issues
Student recruitment and admission issues were perceived to be other barriers to innovative
classroom assessment, although the influences were considered indirect. However, the admis­
sion issue was only evident in the group discussion, not in the individual interviews. Teachers
reported an overwhelming workload as they taught parallel, large class sizes with mixed
abilities – beginner to intermediate – as the admission procedures did not follow rigorous
selection criteria, nor were students grouped based on their English ability levels. Teachers
connected this perceived barrier with the availability, accessibility and quality of learning
resources in their schools, which did not support the development of student-centred, technol­
ogy-aided instruction. Teachers advised the schools to apply admission selection criteria more
rigorously to ensure that new students meet the academic requirements for acceptance.
However, tensions appeared as teachers were aware that these aspirations often run against
school policies and interests, e.g. accepting all student candidates in order to secure a greater
amount of financial support from the government [GD1], as the support from the government is
provided based on the population of students at the school. Additionally, schools and principals
considered it important to accept a huge number of students to guarantee a minimum of 24
teaching loads a week for teachers, in order to enable them to receive financial certification
benefits. The following conversation from the group discussion [GD1] illustrates this situation.
Researcher : In order to facilitate students to be able to talk in English, what support do you need?
 ASIA PACIFIC JOURNAL OF EDUCATION 11

DD: I think school facilities, learning resources, book, and dictionary. Around 20% of funding aids
from government can be used to provide and enrich the learning resources.
HH: I think the curriculum of English needs to be reviewed because I think there are some over­
lapping in the curriculum for junior and senior secondary schools. The majority of my students in
senior high school have no ability in English at all, although they have been learning English for
3 years in junior high school level.
DD: Support from parents are also very limited
HH: I think we need to do something like a placement test to check student ability in English as part
of the admission system
DW: However, schools MUST accept all student candidates regardless of their ability
YY: If we [schools] do not accept students, we [schools] are labelled [by society] as a racist, ignorant
school
HH: But schools are also competing with each other to recruit as many students as they could,
because, if a school is able to recruit a lot of students, the school could earn a great amount of
financial aid from government.
DW: If schools do not accept a huge number of students, teachers’ might lose their certification
allowance, because teachers can’t meet the minimum requirement in terms of teaching hours.
Accepting a huge number of students will secure the compulsory 24 hours a week of teaching
loads
This extract illustrates a very complex situation whereby there are internal- and external-to-school
factors and societal pressure and interest in relation to admission systems, which are not easy for
teachers and schools to resolve.

External-to-school influences
Besides internal influences, teachers reported that their practices of assessment were influenced by
the degree of involvement and commitment of education stakeholders, such as the government and
parents, in facilitating education, but there were only a few propositions associated with these
factors, expressed by only two interviewees. This suggests that these perceived influences did not
significantly affect teachers’ assessment thinking and practices. One teacher from a vocational school
was very critical regarding the government’s commitment to quality improvement in secondary
education and urged that a strong intervention should be applied in regard to disciplining teachers
with a lack of teaching commitment.
I expect that there is a strong commitment and intervention from government to ensure that the learning
processes at school meets an adequate standard from planning, process and evaluation of student learning. This
needs to be conducted continuously and tightly controlled. If needed, teachers who have shown a poor
teaching commitment can be given a serious sanction. Government must take serious control over this.
I don’t see serious commitment from government to tackle this issue. What government does now is just
APPEALING to teachers to provide excellent teaching, that’s it, no further action. Appealing is not enough.
Government must give heavy penalty for naughty teachers. But it is not happening now and as a result, teachers
neglect innovative practices and are resistant to change but maintain old practices. Government needs to be
more firmed and demanding [Fathul].

Teachers also claimed that the lack of ongoing support from parents was another challenge, as
parents were perceived to be ignorant about their children’s learning development and to hold
conventional and negative views about education [GD1]. For some parents, letting their children
attend education was a progressive move on their part [GD3]. In such a case, instead of giving
support to their children in learning, children often had restricted access to resources for learning at
12 M. ARSYAD ARRAFII

home. Some participants in the group discussion [GD2 and GD3] argued that parents’ ignorance
stemmed from economic disabilities and their values regarding education in general.

Discussion
This paper explored perceived contextual influences (barriers and opportunities) regarding the
implementation of the new Indonesian curriculum, which requires teachers to make a greater use
of AfL in class (MoE, 2016a, 2016b), which is collaborative, qualitative and responsive to learners’
perceptions of their own needs and preferences (Black & Wiliam, 2009; James, 2006; Wiliam &
Thompson, 2007). However, teachers’ responses to this requirement indicated resistance.
A number of perceived barriers that caused teachers’ assessment practices to remain unchanged
or at least move very slowly towards making the change were identified. Teachers expressed that
their practices of assessment were contextually dependent on a number of influencing and predict­
ing factors, including inside and outside-school influences (Izci, 2016). The internal-to-school factors
encompassed individual teacher and student-related influences, the support received at school, and
issues related to student recruitment and admission processes. The external-to-school factors
included a lack of ongoing government monitoring of the changes and inadequate parental
involvement and support for children’s learning. These internal and external factors replicate the
micro, meso and macro-level influences (Fulmer et al., 2015; Kozma, 2003). However, respondents in
this study emphasized the micro and meso-level as the strongest influencers of their assessment
practices, which concurs with the international literature (Fulmer et al., 2015). Two variables at the
micro-level, teachers and students, and two meso factors (resource constraints and admission issues)
significantly influenced why and how teachers selected and assessed learning.
The AfL literature has documented that students and teachers are the main prerequisite for
effective AfL practices (Fulmer et al., 2015; Heitink et al., 2016). However, in Indonesia, the change to
AfL practices has been hindered rather than promoted due to the embedded negative conceptions
related to teachers’ and students’ readiness to change, as argued by several researchers (e.g., Izci,
2016; Kennedy, 2007). Teacher readiness in this study was associated with a lack of content and
pedagogical competencies, skills and assessment literacy among teachers (Herman et al., 2015),
which led teachers to think that they were unable to immediately change their established peda­
gogic practices to AfL. In addition, teachers reported some propositions that indicated poor teacher
commitment to student learning (Frelin & Fransson, 2017) as a barrier to AfL work. This perceived
incompetency and lack of commitment were intensified by inadequate learning resources at school,
e.g. materials and funding (Dufresne, Gerace, & Mestre, 2011) and limited support for teachers in
terms of opportunities to attend effective professional development programmes, which were
believed to be an effective tool to scale up teachers’ knowledge of, and performance in using AfL
strategies (Birenbaum, Kimron, & Shilton, 2011; Black et al., 2003). In this study, teachers who had
attended training expressed dissatisfaction about the outcomes and the impacts of the training on
their pedagogic knowledge and classroom practice. This reaction among teachers could have been
disempowering, which explains their lack of cognitive participation and collective action in regard to
the AfL reform (Arrafii, 2020b; May & Finch, 2009).
In addition to teachers’ readiness, teachers seemed to disobey AfL practices because they had
concerns regarding students’ readiness and capacity to get involved actively and independently
as learning assessors and regulators. Most students lacked motivation and suffered from “lan­
guage anxiety” (Horwitz, 2010), as they might have had a negative experience of learning
English and viewed learning a foreign language as difficult (Lamb, 2002). This caused the
teachers to experience difficulties getting them through the prescribed curricula (Arrafii,
2020a). Teachers also reported that students in their classes had a mixture of English abilities,
a challenging situation for the implementation of student-centred learning and assessment
approaches, where interactive and dialogic discourses in class are strongly promoted (Black &
Wiliam, 2018). In mixed ability classes with some strong students, anxious learners may lack
 ASIA PACIFIC JOURNAL OF EDUCATION 13

opportunities or be reluctant to demonstrate their communicative participation (Anandari, 2015;

Horwitz, 2010; Marwan, 2008). In this particular context, establishing AfL or any kind of innova­
tive practice that demands active participation of students requires not only appropriate
techniques but also adjustments to teaching methods, focusing on affective factors such as
reducing anxiety and engaging learners in a supportive environment where errors are under­
stood to be part of the learning process (Rach, Ufer, & Heinze, 2013; Tulis, Steuer, & Dresel, 2016;
Arrafii, 2021), and pedagogic skills in terms of how to deal with the learners’ errors (Seifried &
Wutkke, 2010). If a safe behaviour is to remain silent, encouraging learners to actively engage in
learning is challenging, which subsequently may reduce opportunities to learn. More impor­
tantly, EFL teachers need to “reflect on and develop teaching techniques to suit real classrooms,
not techniques developed for classrooms in distant lands” (Holliday, 1994, p. 9) and design
classroom tasks that suit the capacity of their learners. These will help increase learners’
motivation.
Considering students’ motivation, in many EFL and ESL contexts where the purpose of
language learning is institutional rather than personal or even instrumental, what Adrian
Holliday (1994) called TESEP (Teaching English for State Education Purposes), the problem with
students’ learning motivation is that “the learner is [. . .] an almost robotic entity whose sole
purpose for being in the classroom seems to be to learn language” (Holliday, 1994, p. 8). Extrinsic
rather than intrinsic motivation and anxiety about external-related factors were prominent
motivations reported by teachers in this study, whereby students were forced to learn English
at school, as was also found in other studies e.g. Takahashi (2018). A lack of intrinsic motivation
may explain the phenomenon that despite the fact that Indonesian students have learned
English from junior secondary to higher education level as a compulsory subject, their English
performance remained at an “elementary” level (Lamb, 2000), except among a small number of
learners who demonstrated learning resilience and an adequate level of English performance
and competence (Lamb, 2002). In Lamb’s study, the success of this minority group of learners
seemed to be the result of a greater self-motivation, self-reinforcement and learning manage­
ment by the students themselves in their social life beyond the school, rather than resulting from
the services provided by educational institutions (Lamb, 2002). If the out-of-school practice of
independent learning helps improve performance (Smith, Kuchah, & Lamb, 2018), this highlights
the significance of promoting “in-class” autonomous learning practice in language learning
(Benson, 2007), which has been seen as important but unfortunately has often been abandoned
(Arrafii, 2020b).
School policy, the meso-level influence, also affects teaching and assessment practices (Fulmer et al.,
2015; Izci, 2016). Less stringent criteria and the absence of a reliable placement test for student
recruitment had caused more and more students to be accepted. As a result, there were discrepancies
between school resources and capacities that significantly influenced the way teachers managed their
teaching and assessment practices. Teachers maintained an old-fashioned assessment strategy. Teachers
trying to implement pedagogic and assessment innovations (e.g., independent learning or group work)
reported that these activities lacked preparation and did not work effectively (Arrafii, 2020b). Moreover,
teachers who worked under the provision of grades and an “all students should pass” policy tended to
assign passing grades, despite the fact that the students were low achievers (Arrafii, 2020a)
Beyond the internal influences, external-to-school barriers persisted. Teachers stated that govern­
ment monitoring of classroom practices was insufficient and brought no deterrent effect for “unethi­
cal” teachers. Likewise, the support and involvement of parents in education were argued to be
minimal. Financial insecurity among parents and relatively low education qualifications contributed to
ignorance about their children’s education, resulting in “your mum and dad can’t teach you”
responses (Lamb, 2013) from parents. For some parents, formal education might be considered to
be exclusively the responsibility of the school. This magnifies the importance of all education
stakeholders, including parents, understanding, being involved in, and promoting the suggested
14 M. ARSYAD ARRAFII

innovations. The active participation of external-to-school stakeholders will stimulate collective mon­
itoring of the reform implementation (May & Finch, 2009) and give embedding a greater chance.

Conclusion
A number of contextual influences that present barriers and opportunities were considered influen­
tial for shaping teachers’ thinking regarding the use of AfL. Insufficient school resources and support
for teachers were prominent. Access to training was patchy and, in many cases, reported to be
arbitrarily distributed. Teachers’ AfL practices were also influenced by student and teacher readiness
regarding innovative ideas, as teachers and students suffered from a lack of assessment literacy,
motivation and commitment. The admission system in schools was also considered a problem. A lack
of stringent assessment criteria and an absence of reliable placement tests for school admission
resulted in an enormous enrolment and a greater mixture of English abilities among students,
a situation that was argued to be less supportive for a student-centred pedagogy and assessment.
Beyond internal influences, external-to-school barriers also persisted. Education stakeholders lacked
involvement and support. These contextual influences led teachers to believe that the reform was
difficult to carry out. There is a danger that when teachers are mandated to change, but without
adequate resources and support, their implementation will be patchy and superficial. The study
suggests that taking into account the readiness of regions across Indonesia, the incremental
approach, rather than the concurrent approach (as applied recently), to the reform implementation
seems to be more appropriate and thus should be undertaken. The incremental approach should be
supplemented by accountable and sustainable professional development specific to AfL, preceded
by a precise needs analysis of teachers and students and consideration of contextual influences.

Disclosure statement
No potential conflict of interest was reported by the author.

Funding
This work was supported by the Endowment Fund for Education (LPDP).

Notes on contributor
Mohammad Arsyad Arrafii, a PhD holder in the field of language assessment, is currently a lecturer and researcher at
Mandalika University of Education (UNDIKMA). He was an English teacher in the secondary level for 7 years before
embarking into the higher education sector. His research interest includes classroom assessment, curriculum and
educational evaluation, language pedagogy, learner autonomy and self-regulation.

ORCID
Mohammad Arsyad Arrafii http://orcid.org/0000-0003-4441-243X

References
Anandari, C.L. (2015). Indonesian EFL students’ anxiety in speech production: Possible sources and remedy. TEFLIN
Journal, 26(1), 1–16.
Arrafii, M. A. (2020a) Grades and grade inflation: exploring teachers' grading practices in Indonesian EFL secondary
classrooms. Pedagogy, Culture & Society, 28(3), 477–499
Arrafii, M. A. (2020b) Towards formative assessment: exploring English teachers' conceptions and practices of assessment in
Indonesia. Ph.D. Dissertation: University of Leicester, UK.
 ASIA PACIFIC JOURNAL OF EDUCATION 15

Benson, P. (2007). Autonomy in language teaching and learning (state of the art article). Language Teaching, 40(1),
21–40.
Birenbaum, M., Kimron, H., & Shilton, H. (2011). Nested contexts that shape assessment for learning: School-based
professional learning community and classroom culture. Studies in Educational Evaluation, 37(2011), 35–48.
Black, P., Harrison, C., Lee, C., Marshal, B., & Wiliam, D. (2003). Assessment for Learning: Putting it into practice. New York:
Open University Press.
Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7–74.
Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and
Accountability, 21(1), 5–31.
Black, P., & Wiliam, D. (2018). Classroom assessment and pedagogy. Assessment in Education: Principles, Policy & Practice,
25(6), 551–575.
Carless, D. (2011). From testing to productive student learning: Implementing formative assessment in confucian-heritage
setting. London: Routledge.
Cohen, L., Manion, L., & Morrison, K. (2011). Research methods in education (7th ed.). London: Routledge.
Denscombe, M. (2007). The good research guide: For small-scale social research projects (3rd ed.). Buckingham, UK: Open
University Press.
Dufresne, R.J., Gerace, W.J., & Mestre, J.P. (2011). Assessing-to-learn formative assessment materials for high school physic.
(UNCG Technical Report No. 2011-002). University of North Carolina. Retrieved from http://libjournal.uncg.edu/ojs/
index.php/UNCGTechRpt/article/viewFile/289/125
Earl, L. (2003). Assessment as learning: Using classroom assessment to maximize student learning. Experts in Assessment
series. Thousand Oaks, California: Corwin Press, Inc.
Earl, L., & Katz, S. (2008). Getting to the core of learning: Using assessment for self-monitoring and self-regulation. In
S. Swaffiled (Ed.), Unlocking assessment: Understanding for reflection and application, 90–104) London: Routhledge.
Flick, U. (2009). An introduction to qualitative research (4th ed.). London: Sage Publications.
Frelin, A., & Fransson, G. (2017). Four components that sustain teachers’ commitment to students - a relational temporal
model. Reflective Practice, 18(5), 641–654.
Fulmer, G.W., Lee, I.C.H., & Tan, K.H.K. (2015). Multi-level model of contextual factors and teachers’ assessment practices:
An integrative review of research. Assessment in Education: Principles, Policy & Practice, 22(4), 475–494.
Fulmer, G.W., Tan, K.H.K., & Lee, I.C.H. (2019). Relationships among Singaporean secondary teachers’ conceptions of
assessment and school and policy contextual factors. Assessment in Education: Principles, Policy & Practice, 26(2),
166–183.
Heitink, M.C., Van der Kleij, F.M., Veldkamp, B.P., Schildkamp, K., & Kippers, W.B. (2016). A systematic review of
prerequisites for implementing assessment for learning in classroom practice. Educational Research Review, 17
(2016), 50–62.
Herman, J., Osmundson, E., Dai, Y., Ringstaff, C., & Timms, M. (2015). Investigating the dynamics of formative assessment:
Relationships between teacher knowledge, assessment practice and learning. Assessment in Education: Principles,
Policy & Practice, 22(3), 344–367.
Holliday, A. (1994). The house of TESEP and the communicative approach: The special needs of state English language
education. ELT Journal, 48(1), 3–11.
Horwitz, E.K. (2010). Foreign and second language anxiety. Language Teaching, 43(2), 154–167.
Izci, K. (2016). Internal and external factors affecting teachers’ adoption of formative assessment to support learning.
International Journal of Social, Behavioral, Educational, Business and Industrial Engineering, 10(8), 2541–2548.
James, M. (2006). Assessment, teaching and theories of learning. In G. John (Ed.), Assessment and learning,47–60.
London: SAGE Publication.
Kennedy, K.J. (2007). Barriers to innovative school practice: A socio-cultural framework for understanding assessment
practices in Asia. Paper presented at the redesigning pedagogy- culture, understanding and practice conference.
Nanyang Technological University, National Institute of Education, Centre for Research in Pedagogy and Practice.
Kozma, R.B. (2003). ICT and educational change: A global phenomenon. In R.B. Kozma (Ed.), Technology, innovation, and
educational change: A global perspective (pp. 1–18). Eugene: International Society for Technology in Education.
Krippendorff, K. (2013). Content analysis: An introduction to its methodology (3rd ed.). London: SAGE.
Lamb, M. (2000) Can we change our students’ learning behaviour? Paper presented at Conference on Education in
Indonesia at the University of Leeds, Leeds, UK
Lamb, M. (2002). Explaining successful language learning in difficult circumstances. Prospect, 17(2), 35–52.
Lamb, M. (2013). ‘Your mum and dad can’t teach you!’: Constraints on agency among rural learners of English in the
developing world. Journal of Multilingual and Multicultural Development, 34(1), 14–29.
Lumadi, M.W. (2013). Challenge besetting teachers in classroom assessment: An exploratory perspective. Journal of
Social Sciences, 34(3), 211–221.
Maclellan, E. (2004). Initial knowledge states about assessment: Novice teachers’ conceptualizations. Teaching and
Teacher Education, 20(5), 523–535.
Marshall, B., & Drummond, M.J. (2006). How teachers engage with assessment for learning: Lessons from the classroom.
Research Papers in Education, 21(2), 133–149.
16 M. ARSYAD ARRAFII

Marwan, A. (2008). The exploration of factors triggering foreign language anxiety: Learners’ voices. TEFLIN Journal, 19(2),
119–126.
May, C., & Finch, T. (2009). Implementing, embedding, and integrating practices: An outline of normalization process
theory. Sociology, 43(3), 535–554.
MoE. (2016a). Peraturan Kementrian Pendidikan dan Kebudayaan Republic Indonesia Nomor 23 Tahun 2016 tentang
Standar Penilaian Pendidikan Indonesia. (The Ministry of Education’s Regulation No. 23, Year 2016 about Assessment
Educational Standards). Jakarta: Ministry of Education Republic of Indonesia.
MoE. (2016b). Peraturan Kementrian Pendidikan dan Kebudayaan Republic Indonesia Nomor 22 Tahun 2016 tentang
Standar proses Pendidikan Indonesia. (The Ministry of Education’s Regulation No. 22, Year 2016 about The Standards
of Educational Process in Primary and Secondary Education). Jakarta: Ministry of Education Republic of Indonesia.
Popham, W.J. (2011). Assessment literacy overlooked: A teacher educator’s confession. The Teacher Educator, 46(4),
265–273.
Pryor, J., & Crossouard, B. (2008). A socio-cultural theorisation of formative assessment. Oxford Review of Education, 34(1),
1–20.
Rach, S., Ufer, S., & Heinze, A. (2013). Learning from errors: Effect of teachers’ training on students’ attitudes towards and
their individual use of errors. PNA, 8(1), 21–30.
Ramaprasad, A. (1983). On the definition of feedback. Behavioural Science, 28(1), 4–13.
Rudner, L., & Schafer, W. (2002). What teachers need to know about assessment. Washington, DC: National Education
Association.
Sach, E. (2015). An exploration of teachers’ narratives: What are the facilitators and constraints which promote or inhibit
‘good’ formative assessment practices in schools? Education, 3–13(43), 322–335.
Sadler, R.D. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119–144.
Seifried, J., & Wutkke, E. (2010). Student errors: How teachers diagnose and respond to them. Empirical Research in
Vocational Education and Training, 2(2), 147–162.
Smith, R., Kuchah, K., & Lamb, M. (2018). Learner autonomy in developing countries. In A. Chik, N. Aoki, & R. Smith (Ed.),
Autonomy in language learning. Basingstoke: Palgrave Macmillan. doi:10.1057/978-1-137-5299-5_2
Takahashi, T. (2018). Motivation of students for learning English in Rwandan schools. Issues in Educational Research, 28
(1), 168–186.
Thornberg, R., & Charmaz, K. (2014). Grounded theory and theoritical coding. In U. Flick (Ed.), The SAGE handbook of
qualitative data analysis (pp. 153–170). London: SAGE Publications.
Tulis, M., Steuer, G., & Dresel, M. (2016). Learning from errors: A model of individual process. Frontline Learning Research,
4(2), 12–26.
Van der Kleij, F.M., Vermeulen, J.A., Schildkamp, K., & Eggen, T.J.H.M. (2015). Integrating data-based decision making,
assessment for learning and diagnostic testing in formative assessment. Assessment in Education: Principles, Policy &
Practice, 22(3), 324–343.
Waters, A. (2009). A guide to methodologia: Past, present, and future. ELT Journal, 63(2), 108–115.
Wiliam, D. (2000). Formative assessment in mathematics part 3: The learner’s role. Equals: Mathematics and Special
Educational Needs, 6(1), 19–22.
Wiliam, D., & Thompson, M. (2007). Integrating assessment with instruction: What will it take to make it work? In C.
A. Dwyer (Ed.), The future of assessment: Shaping teaching and learning (pp. 53–82). Mahwah, NJ: Lawrence
Erlbaum.
Wiliam, D. (2007). Keeping learning on track: Classroom assessment and the regulation of learning. In F.K. Lester Jr. (Ed.),
Second handbook of mathematics teaching and learning (pp. 1053–1098). Greenwich, CT: Information Age.