See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/299850425

Holistic Assessment: Creating Assessment with Students

Chapter · January 2015

DOI: 10.1007/978-981-287-399-6_36

CITATION READS

1 25,376

2 authors, including:

Simon Williams
Higher Colleges of Technology
10 PUBLICATIONS   28 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

The Construction of the Refugee School as an Ethnic-Safe Space: Towards a Conceptualisation View project

An investigation of the role and function of learning centres for refugee communities in Malaysia. View project

All content following this page was uploaded by Simon Williams on 14 May 2018.

The user has requested enhancement of the downloaded file.

Holistic Assessment: Creating Assessment
with Students

Simon Williams and Lim Chong Hin

Abstract Assessment can be a complex area for both the student and teacher to
understand and use in the classroom. This study investigated the possibility of creat-
ing assessments with the students to reduce the subjectivity of it. A classroom of
lower intermediate students were involved in a project where they used a number of
alternative assessment methods, as well as dually created assessment method with
the teacher. The study takes an action research approach with observations from the
teacher as one of the sources of data. The study found that both the teacher and
student had different ideas on what should be assessed, prompting the teacher to
adjust the assessment criteria to more closely align with the expectations of both the
learner and assessor. The study then surmises that creating assessment with students
may reduce subjectivity and increase understanding on the part of the learner on the
assessment process. The paper finishes with suggestions and approaches on how
assessment can be created with the assistance of learners.

Keywords Alternative assessment • Holistic learning • Project-based learning •

Cocreated assessment

1 Literature Review

1.1 Holistic Learning

Holistic learning – though not defined as a method but a paradigm by Miller (1992) –
aims to address education in terms of the student as a whole. In comparison to what
is seen as ‘traditional education’ with a focus on meeting set standards and test tak-
ing, holistic learning promotes the student as an individual, with individual needs
and goals (Forbes 2003). Therefore, a standardised testing model creates a quandary

S. Williams (*) • L.C. Hin

Taylor’s University, Subang Jaya, Malaysia
e-mail: Simonjohn.williams@taylors.edu.my; ChongHin.Lim@taylors.edu.my

© Springer Science+Business Media Singapore 2015 389

S.F. Tang, L. Logonnathan (eds.), Taylor’s 7th Teaching and Learning
Conference 2014 Proceedings, DOI 10.1007/978-981-287-399-6_36
390 S. Williams and L.C. Hin

in holistic learning, as its very aim is to see education in a new light and to shed
away the past factory-like conditions of the classroom. To move learning in a more
holistic direction, alternative approaches are now being utilised such as project- and
problem-based learning. And, to align with these approaches, alternative assess-
ment methods are being used to gauge learning, such as portfolios, self-assessment
(SA) and peer assessment (PA), with a greater focus placed on formative means than
summative to assess learning outcomes.

1.2 Holistic Assessment

The literature on assessing holistic learning is limited which in part is due to the
wide variety of holistic initiatives available. This, according to Forbes (2003), ‘ham-
pers broader research’ (p. 3) and a deeper understanding as to what actually consti-
tutes holistic learning. Nevertheless, researchers have attempted to assign attributes
that can be considered holistic. Hare (2006) proposes a number of characteristics
associated with a holistic learner – they include: high achiever, belief in own self-
worth, compassion and loyalty (p. 308). This is just one interpretation of holistic
outcomes; however, assessing these values may prove to be more difficult in a class-
room setting.
To achieve a curriculum that moves away from a so-called traditional approach,
project- and problem-based learning and a system that employs portfolios to track
learning could be considered. To assess this kind of work, there are currently two
methodologies available: formative and summative. Summative assessment –
administered at the end of a semester in the form of a test or exam – usually assesses
knowledge acquired over the length of that course. Conversely, formative assess-
ment is a process and is usually not connected to a particular test. Instead, evidence
of student learning is collected and used to create a wider picture of the student’s
learning (McManus 2008). This method can be seen as more holistic compared to a
summative approach, as work is collected from various sources to help build a big-
ger picture of a student as a learner.
Two methods, in particular, which have shown promise by providing an alterna-
tive to standardised testing, are self- and peer assessment. Boud and Falchikov state
that self-assessment is how learners make judgements on what they are learning in
terms of success (1989). Peer assessment is a method by which peers can assess or
provide feedback on peer’s work. Assessing a holistic learning environment could
very well start with these two approaches as they, by their very nature, are more
inclusive whereby centralising the student in the assessment process instead of
being a passive reciprocation of it. The next section will look at how formative
assessment and more specifically SA and PA can be created holistically.
Holistic Assessment: Creating Assessment with Students 391

1.3 Creating Assessment Holistically

Assessment can be a contentious issue and some researchers have taken to pointing
out various contradictions. Reynolds and Trehan (2010) questions the illusion of
alternative methods of assessment. He says that even though SA and PA seem inclu-
sive, the assessment itself is created by the teacher. Thornbury (2013), then, affirms
that: ‘…it is the tester – not the test taking – who decides what counts as knowledge,
and how you count knowledge’. Reynolds & Trehan and Thornbury highlight the
notion that assessment, even assessment we deem to be more learner-friendly and
holistic, is in essence flawed, as in some cases it continues to be dominated by the
teacher and their view of what constitutes academic achievement.
To address the concerns, Heritage (2008) and McManus (2008) offer advice on
the creation of formative assessments, including SA and PA, and introduce the
notion of involvement. This refers to both learner and teacher being involved in the
creation of assessment and agreeing upon learning outcomes. They state that this
involvement creates transparency of the learning process, which allows the learners
to keep track of their own progress. To develop the idea of involvement further,
Stoynoff (2012) states that reliability with assessment can be increased when learn-
ers and teachers are trained in the design of it. This sentiment is also partially shared
by Ross (2006) and Chang et al. (2013) who also suggest that reliability can be
increased with training for both the learner and teacher, though they do not go as far
as to say the learners should be involved in the design.
In sum, creating assessment with learner involvement may create a stronger part-
nership with the teacher, which in turn shifts the paradigm of the classroom from the
learner and assessor to a mutual team with the same outcome. There is limited
research on involvement and its actual application in the classroom. Therefore, a
research needs to be conducted to investigate if learners and teachers actually agree
on what should be assessed. Furthermore, to investigate the benefits of such involve-
ment as it does involve extra work for both the teacher and learner, and finally to
research possible practical ways in which this process can be carried out.
To investigate the items mentioned above, three research questions were created
and they are:
• Do teachers and learners agree on what should be assessed?
• What are the benefits of creating assessment with learners?
• How can assessment be created with learners?

2 Methodology

An action research approach was taken for this study and adapted from a model by
Kemmis and McTaggart (2005). In addition to this model, the work of Jean McNiff
(2002) was also used as it provides practical tips for educational practitioners.
392 S. Williams and L.C. Hin

Fig. 1 Adapted Kemmis and McTaggart action research model

The original Kemmis and McTaggart model employed a three-step, two-cycle

framework. However, the adapted model in Fig. 1 increases the steps to five in order
to collect more data. The adapted model also employs a two-cycle system with the
second cycle in place to introduce improvements and suggestions made after the
first cycle. Action research can obtain data from a number of areas, though the two
main points in this study were observation and reflection.

3 Data Collection and Analysis

Fox (1998) ascertains that observation is not a matter of passively watching but a
method which engages the brain, eyes and ears, in order to understand what we are
seeing. Erickson, then, states that observational fieldwork can be beneficial when
there is a need to find out more about specific occurrences (1986). Dewalt and
Dewalt (2011) claim that observation can be a way to increase validity by helping
the research to understand the study in context.
Therefore, for this study, the researcher – also the teacher – made detailed notes
in line with Erickson’s (1986) recommendations for note taking during observations
and recorded for later analysis and reflection.
The reflective aspect of action research is important as it provides the research
with an avenue to critically reflect on practices in order to make improvements
(McNiff 2002). The role of reflection in research and its validity has been researched
and supported by a number of studies to date (Bretag 2004; Chamot et al. 1998). In
this study, the researcher reflected on the collected data and observations after each
completed cycle in order to make improvements for the following cycle and to cre-
ate an overall picture of the research.
Aside from observations and reflections as a data point, McNiff (2002) recom-
mends the creation of a ‘critical friend’. The purpose of the friend is to act as a
conduit for reflection. As the researcher discussed the research, the critical friend
offered a different viewpoint that may help the researcher to reflect more critically
on the research being carried out. In this research, the critical friend provided the
Holistic Assessment: Creating Assessment with Students 393

researcher with an additional point of reflection by providing insight and opinion on

the study which helped the researcher to re-examine areas of the research.
The participants in the study attended a private university in Malaysia. The par-
ticipants came from Korea, China, Vietnam and Malaysia and were attending a
6-week English preparatory course to gain English proficiency before beginning
foundation and degree courses. There were eight students in total and their ages
ranged from 18 to 24.

4 Data Analysis

This research sets out to answer three questions (below). In the next section, each
question will be addressed and answered with data from observations, reflections
and discussions with the critical friend:
1. Do teachers and learners agree on what should be assessed?
2. What are the benefits of creating assessment with learners?
3. How can assessment be created with learners?

4.1 Do Teachers and Learners Agree on What Should

Be Assessed?

Cycle one of the action research contained a rubric that was created entirely by the
teacher and without learner involvement. The rubric was written in English to suit
the level of the learners and uploaded to a class website to be viewed by the learners.
The learners were then given the outline of a simple project with the expected out-
comes. Though the teacher thought the rubric was clear and concise, he was con-
cerned that the level of student involvement was low. This concern was discussed
with the ‘critical friend’ who echoed the sentiments of Heritage and McManus to
involve the learners more heavily in the process.
Together the teacher and learner created a new rubric, and upon completion of
the second cycle, the researcher compared the rubrics from cycle 1 to 2. Figure 2
shows the rubric that was created solely by the teacher. However, when it is com-
pared to the rubric in Fig. 3, from the second cycle, the differences are apparent. In
Fig. 2, the teacher was focused on traditional written proficiencies such as structure,
grammar and punctuation. Conversely, in cycle 2 (Fig. 3), the students stated that
the weightage should be equal for each skill and, more interestingly, that ‘ideas’ and
‘page presentation’ should be included. In terms of weightage, the students sug-
gested that the original scores should be changed to even scores for each skill. The
teacher had originally intended to weight the easier skills such as vocab and
punctuation lower and to weight the more complex skills (structure, grammar,
sentence structure) higher. The students, however, thought differently.
394 S. Williams and L.C. Hin

Fig. 2 Cycle 1 rubric – teacher created

Fig. 3 Cycle 2 rubrics – teacher and student created

Holistic Assessment: Creating Assessment with Students 395

The class focused on projects as a method to teach writing. And to further engage
students in the writing process, the teacher created web-based writing projects
where the learners created web pages. In cycle 1, the teacher was more concerned
with specific writing outcomes and mostly ignored creative aspects of the task,
which, unbeknownst to him, seemed to have been important to the learners.
Therefore, when the learners were asked to be involved in the assessment, they
clearly stated that the creative aspect is an important area that should be taken into
consideration for assessment.

4.2 What Are the Benefits of Creating Assessment

with Students?

From question 1, the researcher established that in this study the teacher and learner
differed on items or skills that should be assessed. Question 2, though, asks what the
benefits are of doing this. Ross (2006) and Chang et al. (2013) both agree that with
training the validity of SA and PA can be increased. The training suggested involves
multiple attempts of the assessment in order for the learner to gain a deeper under-
standing of its use. So, by involving the learner in the actual creation of the SA or
PA, this may further increase the learner’s understanding of it as both the teacher
and learner share the same objectives. Moreover, this approach to co-assessment
creation also creates an environment of transparency where learners are more
informed as to how they are being assessed and on what skills specifically (Heritage
2008; McManus 2008). Finally, involving our learners in the assessment process
may create a more holistic classroom where there is a shared emphasis on learning.
Instead of a classroom that is knowledge based and fixed with a summative test,
more engaging learning environments can be created so that learning can take on a
more holistic feel and, more importantly, learners can be assessed on criteria which
they understand and are fully supportive of.

4.3 How Can Assessment Be Created with Learners?

On the completion of the action research, the researcher had used a number of sys-
tems to create assessment with the learners and had identified a number of common
issues that arose. On the first attempt, the researcher noted that most – if not all – of
students had no concept of assessment and how it was created. The teacher com-
mented how surprised he was by the blank stares that were thrown back at him when
he asked the students how they should be assessed on a task. This is an important
point to consider, as if the learners do not know how assessment is created, how are
they to know how it works or why it even exists? Very often the learners simply
asked the teacher to decide, a response that the researcher often noted down.
396 S. Williams and L.C. Hin

Therefore, if involving learners in assessment creation seems like an unusual

concept for the teacher, it will be equally or even more difficult for learners to
become accustomed to. Hence, time and training was needed for both the learner
and teacher to adapt to the new process.
As the assessment process was new to all the students in this study, the teacher
first carried out some simple activities or projects and provided the learners with a
simple self- or peer assessment, which were created by the teacher. By using the
teacher-created SA or PA, the students became accustomed to SA and PA all the
while familiarising them with the layout and items that can be assessed. The
researcher also discovered that it is useful to have multiple SAs and PAs so that the
learners see that various skills can be assessed. This essentially provides the learners
with a benchmark of what an SA or PA is like and how it can be used.
The assessments were completed as a class and in small groups. In this study, the
teacher used it as a task for the learners by asking them to produce a number of
items that should be assessed on the next project. With this list, the teacher included
the student-generated ideas in the rubrics. The teacher, of course, created the rubrics,
due to the lack of experience that the learners process with rubric creation.
Nevertheless, the items from the students were included, as well as items that the
teacher felt were important. Then to continue the assessment collaboration, the
teacher showed the students the rubric in the next lesson and asked them for feed-
back and comments. Finally, the weightage of each section was mutually agreed by
the teacher and the learners.

5 Conclusion

This study investigated the involvement of students as co-assessors in their learning

and found that for this study, there is a slight disagreement on the part of the teacher
and learner in terms of what should be assessed and the weightage attached to each
outcome. By using an action research model, the teacher/researcher was able to
reflect on the assessment used in the classroom and significantly alter the way in
which it was carried out by bringing the students into the process.
Though this study makes no claims to the wider educational community given
the small classroom-based nature of it, the researcher can conclude that for this
study, forming a partnership with the learners helped the class to form a shared view
of what should be assessed and how. More research, however, into the actual out-
comes of this approach will need to be conducted on a larger scale to discover if
learner/student-created assessments process improves learning outcomes or
achievement.
Encouraging learners to be involved in the creation of assessment could very
well help to create a more holistic learning environment, although, to effectively
involve the learners in the process of assessment creation, firstly the teacher must be
prepared to accept the learners as partners in the assessment process. When this hap-
pens, holistic learning can really take place, as assessment is the main driver behind
learning that decides the learning approach and desired outcomes of the class.
 Holistic Assessment: Creating Assessment with Students 397

References

Boud, D., & Falchikov, N. (1989). The role of self-assessment in student grading. Assessment and
Evaluation in Higher Education, 15(1), 101–111.
Bretag, T. (2004). Using action research to support International ESL students in information sys-
tems courses. International Education Journal, 5(4), 531–539.
Chamot, A., Barnhardt, S., & Dirstine, S. (1998). Conducting action research in the foreign lan-
guage classroom. In Northeast conference. New York: National Capital Language Resource
Centre.
Chang, C., Liang, C., & Chen, Y. (2013). Is learner self-assessment reliable and valid in a web-
based portfolio environment for high school students? Computers & Education, 60(1),
325–334.
Dewalt, K. M., & Dewalt, B. R. (2011). Participant observation: A guide for fieldworkers
(2nd ed.). Plymouth: Altamira Press.
Erickson, F. (1986). Qualitative methods in research on teaching. In Handbook of research on
teaching (3rd ed., pp. 119–161). Teaching, Institute for Research on Teaching, Michigan State
University. Retrieved from http://books.google.com/books?id=LGVHAAAAMAAJ&pgis=1
Forbes. (2003). Holistic education: Principals, perspectives and practice. Brandon: Foundation
for Educational Renewal.
Fox, N. (1998). Trent focus for research and development in primary health care: How to use
observations in a research project (pp. 1–31). Trent Focus.
Hare, J. (2006). Towards an understanding of holistic education. Journal of Research in
International Education, 5(3), 301–322. doi:10.1177/1475240906069453.
Heritage, M. (2008). Learning progressions: Supporting instruction and formative assessment
(pp. 1–31). Washington, DC: Council of Chief State School Officers (CCSSO).
Kemmis, S., & McTaggart, R. (2005). In N. K. Denzin & Y. Lincoln (Eds.), The sage handbook of
qualitative research (pp. 559–603). Thousand Oaks: Sage.
McManus, S. (2008). Attributes of effective formative assessment (pp. 1–6). Washington, DC:
The Council of Chief State School Officers (CCSSO).
McNiff, J. (2002). Action research for professional development: Concise advice for new action
researchers (3rd ed., pp. 1–42). Accessed at: http://jeanmcniff.com/ar-booklet.asp
Miller, R. (1992). In V. T. Brandon (Ed.), What are schools for? Holistic education in American
Culture (2nd ed.). Brandon: Holistic Education Press.
Reynolds, M., & Trehan, K. (2010). Assessment: A critical perspective. Studies in Higher
Education, 25(3), 267–278.
Ross, J. A. (2006). The reliability, validity and utility of self-assessment. Practical Assessment
Research & Evaluation, 11(10), 1–13.
Stoynoff, S. (2012). Looking backward and forward at classroom-based language assessment.
English Language Teaching Journal, 66(4), 523–532. doi:10.1093/elt/cco41.
Thornbury, S. (2013). O is for outcomes. An A-Z of ELT. Retrieved August 18, 2014, from
http://scottthornbury.wordpress.com/2013/02/10/o-is-for-outcomes/

View publication stats