Students’ perceptions of pedagogy for 21st century learning instrument (S-POP-21): Concept, validation,

and initial results

Literature review

21st century skills and pedagogy

The need for students to learn and develop 21st century skills has long been advocated by industryand policy
makers alike. While the inclusion of 21st century skills has its critics, it has made its way into the education
policies of most countries around the world (Care et al., 2016). There is some contention regarding the most
suitable terminology – “21st century skills”, “key skills”, “future skills” or “transversal competencies” – and
what constitutes those skills. However, reviews in the area consistently highlight, ‘collaboration’,
‘communication’ ‘creativity’, ‘critical thinking’ ‘problem solving’, ‘self-direction’ and ‘technological fluency’
as the most prominent skills featured in frameworks such as P21’s Framework for 21st Century Learning and
the OECD (Dede, 2010; Voogt & Roblin, 2012; Care, Anderson & Kim, 2016; Kotsiou et al., 2022).

What is less clear is the extent to which teachers use 21st century pedagogies in the classroom. Early
expectations on what a classroom looked like and how children should learn created a mould that was difficult
to break in many countries for over 100 years. With the rise in popularity of constructivist and social
constructivist pedagogy in the mid-late 20th century, many countries experienced a paradigm shift from
teacher-centred to learner-centred pedagogic approaches. Making such a significant change within a workforce
takes time and thus, the use of learner-centred pedagogic approaches has varied significantly between and
within countries over the past 60 years. Its use tends to be more predominant in primary education than in
secondary where we have encountered the heaviest critics of the suitability of 21st century pedagogies due to
structural and curricular demands. While 21st century pedagogies do not represent such a seismic shift, and
broadly fit within social constructivist perspectives, they remain sufficiently different to current practice that it
is still reasonable to assume that there will be challenges in engaging teachers in the use of new pedagogic
approaches. This may be particularly notable for those who are well established as secondary school teachers,
given the oft over-crowded nature of curricula and the pressures of terminal examinations, which we have found
in our own experience in Ireland and is a common occurrence globally (OECD, 2018).

21st century pedagogies include problem-based learning, design thinking, bridge21 and dialogic teaching
(Savery & Duffy, 1995; Luka, 2014; Lawlor et al., 2018; and Teo, 2019). These are approaches that are rooted
in social constructionism but with a focus on developing skills as well as knowledge. However, many teachers
lack experience with these approaches, and while a few outstanding teachers will incorporate them into their
pedagogic repertoire, most are likely to struggle to adopt a skills-based approach without substantial support
and reform to the existing curriculum and approaches to assessment (Retna, 2016).
Without exposure to 21st century pedagogic activities, students will lack opportunities to develop corresponding
skills. However, given that it is broadly accepted that teachers use a variety of pedagogies from class-to-class
and within individual lessons (Husbands & Pearse, 2012) – essentially cherry picking from a range of
approaches they are familiar with – the question remains as to how best to determine the level of usage of 21st
century pedagogies in the classroom?

The importance of students’ perceptions

Observing teachers and students in the classroom, reviewing planning documents and asking teachers directly,
all provide practical routes to begin to explore whether and how teachers use 21st century pedagogies in the
classroom. Another approach is to ask students about their perceptions of pedagogy.

Understanding students’ perceptions of their learning environment provides an avenue to understanding the
opportunities for learning afforded to each student in a given class (Knight & Waxman, 1991). Although student
experience may not be consistent with the observations of external actors (researchers, inspectors, or even other
teachers), or the intentions of the classroom teacher, these perceptions reflect the range of ways in which
students experience classroom pedagogy (both instructional and environmental).

It has long been argued that students ultimately respond to what they perceive as important in education
(Schultz, 1979) with Knight and Waxman (ibid) arguing that how students perceive and react to specific
learning activities and general pedagogical approaches may be more important than the quality of teaching in
terms of influencing student outcomes. Accordingly, student voice has increasingly become recognised as an
important element in improving teaching and learning and it provides one route to understanding students’
perspectives on pedagogy. Internationally, student perspectives have been used to explore perceptions of the
4Cs - communication, collaboration, critical thinking and creativity skills (USA) (Landon, 2019); pedagogical
transitions during the Covid-19 global pandemic (Australia) (Khanal, 2021) and effectiveness of online classes
(India)(Vikas & Mathur, 2022). However, student voice is sometimes criticised for being superficial and
selective (McLeod, 2011). Notably, Fielding (2004)highlighted that, in a context of performativity and
accountability, students may push for more teacher-centred direct instruction. Nearly 20 years on from
Fielding's observations, within the context of secondary school, high-stakes terminal examinations, this is still
likely to be the case.

Existing scales
The assessment of 21st century skills has been a significant area of interest for some time but there are few
scales that measure students’ exposure to 21st century pedagogy. In the earlier part of the century, there was
considerable focus on digital competencies, also known as digital literacy and digital fluency, both of students
and their teachers. Research focused on teachers’ own digital skills and students’ assessments of the same,
without linking them necessarily to specific pedagogical approaches (such as Stefl-Mabry et al., 2010). As the
integration framework TPACK (Technological, Pedagogical and Content Knowledge) (Mishra & Koehler,
2006) gained popularity, several instruments were developed to assess school students’ perceptions of their
teachers’ technological knowledge and pedagogic integration of technology (such as Shih & Chung, 2013; and
Tseng, 2016). So, while there has been a growing number of validated scales in the area of digital skills, the
remining 21st century skills have often been largely overlooked.

Our broad review of the literature on 21st century skills and 21st century pedagogy (including terms such as
transversal competencies) found a dearth of quantitative instruments to measure students’ perceptions of 21st
century pedagogy in holistic terms. There are scales that measure teacher's and pre-service teacher's skills,
confidence and general attitudes towards 21st century skills in education, (e.g., Shidiq & Yamtinah, 2019; and
Sang et al., 2018) and their use of 21st century pedagogy in the classroom (e.g., Ravitz, 2014; and Sang et al.,
2018) from their own perspective.

The only validated instrument we found that specifically assesses students’ perceptions of 21st century
pedagogy was by Chai et al. (2015), which comprised of 7 scales: self-directed learning, collaborative learning,
meaningful learning with ICT, critical thinking, creative thinking, authentic problem-solving and knowledge
creation efficacy, with a total of 32 items. While this would not be a problem if the scale were standalone,
within our research context, there was a need for a short scale that could be integrated into a larger student
survey to be administered during the school day. However, one of the well-known challenges of conducting
surveys is survey fatigue, which has the potential to impact response behaviour. Therefore, it was necessary to
keep the number of items small. Additionally, Chai et al.’s (ibid) instrument was designed for primary school
children and validated in a single school. Having been validated in a school where there is a particular focus on
the use of technology to equip students with 21st century skills, means that it may not be a reliable tool outside
this context. Furthermore, the experience of primary school children is typically as part of one class who engage
all or most days with just one teacher, while secondary school students encounter several teachers each day and
are unlikely to even be with the same classmates as they move from lesson to lesson. To transpose the
instrument into a secondary context would require the instrument to be remapped and revalidated, but this
would still not address the issue of length. Finally, because of the focus on one school where pedagogic
practices were known, Chai et al.’s instrument was designed for knowledge construction (Scardamalia &
Bereiter, 2006) and items relating to communication were not included in the instrument. For the purposes of
our wider research agenda, we required an instrument that was broadly aligned to the key features of 21st
century pedagogies, rather than one specific approach.

Thus, the aim of this study was to develop a short, valid and reliable measure of secondary school students’
perceptions of their exposure to 21st century pedagogy.

Insight

The integration of 21st-century skills into education policies worldwide is widely acknowledged, emphasizing
collaboration, communication, creativity, critical thinking, problem-solving, self-direction, and technological
fluency. However, the adoption of 21st-century pedagogies, such as problem-based learning and design
thinking, varies among teachers, particularly in secondary education. Challenges arise due to entrenched
traditional teaching methods, crowded curricula, and examination pressures. While existing scales assess
teachers' perspectives on 21st-century skills, there's a gap in measuring students' perceptions of 21st-century
pedagogy. Developing a concise instrument aligned with key features of 21st-century pedagogies is essential for
understanding and enhancing students' learning experiences.

Conclusion

The text talks about the skills and teaching methods that are important in the 21st century education system.
These skills include working together, communicating, being creative, thinking critically, solving problems,
being independent and knowing how to use technology. Although many countries have included these skills in
their education policies, teachers use them differently in the classroom, especially in secondary education.
Teachers face challenges when it comes to implementing new teaching methods due to the requirements of the
system. It is important to understand how students feel about these teaching methods, as it can help improve
their learning. By asking students about their learning environment, we can discover the struggles and
opportunities they face in the classroom. Students' opinions are crucial in shaping the effectiveness of teaching
methods and influencing student outcomes. Unfortunately, there are not many ways to measure how much 21st
century teaching methods have been used. There are a few ways to determine how well students use technology
in the classroom, but no validated instruments to measure students' perceptions of 21st century pedagogy in a
holistic sense. There is a need for a brief and reliable method to measure high school students' exposure to 21st
century teaching methods.