Applied Measurement in Education

ISSN: 0895-7347 (Print) 1532-4818 (Online) Journal homepage: http://www.tandfonline.com/loi/hame20

st
21 Century Skills: What Are They and How Do We
Assess Them?

Kurt F. Geisinger

st
To cite this article: Kurt F. Geisinger (2016): 21 Century Skills: What Are They and How Do We
Assess Them?, Applied Measurement in Education, DOI: 10.1080/08957347.2016.1209207

To link to this article: http://dx.doi.org/10.1080/08957347.2016.1209207

Accepted author version posted online: 06

Jul 2016.
Published online: 06 Jul 2016.

Submit your article to this journal

Article views: 37

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=hame20

Download by: [Nanyang Technological University] Date: 11 August 2016, At: 19:32
 1

21st Century Skills: What Are They and How Do We

Assess Them?
Kurt F. Geisinger

t
Buros Center for Testing, University of Nebraska-Lincoln

ip
Contact: Kurt F. Geisinger, kgeisinger2@unl.edu, University of Nebraska, Lincoln, Buros Center

cr
for Testing, 21 Teachers College, Lincoln, NE 68588-0353.

us
Abstract

One of the “hottest” topics in the educational measurement community in the past decade has
an
been the assessment of 21st Century skills. This special issue demonstrates work being
M
performed in this realm. The present article provides a context for the four primary papers that

follow and a brief but broad view of some models of 21st century skills. The articles that follow
ed

are active projects developing assessments that focus on the changing cognitive skills required

for this new century. It is clear that education that has emphasized memory and routine problem
pt

solving needs a metamorphosis and the measures described in this issue portend a brave new

educational world.
ce
Ac
 2

The present issue of Applied Measurement in Education is devoted to an important and

timely topic, one that has generated considerable thought and hypothesizing, but far less

empirical research. This issue is devoted to four research studies and summaries of research with

the addition of some commentaries on the assessment of 21st Century Skills in general and these

four articles. Unlike the purported Y2K dilemma (the event that was supposed to adversely

t
ip
affect computers all over the world at midnight on December 31, 1999) that was expected to

crash many computers, the need for 21st Century Skills is real, and it requires a more gradual

cr
change; one that obviously does not indicate a dramatic metamorphosis between those skills

us
needed to succeed in 1999 and those in 2000, 2001, or even 2020. Nevertheless, we are tapping

into relatively new skills in the 21st century and those skills, 21st century skills, must be built into
an
curricula, taught, and assessed in education and elsewhere.

Several proposed systems have been advanced for organizing the new and changing
M
knowledge and skills. This brief introduction mentions only a couple of such systems and
ed

attempts to categorize the papers in this special issue within at least one of those paradigms.

More specifically, this issue of the journal focuses almost exclusively on the assessment of some

four types of 21st Century Skills: collaborative problem solving, complex problem solving,
pt

creativity, and digital and information literacy. The assessment of 21st Century skills is just one
ce

piece of the analysis of these skills and abilities. One must identify such skills, define them,

organize different 21st Century skills into coherent systems, develop, and adapt curricula to
Ac

include such skills in addition to those continuing from previous centuries, consider instructional

approaches to teaching such skills, develop actual instruction and instructional materials to

encompass 21st Century skills contemporaneously to beginning to assess them. From an

educational and curricular perspective, no one seems to be suggesting that the traditional
 3

curricular foci (e.g., reading, writing, mathematics, history, science, and so forth) should be

eliminated, but the primary papers composing this issue have implications for curricular changes.

To some extent they track the history and logic of the skills being assessed from formulation

through their increased importance in the modern world and their assessment. Evidence of the

quality of their assessment is provided in each instance as well.

t
ip
Some 21st Century Skills and Their Organization

cr
The emphasis in 21st century learning and assessment includes but also goes beyond the

basics of reading, writing, interpretation, and synthesis. One approach to organizing 21st Century

us
skills focuses on cognitive skills, intrapersonal skills, interpersonal skills, and technical skills

(Ananiadou & Claro, 2009); hereafter called the OECD approach. Cognitive skills include,
an
among others, non-routine problem solving, critical thinking, and systems thinking.
M
Intrapersonal skills involved a group of metacognitive skills: self-management, time

management, self-development, self-regulation, adaptability, and executive functioning.

ed

Interpersonal skills consist of complex communication, social skills including collaboration,

teamwork, cultural sensitivity, and dealing with diversity. Technical skills are primarily focused
pt

upon research and information fluency skills as well as entrepreneurial skills and financial

literacy.
ce

From a different perspective, meeting the demands of education, work, and commerce in

the 21st century necessitates additional layers for communication skills, technological savvy,
Ac

global view, collaborative practices, digital skills, and more innovative applications rather than

memorization. Communication skills have obviously changed. The curricula of the last century

could not have anticipated the rapid advancement of cellular technology, capacity, and

proliferation across the world community, or that the internet would make global
 4

communications virtually instantaneous as well as inexpensive. Expertise has also become so

specialized that teams working together must solve many problems; no one expert has the skills

to consider, much less solve, most problems. Therefore, collaboration skills, teamwork skills,

and cross-cultural sensitivity are needed as components of problem solving. Indeed, a world that

is changing faster and faster requires proactive problem solving and entrepreneurial skills for

t
ip
people to adapt quickly and successfully. The burden is not exclusive to either education, work,

cr
or commerce, but rather, success depends on a symbiotic relationship between educators and the

forces of work and commerce.

us
Some systems portraying 21st Century skills include financial and entrepreneurial skills.

The need for other 21st Century skills may not always be as obvious as the need for skills relating
an
to problem solving, collaboration, and digital proficiency. Those in education, especially in the

United States, are generally aware of the entrepreneurial skills of Bill Gates, Steve Jobs, and
M
countless others to lesser extents, but one may ask what has changed in the world that now
ed

suggests that educated people must all know about finances. One example in the United States

may be the change from balancing a checkbook to dealing with on-line banking. Another typical
pt

but not so simple US example might provide some guidance here. At one time, many if not most

employees in the United States expected to retire from a company after many years work and to
ce

receive a defined-benefit pension. That is, they knew if they worked a certain number of years at

given salaries, what their monthly pension would be. Today, most employees have defined-
Ac

contribution pensions. The difference is that employees now know how much money is entered

into their program each month or year by the employer and themselves, but not what they will

ultimately receive. In many cases they must make decisions on where their funds are invested so

that they maximize their retirement accounts. Another difference now is that many employees do
 5

not work their entire careers in the same company, or from a set location. Some find that they

must coble together or rollover their different retirement plans to maximize their benefits. This

financial example segues into another set of important skills: global or international skills. Thus,

financial skills are needed today that simply were not required when many employees worked

their entire careers for a paternalistic company that provided define benefit retirement plans.

t
ip
New financial and problem-solving skills are needed to plan for retirement.

cr
Most professionals today grew up with some knowledge of their local and national

economies. However, we have moved to a world economy; national economies are simply

us
nested within this world economy. Before the day begins on the US markets, commentators

inform the public on what has happened previously on the Asian and European markets. Not
an
only must international business people and professionals know about international

developments, but they must know something about the customs and beliefs in the different
M
cultures. This kind of information and skill is needed if one’s intercultural communications are
ed

to be successful and if one is desirous of being well received in the particular culture with which

one is working.
pt

Framing 21st Century Skills

An OECD framework organizes 21st century skills and competencies for new learners
ce

deemed vital for young people to have in order to be effective workers and citizens in the
Ac

knowledge societies of their time (Ananiadou & Claro, 2009). Cognitive skills, intrapersonal

skills, interpersonal skills, and technical skills are conceptualized within three overarching

constructs or dimensions: information – as source and as product; communication –written,

spoken, virtual, art, collaboration, and using information communication technology (ICT); and
 6

ethics and social impact – social responsibility, critical thinking, decisions / judgment, and social

awareness.

In a brief editorial at the beginning of a special issue of The Science Teacher devoted to

21st Century skills, Metz (2011) presented the following listing of skills: core subject matter

knowledge, especially in science; flexibility, adaptability, and innovation; critical thinking,

t
ip
creativity, and non-routine problem solving; complex communication, collaboration, social and

cr
cross-cultural skills; self-direction, productivity, and accountability; and systems thinking.

Metz’s listing parallels that of Ananiadou and Claro (2009). The Metz skills and abilities can

us
also be conceptualized as cognitive skills, intrapersonal skills, interpersonal skills, and technical

skills.
an
Soland, Hamilton, and Stecher (2013) also use a similar model with three of the

components of the OECD framework (Ananiadou & Claro, 2009), leaving out only technical
M
skills. Their model provides the following classification: cognitive competencies (academic
ed

mastery, critical thinking, and creativity); interpersonal competencies (communication and

collaboration, leadership, and global awareness); and intrapersonal competencies (growth

pt

mindset, learning how to learn, intrinsic motivation, and grit).

Another model presented here briefly is that of the Partnership for 21st Century Learning
ce

(2007), which provides four categories: key subjects and 21st century themes; learning and

innovation skills; information, media and technology skills; and life and career skills. Key
Ac

subjects include those knowledge and skills of the traditional subject areas as well as global

awareness; financial, economic, business, and entrepreneurial literacy; civic literacy; health

literacy; and environmental literacy. Learning and Innovation Skills includes creativity, critical

thinking and problem solving, communication, and collaboration. Information, Media and
 7

Technology Skills consists of informational literacy; media; and information, communication

and technology literacy. Life and Career Skills entails mostly intrapersonal skills: flexibility &

adaptability, initiative and self-direction, productivity and accountability, and leadership and

responsibility. In this model, the key subjects and 21st century skills underlie or cut-across all of

the other skills. In addition to this model, in the Herde, Wüstenberg, and Greiff paper in this

t
ip
special issue and also in the Ercikan and Oliveri paper that conclude this special issue, those

cr
authors provide still another model (i.e., Binkley et al., 2012).

Many of the skills comprising the different models of 21st Century skills overlap

us
considerably, as might be expected and, in this, many of the skills could be combined. Thus, in

the Herde et al. (this volume), knowledge acquisition is found to be a key element of complex
an
problem solving. However, knowledge can change, both in life and in their laboratory settings,
M
and therefore, one must use both learning skills as well as flexibility and adaptability and perhaps

even innovation skills.

ed

The Focus of the Following Papers on 21st Century Skills

The editors of this special issue (Greiff and Kyllonen) provide a brief orientation to the
pt

issue about 21st Century skills. Ercikan and Oliveri provide a summary for each of the papers at

the conclusion of the issue. This brief article seeks to provide readers with context from 21st
ce

Century skills literature for the papers included in this issue, rather than discussing said papers in
Ac

detail. Three of the four papers in this issue represent empirical studies of skills clearly identified

as 21st Century skills; the fourth is a summary of a series of research studies. The first three

papers in this issue assess collaborative problem solving, complex problem solving, and

creativity. All of these are studies of cognitive processing generally, although all also have

implications for other 21st Century skills.

 8

The Care et al. paper (this issue), while focusing on collaborative problem solving, deals

with such other skills as adaptability, decision-making, leadership, interpersonal skills, and

communication. So while the primary skill being assessed is problem solving, one might argue,

it is affected by a host of other important skills, and it is certainly not a measure of problem

solving alone. The assessment is performed in an on-line environment, thus making

t
ip
technological skills a factor as well.

cr
The Herde et al. paper (this issue) is concerned with complex problems solving as

assessed via computerized simulations. Complex problem environments are situations where

us
multiple factors are in play and those taking the assessments must make decisions on how to

behave given the complex interactions of the different factors. The effects of the factors can also
an
change during the assessment, providing a dynamic situation that necessitates adaptation,

flexibility, and other factors to achieve success on the assessment. The authors of that article
M
believe that other 21st Century skills such as creativity, innovation, critical thinking, learning to
ed

learn, and metacognition are all involved in solving a complex problem. Again, this construct is

clearly a multifaceted one.

pt

Creativity research, especially that fostered by Guilford (1950) and Torrance (1972), was

performed by psychologists, and educational researchers during the middle of the 20th Century.
ce

Much of their work was based upon the notion of divergent thinking as a component of

creativity. Lucas’s (this issue) research combines such a perspective with persistence,
Ac

collaboration, and self-discipline. Creativity is another form of problem solving, one where there

may be no correct answer. Lucas helps to advance knowledge about creativity assessment in at

least two ways. First, this paper provides an excellent listing of the different kinds of ways that

creativity can be assessed. Second, he provides evidence that teachers can assess their students
 9

in the behaviors assessment of creativity ahead. Lucas also implies an important relation in the

paper. Creativity has perhaps been devalued in recent years because of its historical association

with the arts and their devaluation in education, due mostly to fiscal grounds. If creativity is also

seen in the realm of the sciences and technology, then its advancement is perhaps more likely

and, in addition to this new focus, more relevant.

t
ip
Ainley, Fraillon, Schulz, and Gebhardt (this issue) conducted a study on computer and

cr
information literacy, which is perhaps most different in content than the others. They provide a

brief but excellent review of the work that has been performed in this realm to date. They

us
reported that they believe computer and information literacy to involve information management,

collaboration, communication and sharing, creation of content, and problem solving. They
an
developed four modules, each test taker took two of these. Because international comparisons

were used in this research, the 62 questions and tasks composing the measure were translated
M
into 21 additional languages (for 22 total). Their research found that women outscored men and
ed

there were strong effects favoring those with high socio economic status (SES) and educational

orientations (including such factors as parental education). The latter effects especially should
pt

not be surprising and might be seen as validation evidence.

Some Summary Comments

ce

The four primary papers in this issue demonstrate that it is possible to assess a number of

the very complex knowledge and skills considered a part of what is being termed 21st Century
Ac

skills. In the context of the discussion of the various models of 21st Century skills, it should be

clear too that this sampling is not a complete coverage of all the skills that could be assessed.

It must be noted that all of the latent constructs measured in these studies are complex.

They cannot easily be evaluated via simple paper-and-pencil or multiple-choice assessments. Of

 10

note, the evidence provided or recited in this special issue suggests that the constructs were

assessed reliably. Perhaps somewhat less clear is evidence for the validity of the various

measures. Most were clearly built using the techniques associated with content coverage; there

were careful analyses of the content to be covered and apparently adequate sampling of the

domain in question. Ercikan and Oliveri (this issue) make a strong case that the construction of

t
ip
these measures followed, implicitly or explicitly, the approach of evidence-centered design. In

cr
fact, they appear to be excellent examples of professional test design.

The next step for most of these measures is empirical validation efforts. We also need to

us
know more about the relations between different measures of 21st Century skills as well as with

more generalized variables such as general mental ability or developed academic skills.
an
Moreover, increasingly, perhaps through measures of the Common Core in the United States, the

kinds of skills assessed by these measures and those listing among the 21st Century skills must be
M
integrated into more general assessments of academic and subject-matter knowledge and skills.
ed

It is also clear that technology will be critically involved in the administration of these measures

and that technologically-administered and perhaps technologically adapted measures may well
pt

help test developers to assess these complex constructs.

To what uses can these tests be put in a more general way? There are a number of
ce

possibilities. They can be used as part of curricular evaluations, especially where the intentional

instruction in 21st Century skills has been implemented. With the levels of evidence-centered
Ac

design that they employed, they are perhaps best used as outcomes assessments where programs

are being assessed, especially if such analyses are performed for formative reasons. They can be

used as part of school, state, and national comparisons, if so desired. Their use as selection

measures, whether for higher education or employment, however, can probably not yet be
 11

justified. More empirical validation research, including research on the fairness among cultural,

ethnic, gender, and SES groups would be advisable prior to such uses. From the articles in this

issue it is clear that we have identified and appear to be able to assess important skills, most of

which could be considered new skills. Now is time both for validation of those measures, but

also for instruction to catch up. Instruction too much change dramatically to address these new

t
ip
needed skills.

cr
us
an
M
ed
pt
ce
Ac
 12

References

Ananiadou, K., & Claro, M. (2009). 21st century skills and competences for new millennium

learners in OECD countries. OECD Education Working Papers, No.41, OECD Publishing.

Binkley, M., Erstad, O., Herman, J., Raizen, S. Ripley, M., Miller-Ricci, M., & Rumble, M.

t
(2012). Defining twenty-first century skills. In P. Griffin, B. McGaw, & E. Care (Eds.),

ip
Assessment and Teaching of 21st Century Skills (pp. 16-66). Dordrecht, Netherlands:

cr
Springer.

Ercikan, K. & Oliveri, M. E. (this issue). Assessing 21st Century skills: In search of validity

us
evidence in support of the interpretation and use of assessments of complex constructs.

Applied Measurement in Education.

an
Guilford, J. P. (1950). Creativity. American Psychologist, 5, 444-454.
M
Metz, S. (October, 2011). Editor’s corner: 21st century skills. The Science Teacher, 78, 6.

Partnership for 21st Century Learning. (2007). Framework for 21st Century Learning.
ed

Washington, DC: Partnership for 21st Century Learning. (Downloaded 2/28/2016 from

http://www.p21.org/storage/documents/docs/P21_framework_0116.pdf.)
pt

Partnership for 21st Century Learning. (2007). Assessment: A 21st Century Skills

Implementation Guide. Tucson, AZ: Partnership for 21st Century Learning. (Downloaded
ce

2/28/2016 from http://www.p21.org/storage/documents/p21-stateimp_assessment.pdf.)

Ac

Torrance, E. P. (1972). Predictive validity of the Torrance tests of Creative Thinking. The

Journal of Creative Behavior, 6, 236-252.