Professional Documents
Culture Documents
data). After the variable relationships have been discovered in the problem space
(top part) by applying strategies such as VOTAT, the student is asked to indicate
Importantly, computer-based process data are collected and stored in log files while
students work on the complex problems within such a microworld. These log files
capture all relevant components of a student's interaction with a given problem,
including time spent on an item, all variable manipulations and their corresponding
time stamps, and whether the student achieved or at least approximated the predefined
goals (Xu et al., 2018). A common approach to extracting valuable information from this
computer-based process data after CPS assessment is to write an automated program
that iterates through all log files and parses the relevant data points for further
subsequent statistical analyses. For instance, if data are stored in XML-format, this can
be achieved using so-called XML parsers (i.e., programs that automatically extract the
desired information from raw log files; e.g., Applen and McDaniel, 2009).
The importance of CPS for both educational and career success as the ability to manage
novel and dynamic situations in a systematic way has been discussed in multiple studies
(Schweizer et al., 2013; Wüstenberg et al., 2012). Thus, it does not come as a surprise
that a large body of research on CPS using computer-based process data from large-
scale assessments, including PISA, has accumulated in recent years (Greiff et al.,
2015; Han et al., 2019; Liu et al., 2020). For instance, research has established the
particular benefit of applying strategies such as the aforementioned VOTAT strategy
alone or in concert with other related strategies for successful CPS performance (Molnár
and Csapó, 2018). Additionally, the time students spend on individual complex
problems has been shown to influence their probability of solving the problem
successfully or unsuccessfully (Scherer et al., 2015). A study by Ren et al.
(2019) uncovered how students can successfully balance multiple goals to be achieved
simultaneously in CPS. Additional research has sought to identify different levels of CPS
proficiency in students, for example based on how they approach a CPS assessment test
(Greiff et al., 2015; Stadler et al., 2020). Eichmann et al. (2020) used PISA computer-
based process data to uncover differences in students' CPS performance based on their
sex and ethnic background: the higher CPS success rates found among boys compared
to girls were attributable to differences in exploration behavior between the sexes,
whereas the behavioral differences investigated were unable to account for the
performance differences between students with versus without migration backgrounds.
In summary, as demonstrated by the selected research endeavors described above,
computer-based process data analysis has greatly supported the investigation of
students' competence levels in crucial skills such as CPS (for an overview of computer-
based process data use in CPS, see also Herde et al., 2016). At the same time, computer-
based process data analysis has helped to uncover opportunities to improve students'
skills in these areas by means of training programs in digital learning environments
(e.g., Azevedo, 2007). Moreover, as computers and digital learning and assessment tools
become increasingly prevalent in the educational context, we can expect further
advancements based on computer-based process data analysis in the near future.
Consequently, we will now discuss potential challenges and future focus areas for
computer-based process data analysis, after providing a broad summary of its current
state of application in the educational measurement domain.
View chapterExplore book
the facility for students to assess their own learning of concepts, through
computer-based qualitative and quantitative assessment modules.
The benefits for faculty involved in the projects were found to include: job satisfaction
flowing from the improved learning of their students; an improved understanding of
student learning, student needs, and difficulties; an improved understanding of their
own discipline area; enhanced enthusiasm for teaching; and, for some, an increase in
their own personal profile.
For departments or faculties, the major benefit was the faculty development
opportunities afforded by faculty member's participation in the projects. This
sometimes led to significant change in teaching approach in areas other than the
designated project, as faculty developed enhanced understanding of learning and
teaching. For some departments, the project outcome helped faculty to cope with
decreased resources, without a commensurate decrease in the quality of teaching.
Finally, the teaching profile of some departments was raised as a result of external
recognition of the innovation.
The study emphasizes the fact that it is not the presence of information and
communication technologies by itself that accounts for enhanced learning, rather it is
the design of the learning experience, the support of the learners undertaking that
experience, and the students' perception of the context in which they learn.
In a more recent study of the use of lecture recording technologies used across four
universities, Gosper et al. (2008) report that students perceive the following benefits of
lecture recording technologies in improving their learning: picking up on things they
missed in class; to revise for exams; to revisit complex material, ideas, and concepts;
and to work through material at their own pace.
Faculty have expressed two main concerns around the use of these lecture recording
technologies: that students would not pace their use of these technologies, resulting in
the need to cram learning at the end of the semester, and/or that they would stop
coming to lectures. This fear was somewhat confirmed by the Gosper et al. study in their
report that only approximately 50% of students listened to the recordings on a regular
basis and almost 40% listened to several weeks at the one time. Of the students who no
longer attended lectures, as reported in the study, 68% said they had chosen not to
attend because they could learn as well from the recorded lectures as they could from
the face-to-face lectures.
Critique of Flexible Delivery
Alongside the benefits ascribed to flexible delivery using the information and
communication technologies discussed above, there are also significant challenges. First
and foremost, there remain, despite the rhetoric about the high levels of IT literacy
of generations X and Y, whole cohorts of students who do not have sufficient levels of IT
literacy to access the course materials. Adequate technical support is essential to these
learners in achieving successful learning outcomes.
A second factor is that of students reporting insufficient time to devote to the
course. Mason (2001) famously proclaimed that “time is the new distance,” since lack of
time, rather than long distance, has become one of the primary reasons that students
withdraw from courses. The self-discipline required to undertake a course that has been
delivered to the door is far greater than that required by students whose attendance
requirement at face-to-face lectures serves as a time-management strategy. It is often
only the experienced learners who have the time-management skills to undertake such
self-directed study, yet the majority of students in the higher education system are
undergraduates, who have come directly from high school, and who are studying full
time.
A third critique has centered around the degree to which information and
communication technologies are in fact innovative. Many authors have pointed to the
long-term existence of books and papers which have for many years facilitated learners'
access to the content of learning in a way that is time and location independent. These
authors also point to the fact that learners have control over the order in which they
read printed material, thus facilitating learner control over the content of learning that
is held up as a unique feature of information and communication technologies. Others,
such as Alexander and Boud (2001), argue that the potential for the use of information
and communication technologies to enhance learning has been lost because faculty
have, by and large, simply used the new technologies to automate existing didactic
practices. Much of what passes for innovation in learning is little more than lectures that
have been turned into podcasts and vodcasts, and textbooks which have been
repurposed as websites with electronic page turning for example.
The view of flexible learning described above which equates flexible learning with the
use of technologies for flexible delivery of teaching/learning has been criticized for being
a particularly narrow view of flexible learning. It does, nonetheless, remain a prominent
discourse in both everyday discussions and promotional literature.
There are, however, other views of flexible learning which are discussed below.
View chapterExplore book
However, the fact-triangle intervention in the Walker et al. (2013) study may have failed
because of the ineffective manner in which the training was implemented. The model
used was (a) perhaps only semiactive (Section 3.3, Point 2), (b) not designed to prompt
reflection (Point 3), (c) not particularly meaningful (Point 4), (d) without explicit
connections between procedures and concepts (Point 5), (e) relatively short in duration
(Point 6), (f) dependent on a single (not multiple) representations (Point 7), and (g) not
purposeful and engaging (Point 8). In regard to Point 4, their fact-triangle training
involved only two of seven steps in a HLT for fostering the meaningful memorization of
subtraction combinations (Baroody, 2016a). In contrast, virtual concrete experiences
designed to be consistent with Points 2–4 and 6–8 were significantly efficacious in
promoting fluency with unpracticed subtraction combination—transfer, which is a
primary goal of education (Baroody, Purpura, Eiland, & Reid, 2014; Baroody, Purpura,
Eiland, Reid, & Paliwal, 2016). Moreover, although Walker et al. did attempt to gauge
transfer, they did not measure which method was more effective in fostering the
conceptual understanding that addition and subtraction are related operations. Finally,
aside from a fluency pretest, these researchers did assess internal factors such as
developmental readiness to benefit from fact-triangle instruction.
Mix (2010) concluded that manipulatives “play different roles in different situations” (p.
41) and the key question is not “do such educational tools work” but “do these materials
used in this particular way activate this particular mechanism in this particular
learner?” Moreover, citing Ginsburg and Golbeck (2004), she noted, “almost no
research has addressed how or why these materials might help” (p. 41). To address such
a question effectively, researchers need to spell out the role manipulatives are presumed
to play—a factor that dictates the outcome measures. Importantly, both the theoretical
model and intervention effort need to take into account Dewey's (1963) principle of
interaction—how external factors are intended to mesh with internal factors.
Evaluations of manipulatives involving interventions that violate this principle (e.g.,
attempt to impose a manipulative procedure via direct instruction quickly) confound
instructional approaches with the potential value of the manipulative in a learning
environment that honors the principle (e.g., encourages a child to use their existing
knowledge to reflect on how to use the manipulative). Furthermore, a fair evaluation of
manipulatives should include assessing the developmental readiness of participants
along a HLT. To date, such internal factors have been largely overlooked when
researchers construct hypotheses regarding why and how manipulatives work and
evaluate the impact of manipulatives.
View chapterExplore book
Hazmat Hotline uses maps in a slightly different way, still giving context to their users,
but on a much more local level (Schell et al., 2005). Giving them locations of victims, of
the source of the hazardous material, and of their crew, this game allows users to think
about how to best handle the situation, given where everything is located in relation to
each other.
Although GIS and spatial components are critical components of disaster training
serious games, there are other factors seen within released games that strengthen a
game’s viability as a training tool. One of these factors is the inclusion of a stress
component to portray the reality of the situation at hand. There are several ways to
address the stress component within a gaming context, one of which is using time as a
key game mechanic. Haferkamp et al. (2011) demonstrates how each is portrayed within
the game, DREAD-ED. DREAD-ED works based on limited time for team discussion
and decision-making, giving the team between 30 and 45 min to reach a decision. To
introduce a stress component as well, the game displays four scales to the players which
change based on decisions they make to give real-time feedback after every more. Both
poor and wise decisions come with feedback. The tactical decision games created
by Crichton and Flin (2001) reflect upon similar components, allowing only an hour and
a half for participants to completely work through their game, and introducing
contingencies throughout the entire duration. The time component emphasizes the need
for emergency responders to act quickly in light of a disaster. In addition to a time limit,
stress is also factored into serious games through the inclusion of an unpredictable
factor. Created in conjunction with VSTEP and several agencies around
Europe, RescueSim is a flexible gaming environment that is controlled strictly through
an instructor toolbox (VSTEP B.V., n.d.). The instructor not only creates the original
scenario that will be presented to the players but also is capable of changing the weather
in real time, showing the progression of an incident as it would look in real life, and
introducing secondary events off of the primary one. Each of these changes is not known
or able to be predicted by the players. SPOEL, a virtual environment created for the
management and training of mass evacuations, allows for stress to be portrayed in a
slightly different manner, working with the changes in human behavior as well as
resource distribution and management as their primary sources of stress (Kolen et al.,
2011). Victims within the game are able to change their opinions and actions based on
media and decisions of the emergency crews. Road systems are also a limited resource,
as they are capable of degrading within the game, or becoming too congested to use as
viable evacuation routes.
Another component present in released disaster management games is the use of news
stories or information recaps within the game scenario. Information provided to the
players throughout the game scenario is another crucial piece to their ability to fully
understand about what is going on as the incident revolves around them. IMACSIM
provides this through use of waypoints (Benjamins and Rothkrantz, 2007). As the users
make their way through the simulated environment they are able to visit numerous
waypoints which provide information on the current state. These waypoints are flexible
with scenarios, meaning that they can fit to a variety of different conditions and
emergencies, and they are also able to accurately reflect any changes that occur
throughout game play. Disaster in my Backyard also takes advantage of the opportunity
to introduce information throughout the game, using QR codes and victims as the
information source (Meesters and van de Walle, 2013). Set up as a live walk-through
game, this scenario is much more hands-on in their information presentation. As
players make their way through from start to finish, they are able to interact with actors
who are playing victims within the game, receiving various amounts of information as
they interact with them. Similarly, participants are also given an app which allows them
to interact with QR codes that are placed throughout the game environment. These QR
codes contain relevant information and allow communication between people as the
game plays out.
View chapterExplore book
Recommended publications
Computers in Human Behavior
Journal
Computers & Education
Journal
The Journal of Mathematical Behavior
Journal
Applied Ergonomics
Journal
Browse books and journals
Featured Authors
1. Hoyles, Celia
2. Cai, Su
3. Chiang, Fengkuang
About ScienceDirect
Remote access
Shopping cart
Advertise
Contact and support
Terms and conditions
Privacy policy
We use cookies to help provide and enhance our service and tailor content and ads. By
continuing you agree to the use of cookies.
All content on this site: Copyright © 2023 Elsevier B.V., its licensors, and contributors. All
rights are reserved, including those for text and data mining, AI training, and similar
technologies. For all open access content, the Creative Commons licensing terms apply.