Int. J.

Human-Computer Studies 117 (2018) 4–19

Contents lists available at ScienceDirect

International Journal of Human-Computer Studies

journal homepage: www.elsevier.com/locate/ijhcs

Mymemory: A mobile memory assistant for people with traumatic brain

injury
Carole Chang, Annika Hinze∗, Judy Bowen, Leah Gilbert, Nicola Starkey
University of Waikato, Hamilton, Waikato, New Zealand

a r t i c l e i n f o a b s t r a c t

Keywords: Traumatic Brain Injury (TBI) is a major cause of disability in young people in New Zealand, and has long-term
Traumatic brain injury effects on memory and other cognitive functions. This article introduces MyMemory, a mobile augmented memory
Mobile memory aid system that aims to assist TBI survivors in coping with their memory impairments. We here present an exploration
Autobiographic memory
of design requirements for mobile memory aids for people with TBI, the MyMemory conceptual design and high-
Cue-based memory retrieval
level details of the prototype implementation. We report on the results of our A-B-A-B study with six TBI survivors
and three caregivers. The participants with TBI all reported improvements when using MyMemory with regards
to their well-being, memory function and autobiographical memory. The caregivers confirmed these observations
of TBI participants, however, the results regarding possible reductions of caregiver burden are mixed.

1. Introduction This article introduces MyMemory, a mobile augmented memory sys-

Traumatic Brain Injury (TBI) is one of the most common causes of
disability and death in young adults throughout the world (Carroll et al.,
2004; Feigin et al., 2010), and is often referred to as a “hidden epi-
demic” (Cardinale, 2016; Jahr, 2017; Starkey, 2013; Stevens, 2017).
The World Health Organisation refers to TBI as “an acute brain injury
resulting from mechanical energy to the head from external physical
forces” (Carroll et al., 2004). In New Zealand, TBI incidence was esti-
mated at 749 cases per 100,000 in 2012, and globally the incidence of
TBI appears to be increasing (Feigin et al., 2012). The outcomes from TBI
differ depending on the severity and site of injury but many survivors
have ongoing symptoms which not only impact their lives, but also the
lives of their families and friends, and the community. Memory impair-
ment and other long-lasting cognitive difficulties are one of the most
common symptoms of TBI and affect many TBI survivors (Barker-Collo
et al., 2008; Hewitt et al., 2006). Long-term memory issues are preva-
lent for many people even after a mild TBI. The memory impairments
take many forms depending on the nature of the injury (Dritschel et al.,
1998). They include relatively minor memory slips and lapses, such as
those we all experience from time to time (e.g., forgetting a person’s
name) to more severe problems such as anterograde amnesia (the in-
ability to form new memories). Long-term care and support for people
with TBI is complex and poses a significant economic burden for indi-
viduals, families and society (Te Ao et al., 2014).
tem that aims to assist TBI survivors to cope with their memory impair-
ments, in particular with their autobiographical memory deficits (i.e.,
impairments to the recollection of episodes from a person’s life). In
an initial study, we identified factors (e.g., location and other people
present) that may help TBI survivors with their memory retrieval. Our
MyMemory app uses these factors to record people’s (autobiographic)
memories and, at the same time, train their own ability to remem-
ber. The MyMemory concept incorporates two strategies to support TBI
survivor’s memory and remembering. Firstly, MyMemory helps people
record memory cues about events in their lives, which may be looked
up later to trigger remembering. Secondly, MyMemory offers memory
training based on these memory cues, using the concept of memory
training flashcards and memory-ticker widgets. Paper-based flashcards
are typically used for studying or practice; they typically bear a ques-
tion on one side and an answer overleaf. Our digital version of flashcards
first shows a question (automatically generated from the user’s memory
data) and, after a user prompt, secondly shows the correct answer. The
flashcards can be used to actively train memories through a series of
questions that are automatically created from the stored memory cues.
The memory-ticker aims to passively train memories through a contin-
uous stream of memory cues that are shown on the home screen of a
smartphone.
After initial usability studies, the effectiveness of the MyMemory
prototype app was explored through user studies involving both TBI
survivors and their caregivers. With the study, we investigated whether
the TBI survivors experienced an improvement of their autobiographical

∗
Corresponding author.
E-mail addresses: cchang@waikato.ac.nz (C. Chang), hinze@waikato.ac.nz (A. Hinze), jbowen@waikato.ac.nz (J. Bowen),
nicola.starkey@waikato.ac.nz (N. Starkey).

https://doi.org/10.1016/j.ijhcs.2018.02.006
Received 31 May 2017; Received in revised form 3 December 2017; Accepted 18 February 2018
Available online 4 April 2018
1071-5819/© 2018 Published by Elsevier Ltd.
C. Chang et al.
memory functioning and/or their well-being (using the WHO well-being
index (Heun et al., 2001)). We also sought caregiver corroboration of the
TBI survivors self-reports, as well as exploring reductions in caregiver
burden. MyMemory was evaluated using a within-subject single case
A-B-A-B design study, a method typically used in psychology research
to verify the impacts of an intervention treatment (Heffner, 2004; Svo-
boda and Richards, 2009). Our A-B-A-B study tested periods without the
intervention, i.e., only using the participant’s established memory aids
(baseline phase A), followed by periods with the intervention, i.e. using
the MyMemory app (active intervention phase B). Our study partici-
pants reported that MyMemory not only positively influenced the TBI
survivors’ memory ability but also their well-being and that of people
around them. The contributions of this article are three-fold:
1. Design requirements and MyMemory conceptual design
2. Interaction details of MyMemory mobile prototype app
3. Results of A-B-A-B study with TBI survivors and caregivers
The remainder of the article is structured as follows: Section 2 pro-
vides background information on traumatic brain injury and explores
the similarities and differences in memory impairments following TBI
with those of Alzheimer’s Disease. Section 3 gives an overview of exist-
ing research related on memory aids an augmentation. Section 4 sum-
marises our design explorations for augmented memory systems for peo-
ple with TBI through a number of user studies (using interviews and
paper prototypes). Section 5 introduces the MyMemory prototype by
walking through the prototype interface and exploring a scenario of
use. Section 6 reports on the results of our user A-B-A-B study. The
paper concludes with a discussion of insights gained through this re-
search (Section 7), a short summary, and a discussion of future work
(Section 8).
2. Background
This section presents background information that motivated our
research, in particular: the long-term effects of traumatic brain injury
(TBI) and the differences between TBI and Alzheimer’s Disease.
2.1. Long-term effects of TBI
The majority of Traumatic Brain Injury cases are mild TBI (mTBI),
which are caused by closed head traumas and are not always recog-
nised in time or treated in a hospital. TBI mostly affects young people
between ages 15 and 25, with males being affected almost twice as of-
ten as females, and rates for Māori (New Zealand indigenous population)
being significantly higher than for the rest of the population. There are
an estimated half million prevalent cases of TBI in New Zealand, over
95% of which are mTBI (Te Ao et al., 2014). Preliminary analysis indi-
cates up to 30% of affected people have ongoing cognitive issues one
year post injury. The most common issues are problems with memory,
attention, and concentration, with memory problems being the most no-
ticeable (Barker-Collo et al., 2008). The initial impact on the family can
be categorised into three broad phases: the acute phase, the rehabili-
tation phase and the reintegration phase (Cavallo and Key, 2005). The
acute phase focuses on the medical stabilisation of the survivor, which
demands time and energy from the family. Second is the rehabilitation
phase in which family roles are reorganised, for example a family mem-
ber may become a caregiver for their TBI-affected relative. A possible
reintegration phase means that the individual recovers from the brain
injury and attempts to return, as much as possible, to a level of inde-
pendence and productivity in the community. This would usually take
several years.
While the perception exists that patients with a TBI require lit-
tle further treatment after the rehabilitation phase, Masel and deWitt
suggest we should see TBI as the beginning of a chronic disease pro-
cess (Masel and DeWitt, 2010)—“a process not an event”. Chronic ill-
nesses are irreversible conditions that last for long periods of time, and
5
Int. J. Human-Computer Studies 117 (2018) 4–19
can often never be cured completely. They require long-term treatment
or rehabilitation services, possibly including medication and lifestyle
changes. Memory problems of people with TBI often last well beyond
the initial period of recovery. Memory and impaired cognition are also
cited by patients and caregivers as one of the greatest sources of issues
in daily life, and add more to lasting disability than physical issues even
years after initial injury (McCullagh and Feinstein, 2011). The memory
issues encountered by TBI survivors often affect other areas of rehabil-
itation as memory is critical to learning the techniques on which other
rehabilitation relies. Even mild TBI can lead to long-term problems such
as post-concussion symptoms, concentration issues, and impairment to
executive functions. All these factors in combination mean that even in
apparently very mild cases of TBI, there can be noticeable long-term
cognitive impairment. Even mild injuries can have long-lasting prob-
lems that may often go unrecognised (Starkey, 2013).
The impact of mTBI can also be significant for people living with
the injured individual: for example, carers run a higher risk of hav-
ing poorer quality of life and increased psychiatric problems compared
to the general population (Feng, 2014). Increased levels of depression
and anxiety, relationship problems, decreased income, and loneliness
can potentially add further costs through the use of community support
structures. Knight and O’Hagan (2009) also identified a number of so-
cial problems caused by the kinds of autobiographical memory deficits
that typically occur for people with TBI. They observed that people with
TBI were consistently impaired in the ability to provide autobiographi-
cal episodic memories of events that occurred, before, during and after
the injury. They had difficulty in acquiring new memories and com-
municating with others. This was observed to often make interpersonal
exchanges with people with TBI frustrating, potentially leading to in-
creased social isolation.
2.2. Alzheimer’S disease and TBI
A number of memory support tools have been developed for peo-
ple with Alzheimer’s Disease, and it has often been suggested that these
tools could be used for people with TBI. There are however a number
of differences between TBI and Alzheimer’s Disease that limit this pos-
sibility. Alzheimer’s disease (AD) is a progressive irreversible brain dis-
ease that impacts on daily living through memory loss and cognitive
changes. Common early symptoms of Alzheimer’s disease include con-
fusion, disturbances in short-term memory, problems with attention and
spatial orientation, changes in personality, language difficulties and un-
explained mood swings (Bright Focus Foundation, 2011). While there
are some similarities in memory symptoms for TBI survivors and AD
patients, these have both different causes and effects. Here we briefly
discuss the memory dysfunctions which accompany Alzheimer’s Disease
in comparison to those of TBI.
AD normally affects people over 65 years of age, while TBI predom-
inantly affects young adults. In New Zealand, 1% of the population is
affected by AD and 2% are affected by TBI. As TBI is caused by a brain
injury, memory retention may be improved with medication, surgery or
therapy. In contrast, AD is caused by retrograding or dying brain cells,
which cannot be revived. AD patients may use medication to slow the
progression of the disease and delay cognitive decline.
As a consequence, the memory issues encountered by people with
AD and TBI typically differ and the prospects of influencing memories
and memory recall through cognitive assistants may require different
strategies. While for people with AD support of daily-living functions is
important, for people with TBI memory training and cue-based retrieval
may be more appropriate.
3. Related work
This section discusses related approaches for augmenting memory,
for a general audience, for people with TBI and other cognitive impair-
ments.
C. Chang et al.
3.1. Cognitive orthotics and cognitive assistants
Cognitive orthotic systems aim to help people in adapting to cognitive
declines that may be caused by natural ageing processes or diseases and
accidents. An example is the Autominder system (Pollack et al., 2003).
Autominder provides personalised reminders for activities of daily liv-
ing, e.g., for supporting people who remain in their own homes during
old age. The system is embedded in a personal robot assistant. Such
tools for assisted living for the elderly have attracted increased atten-
tion due to an ageing society (Haux et al., 2016; Rashidi and Mihailidis,
2013). Many of the tools for assisted living employ means of ambient in-
telligence, which are digital (embedded) environments that are respon-
sive to human activities and interactions (Acampora et al., 2013). So-
called assisted smart home projects employ ambient stationary sensors,
e.g., to address the needs of dementia patients (Andreadis et al., 2016;
Rashidi and Cook, 2009), to provide health support for seniors (Adami
et al., 2010; Costa et al., 2012; Rantz et al., 2011) and inform robotic
carers (Chen et al., 2011). Assistive robotic systems (Smarr et al.,
2011) may help home-based elderly (Beer et al., 2012) or people with
disabilities (Chen et al., 2013). They may give support for house-
hold activities (service) and act as a companion (Broekens et al.,
2009). Mobile and wearable sensors can support people in keeping ac-
tive (Pantelopoulos and Bourbakis, 2010), and are used for the moni-
toring, e.g., of people with mental disorders (Gravenhorst et al., 2015),
Bipolar Disorder (Grünerbl et al., 2015) or depression (Canzian and Mu-
solesi, 2015). Mobile technology may be combined with body-sensor
networks and Internet-of-Things (IoT) technology for health monitor-
ing using sensors inside and outside the human body (Adam and Soh,
2016; Hao and Foster, 2008; Harbouche et al., 2013). They are also em-
ployed for providing mobile assistance, such as for wayfinding for the
blind (Rodriguez-Sanchez et al., 2014).
In addition to direct practical assistance as outlined above, sys-
tems may also provide social support, such as companionship (Mordoch
et al., 2013; Van Erp and Toet, 2015; Wada et al., 2010) or informa-
tional/cognitive assistance. Cognitive assistants have been introduced in
recent years on a number of platforms, for example, Siri for mobile
phones, Google Home for smart homes, Microsoft Cortana for finding
things on a personal computer (Ebling, 2016). While these may have
become commonplace for the general public, providing specific cogni-
tive assistance to people with disabilities is still a field of active research.
Cognitive assistants help their users keeping informed and active. Being
designed to overcome user disabilities, they often combine software and
hardware applications (Costa et al., 2012).
3.2. Memory aids for people with Alzheimer’s disease
While digital aids for TBI survivors often focus on improving
their memory ability (see Section 3.3), memory aids for people with
Alzheimer’s disease aim at retaining their memory ability, often using a
life-logging or capture-everything approach (De Leo et al., 2011; Gurrin
et al., 2014; Lee and Dey, 2007). In a study with a person with AD, a mo-
bile phone was used to take pictures at 5-min intervals for 12 h during
the day (De Leo et al., 2011). The pictures were combined into a video
and made available on a weekly basis. The person and their caregiver
viewed the video and had to answer questions about autobiographical
events of the previous week. The study found that the video significantly
increased the person’s ability to recall recent events.
Lee and Dey (2008) observed that life-logging technologies often col-
lect too much data to review regularly and proposed using a combina-
tion of automated content analysis and caregivers input to identify suit-
able memory cues. One example of life-logging technology is SenseCam,
a wearable digital camera that automatically captures photographs ev-
ery 30 s (Hodges et al., 2011). The SenseCam browser (Doherty et al.,
2012) applies a series of automatic content analysis techniques to struc-
ture the captured images into events. It uses images as cues to trigger
people’s memories. Woodberry et al. (2015) found that AD patients us-
6
Int. J. Human-Computer Studies 117 (2018) 4–19
ing SenseCam remember events better than when using a written diary.
SenseCam has also been suggested to be a useful rehabilitation tool for
patients with memory retrieval difficulties (Browne et al., 2011).
3.3. Memory aid systems
Assistive technology for cognitive rehabilitation for people with ac-
quired or developmental cognitive impairments has been researched for
close to 30 years (Frank LoPresti et al., 2004). The two main approaches
are memory training and reminder-based support.
3.3.1. Memory training and support
A number of mobile applications are used by people with a brain
injuries and their families and caregivers to practice and train mem-
ory functions. Many local community projects and help groups, such
as BrainLine1 and Brain Injury Waikato2 , also often recommend mo-
bile apps or games for people with memory impairments (BrainLine,
2013; Health Care POV, 2012). These recommended apps cover prac-
tical aspects (such as communication, organisation, and location moni-
toring) as well as memory-related aspects (e.g., reminders, memory and
brain training). Most of these are not formally evaluated and their ef-
fectiveness for training people’s memory is uncertain. Example apps for
memory stimulation and training such as Awesome Memory3 (simple
card matching game), Morphos4 (anagram and word scrambling game),
Flashcards Deluxe5 (learning and memorizing tool), n-back6 (for train-
ing working memory), and Spaced Retrieval Therapy7 (memory training
based on repetition) are built around cognitive neuro-science research.
Examples of direct memory training apps are Cogmed and Luminosity
systems, which focus on problems with working memory (i.e., issues
with short-term remembering and manipulating information). While the
systems are popular and commercially successful, it has been widely ar-
gued that the memory improvements do not last and are not generalis-
able (Shipstead et al., 2012). The systems seem to train the execution of
particular tasks, not the memory ability.
Augmented Reality systems have also been used for rehabilitation
after trauma. Augmented reality (AR) presents a view of the existing
physical world (e.g., viewed through a phone camera) that incorporates
additional information. This augmentation may use text, image, video,
or sound. LevelHead8 is a spatial memory game that uses solid plastic
cubes which the player manipulates in front of a camera. The cube’s
side facing the camera is shown on screen with an animated character
in a room that is part of a maze. The aim of the game is to guide the
character out of the maze by tilting the cube to direct the character’s
movement between rooms. By forcing the player to remember and recall
the virtual layout of the maze, spatial memory is utilised. Another AR
example is the Interactive Metronome9 where computer users practice
timing and rhythmicity of various movements of the hands and feet in
synchrony with auditory cues. Changes in sound cues provide feedback
to the user indicating how far off they are from the guiding rhythm. Early
studies reported significant memory improvements for people with mild
to moderate TBI (Nelson et al., 2013). Other AR projects explore the
use of physical movement in a spatial card-game to enhance cognitive
function (Gilbert et al., 2017).
3.3.2. Reminder-based memory support
One tool explicitly designed to help people with memory impair-
ments was the phone app ExpiryDate by kcsoft (which has since been
1
http://www.brainline.org/
2
http://braininjurywaikato.org.nz/
3
http://tangiblegames.com/awesomememory/
4
https://play.google.com/store/apps/details?id=com.BestDamnApp.Morphos
5
http://itunes.apple.com/us/app/flashcards-deluxe/id307840670
6
https://play.google.com/store/apps/details?id=com.tyrske.dualnback
7
http://tactustherapy.com/app/srt/
8
https://julianoliver.com/levelhead
9
http://www.interactivemetronome.com
C. Chang et al.
discontinued). Its concept was to help users manage the expiry dates
of virtually everything (such as food in the fridge or upcoming meet-
ings). It used a simple tile-based visual layout, with each tile showing
an image, name and expiry date. The item with nearest closing time was
listed at the top and image size increased to attract attention. The sys-
tem was well-suited for people with brain injury due to its simplicity,
requiring input of only basic information and providing clear informa-
tion layout. However, the reminder function is not sufficient for most
people with TBI. We identified more specific requirements in our inter-
view user study, see Section 4.
3.4. Augmenting autobiographical memories
Research into augmenting memory had for a long time focused
largely on using digital records aiming to capture everything a per-
son does and sees via videos or photos (life-logging). A number
of researchers argued against this technology-driven approach to
remembering (Schweer, 2011) as a ‘replaying of recorded memo-
ries’ (van den Hoven and Eggen, 2008). They follow a construction-
ist approach, describing autobiographical memory as being stored and
triggered by memory cues (van den Hoven and Eggen, 2015). Stud-
ies have shown that when new information can be related to existing
memories it will be easier to be recalled in the future (Thompson et al.,
2013). Even partial contextual cues can be sufficient to successfully re-
call events (Wagenaar, 1986). Sellen and Whittaker (2010) similarly ar-
gued that an autobiographical memory system should instead focus on
using contextual information as cues to trigger memories. The Digital
Parrot is one example of a cue-based augmented memory system us-
ing contextual information (Schweer, 2011). For each user, the Digital
Parrot builds a memory network consisting of contextual cues (e.g. time,
location, other people present) associated with semantic concepts. When
retrieving stored memory cues, associations between memory items and
semantic concepts can be used as filters (Schweer et al., 2009), for
example geo-locations, timeline or semantic type of event. The result-
ing cues are presented to the user as a memory graph or table. A mo-
bile app version of the Digital Parrot implemented similar visual func-
tions (Richmond, 2012), adapted for smaller screens. While the system
may support recall by triggering memories, it is not suitable for people
with TBI because it provides very complex information and uses an in-
terface design with many details, both of which may overwhelm people
with TBI.
Other approaches using context awareness for augmenting human
memory are reported in Lee and Dey (2006), Vemuri and Bender (2004),
Wood (2015). For example, Nomenclator is a memory aid for names,
which uses machine learning to achieve better energy efficiency in the
implemented system (Wood, 2015). Iwamura et al. (2014) suggested a
context-based approach to retrieve information about the person they
are speaking to based on face recognition and wearable cameras. Re-
cent developments in big data computing have triggered a return to
mining captured images or videos in life-logging approaches (Dobbins
et al., 2014; Gurrin et al., 2014). A number of approaches explore inter-
action between digital and physical memory cues (Barthel et al., 2013;
Neumann et al., 2017; Petrelli and Whittaker, 2010).
4. Mymemory design concept
MyMemory is an augmented autobiographical memory system de-
veloped for mobile devices. It aims to improve TBI survivors’ memory
and support them in their daily lives. MyMemory includes a training
system designed specifically for improving the memory ability of TBI
survivors. MyMemory was designed using HCI qualitative methods and
human-centred design approaches (Chang, 2017).
The design of the MyMemory prototype was based on the results
of two user studies. The first was an interview study, with 21 partici-
pants, which explored memory impairments post-TBI. This study was
designed to investigate aspects of memory impairment that affected
7
Int. J. Human-Computer Studies 117 (2018) 4–19
people with TBI and/or their caregivers, as well as identify which of
these are the most distressing and disruptive. The study also examined
the strategies used by TBI survivors and their caregivers to assist with
remembering. The results of this study were used to identify specific
requirements for memory aids for TBI survivors. The full results were
reported in Chang (2011), but in this section we describe the relevant
aspects as they pertain to the design of the prototype, which we describe
in Section 5.
Our second study explored a conceptual paper prototype (imple-
mented digitally using the Balsamiq wireframe mockup tool10 ). This
study was conducted with TBI participants and aimed to evaluate the
suitability of initial design concepts. It further aimed to find out what
kind of information management techniques and displays were accept-
able to TBI survivors and what special requirements they have for the
design of an augmented memory aid.
4.1. Memory key factors
From the interview study we gathered information about the par-
ticipants’ use of external memory aids (how they used them and what
their attitudes were) as well as the particular techniques they used to
remember different types of events. Although these were typically per-
sonalised to the participants’ individual situations, we were able to iden-
tify common themes. We identified the memory impairments which the
participants found most distressing or most limiting in their daily life.
These were ‘Remembering New Acquaintances’, ‘Remembering Ad-hoc
Events’ and ‘Remembering Instructions’.
Analysis of the data suggested that there are five key factors which
TBI survivors consider when they describe their memory and memory
aid techniques, both in terms of what they want to remember and how
they try to remember. These are: people, location, time, activity and
emotion. For most people with TBI, complex details about new people
they meet make it difficult for them to remember (names, faces, clothes,
appearance etc.) Conversely, when they are with people they already
know, these can act as a cue for remembering events later, e.g. “it was
when you were with your Dad at the garden centre”. This also includes
the second factor, location. All of the participants reported that they
could remember places they visited and used location information to
help trigger memories.
The use of time within memories is represented differently for people
with TBI. Rather than focusing on a specific time of day (e.g. 5.30 p.m.
on Wednesday April 22nd), time is often considered as a component
of an event (e.g. “the afternoon we went to the supermarket” or “the
morning of Dad’s birthday”). Events themselves are considered in terms
of activities, and can also act as effective memory cues, so if they can
remember what they were doing (attending a wedding party), that can
help remind them of things that happened at the same time (met John).
Events which had strong emotions attached to them were the easiest
for the participants to recall. Almost all participants were able to de-
scribe the accident that led to their TBI in great detail because of the
strong emotions associated with it. However, such strong emotions are
not often associated with the day-to-day events and people that the par-
ticipants wish to remember, and as such it is an important cue, but we
consider it the least important factor for our design concepts.
Thus, these five factors of memory (people, location, time, activity,
and emotion) became the key concepts for the design of our mobile
memory application. By structuring the recording of memories around
these five factors we believe they can act as hooks to assist with memory
forming and subsequent recall.
4.2. Memory strategies
TBI survivors do not just want reminders about events in their ev-
eryday life, they want to remember better. All participants reported that
10
https://balsamiq.com/products/mockups/
C. Chang et al.
their inability to remember was a cause of concern (and sometimes dis-
tress) for them. It was clear from this that we should not develop yet
another reminder system, but rather a personalised memory aid appli-
cation that could also incorporate methods for training memory based
around rehearsal. Rehearsal, or repetition, involves being exposed to
the same information repeatedly to make sure it does not get lost from
short-term memory. This in turn improves the likelihood that it will be
transferred into long-term memory. All participants used reminder arte-
facts (leaving notes around in easy-to-see places) in conjunction with
review activities (reviewing their day by writing things down, review-
ing their schedule at regular intervals). These two strategies – recording
information and repetition – were incorporated into our design.
4.3. Technology requirements
All of the participants were already using smartphones and/or com-
puters. However, none of them were using standard applications (such
as online calendars on phones) or complex reminder applications, but
were rather developing their own strategies using the technology. Many
made use of emails and text messages sent to themselves throughout
the day to record things they wished to remember later, and caregivers
also used repeated emails and texts as reminders. Most participants used
some sort of digital recording, from electronic implementations of post-
it notes to keeping an online diary. One participant had found an alter-
nate calendar application which allowed them to colour code all of their
events and activities which they found particularly effective.
We also identified specific problems associated with technology that
are experienced by TBI survivors and concerns that they had with a pro-
posed mobile solution. From this we developed three key requirements
that would be central to our design: simple, customisable, and discreet.
Applications with too many functions can overwhelm TBI survivors.
This was one of the reason many did not use existing memory aids or
applications, they do not have the memory capacity to learn complex
functionality and interactions of software; interactions with an appli-
cation need to be immediate and obvious. Because the participants are
used to developing their own approaches to assist their memories, hav-
ing a degree of customisation is also important (although this needs to
be balanced against the requirement for simplicity). Finally, participants
stressed that do not wish to use mobile applications that draw attention
to their memory deficiencies. Instead they need software with interac-
tions that are socially acceptable and therefore discreet to use.
4.4. Mockup evaluation
Based on the ideas and features described above, we created a wire-
frame mockup for the initial design of our memory app. This was based
on the fundamental concept of having an app that could both record
memories and provide the ability for rehearsal to ‘train’ the user’s mem-
ory, with the five memory features identified used as the key compo-
nents of memory recording. We ran an evaluation study with nine par-
ticipants to examine how well the design and user interface concepts
would support the desired functionality. We invited both people with
TBI as well as those who cared for people with TBI to participate: our
study had four participants with TBI and four participants without TBI
(caregivers) and one participant with Multiple Sclerosis. The participant
with multiple sclerosis (MS) was for the purpose of the study treated
similar to the participants with TBI.
Participants were asked to complete eight tasks following the ‘think
aloud’ protocol (Nielsen, 2012) and also to complete an experience feed-
back questionnaire for each of the tasks. The tasks involved use of the
following features: (1) Adding a memory, (2) Flashcard training, (3)
Flashcard training records, (4) Post-its, (5) Screensaver, (6) Voice memo
recording, (7) Data display and sorting, and (8) Personalised settings.
The sessions were audio-recorded so that the ‘think aloud’ components
could be subsequently analysed.
8
Int. J. Human-Computer Studies 117 (2018) 4–19
The questionnaire asked the participants to rate how easy/difficult
they found each task to complete and provide any feedback on the us-
ability of the implementation of the task. For example, for Task 7 (Data
Display) we asked six questions regarding the functions and usability.
For example, we asked for quantitative feedback: “How difficult was
finding the memory of Easter Holiday Plan? (very easy - easy - normal
- difficult - very difficult)”, and also for qualitative “Which function do
you prefer and why: Sort by Time or Sort by Event)?”. Fig. 1 shows
the layout for displaying added memories as a list and individually. For
each function and task, we noted the time to complete a task and noted
as quantitative findings the average feedback score for the ease of use
of each function (with ‘very easy’ scoring 4 points and ‘very difficult’
scoring 0 points). We also noted the qualitative feedback on the most
liked/useful function. The complete list of tasks and questions is avail-
able in Chang (2017).
The easiest to use function of MyMemory, based on the results, was
the Settings feature. Adding a memory and Post-its were joint second.
The Voice Recording feature scored the lowest for ease of use. The Set-
tings feature was also the most liked, again followed by Post-its and
Adding a Memory, so there was a direct correlation between perceived
ease of use to the preferred functions. Comments on the Adding a Mem-
ory function suggest that the use of the five factors was appropriate for
the participants, as there was a consensus that the information was the
same as the used for their own memory aids but was easier to input and
organise. The least popular feature was the flashcard training records
which was designed to provide training data to the users to raise their
awareness about how often they used the training feature and how well
they did with it. Feedback suggested that this was not important for the
users as their actual memory abilities provided a better understanding
of this.
The feedback for the MyMemory prototype overall suggested that it
was acceptable as an external memory storage and training application
and satisfied the requirements for TBI survivors. However, a more con-
cise main screen display was required. Most asked that this main screen
display show all event data, rather than a control panel which would en-
able access to this. A text message or email feature for backup/sharing
functions was also suggested as an important feature that was missing.
There was also some concern that the data display for the ‘Post its’ fea-
ture with the question/answer pattern displayed too much information
publicly and may breach their personal privacy.
4.5. Prototype development
Based on the results of the studies described above, a prototype was
developed for Android mobile devices. This was based on the mockup
design, but with amendments based on the results of the evaluation.
A new backup feature was added which enabled memories to be sent
by text or email and the question/answer display and structure for the
flashcards and post-it features were redesigned so that information was
kept private until the user explicitly interacted with the training tool.
The training data feature and voice recording were both removed. Once
the prototype had been developed and tested for stability, an evaluation
study was developed and conducted; we discuss this next.
5. The MyMemory Prototype
As part of the design process, two personas were developed, based
on the initial interview studies with TBI participants (see summary in
Section 4). In addition a number of scenarios were developed which
were used as the basis for the initial designs for MyMemory. Here we
present one of the scenarios that will serve as the basis for the user
interaction with the MyMemory prototype shown here. The scenario
combines elements and observations from typical situations that were
described to us in the participant interviews.
C. Chang et al.
Fig. 1. Mockup of memory display.
The MyMemory prototype was developed for Android mobile
phones. All screenshots are made on a Samsung Galaxy phone running
the version ‘Jelly Bean’ of the operating system.
5.1. Scenario
Jim is 38 years old and is a senior editor of a news agency. When
he was 18 years old, he sustained a brain injury as a result of a car
accident. Before the accident, Jim was a high-achieving student, captain
of the school rugby team and popular on the social scene. After the
injury, he developed sensitive hearing, which caused him to avoid public
places. Not only did his demeanour and interests change, but memory
impairments affected his academic performance. He often struggles to be
fully aware and attentive, and due to his limited memory capacity he can
only do one thing at a time. He decided to sacrifice his social life in order
to focus on his academic performance. Because he was self-conscious
about his memory impairment he did not seek support from family or
friends. This situation changed when he met his future wife, Melody,
five years ago. Melody subsequently took on the role of a caregiver for
him.
Jim’s difficulty with remembering ad-hoc events is often a cause for
arguments during their conversations at home. For example, they cele-
brated Jim’s father’s 60th birthday last weekend. During their visit, they
discussed a three-day trip to Lake Tekapo for the upcoming Easter hol-
iday with Jim’s mother. At the birthday dinner Jim promised he would
take care of booking the flight tickets for the trip. On Friday night the fol-
lowing week, Jim told Melody he planned to take a trip up north during
the Easter holidays. Melody tried to remind him that their Easter holi-
day had already been planned last week. As usual, Jim denied knowing
anything about the plan or his promise to get the tickets. When Melody
provided Jim with related information, such as “we discussed this with
mum at your dad’s birthday dinner” and “you told mum you would
book the flights on Monday”, Jim remembered the entire plan and his
promise. Jim then used his smartphone to send an email to himself with
the title “Easter holiday plan”. On Monday morning, Melody sent a text
message to Jim: “don’t forget the Easter holiday plan”. At 4.30 p.m., af-
ter Jim finished a regular meeting and returned to his office, he checked
his phone. When he found Melody’s text message, he turned on his com-
puter and found his own email titled “Easter holiday plan”. Before he left
the office that night, he finally booked the flights for the Easter holiday.
Throughout this section, we will revisit this scenario and show an
alternative version in which Jim uses MyMemory.
9
Int. J. Human-Computer Studies 117 (2018) 4–19
5.2. Setting up MyMemory
Three months ago Jim started using the MyMemory system on his
phone. When he first started the app, he followed the instructions to go
to the settings screen, where he completed the personal information and
then started using MyMemory.
‘Settings’ captures the user contact profile including name, email and
mobile number. MyMemory uses this information for backup of data
and for contacting the user. This is also where the user sets the default
colour for the event labels shown in the main screen. Being able to select
user-defined colours for different event types was one of the participant-
defined requirements that was identified in one of our early design stud-
ies (see Section 4).
5.3. Adding a memory
During Jim’s father’s 60th birthday, the family had dinner together
and Jim’s mum planned a family trip for which Jim volunteered to book
the flight tickets. Before he left the dining room, Jim used MyMemory on
his phone to record this event. Jim clicked the “Adding Memory” icon
on the function which opened the “Adding Memory” screen. He filled
in the related data and clicked the “OK” button for saving. Jim records
the details as:
Date 10/03/2015
Time Evening
Title Easter Holiday Plan
Location Dad’s house
People Mum
Activity talking
Notes 3 days trip to Lake Tekapo. Booking flights∗ 6 to Christchurch.
Jim closed MyMemory and went to the living room joining the family
for his dad’s birthday cake.
The page for “Adding Memory” (see Fig. 2, left) shows event data
fields that are based on the factors of memory described earlier (see
Section 4). Each is represented by an icon and a text field for which
the user can specify their event data. Icons and hints shown in the text
field (in gray) provide intentionally redundant information to make each
field’s purpose easier to understand.
Some fields are auto-populated for simplicity, for example, the cur-
rent date. The time field provides five options (morning, lunch, after-
noon, dinner, and evening) to match the understanding of event-related
C. Chang et al.
Fig. 2. Adding a Memory (left) and overview of memories on main screen
(right).
time described in 4. Next are three fields for related event data: title, lo-
cation, and people. The next field offers nine selections for the current
activity (talking, eating, working, studying, watching, reading, shop-
ping, holidaying, and exercising). The use of a list of options, rather
than free text entry, means less typing for the user. This can speed up
the recording of event data, helping TBI survivors to record details be-
fore they forget them. For the prototype testing the nine selections were
based on commonly occurring activities, but this can be edited and
added to in the user settings. Finally, a user can add additional notes
for an event. Below the text fields are edit options (camera, image, text,
email, and colours) and confirm buttons (cancel and OK). If a photo is
attached, it would then be shown below the notes.
5.4. Overview of memory records
Fig. 2 (right) shows a screenshot of MyMemory’s main screen after
Jim inserted his recent memory about the Easter holiday (some memo-
ries shown are placeholders). TBI survivors require a simple and clear
information display. Therefore, MyMemory presents all event data on
the main screen, sorted by time (most recent event first).
The function bar at the top gives access to the camera function, as
well as functions for sorting the event data (available are ‘sort by person’
or ‘sort by location’). The plus icon is for adding new event data and the
cards icon is the ‘FlashCard Training’ (described in more detail below).
The gear icon gives access to the settings.
5.5. Retrieve/share a memory record
On the way home, Melody asked Jim, “How many tickets do we
need”. “Six, I remember Mum said Alex will go with his girlfriend, but
let me check”, Jim said. He turned on MyMemory and clicked the event
named “Easter holiday plan” on the main page. “Yes, I am right. We
need to book six tickets”, Jim was happy about his ability to retrieve
the information. “Well, I think we need a large place to stay, could you
send me an email about it?”, Melody said. “Sure, give me a second”,
Jim clicked the email icon button on the screen and was presented with
the composed email, see Fig. 3 (left). Jim added Melody’s email address
from the phone’s contacts and sent it. Several seconds later, Melody’s
mobile phone received the mail. She opened and read it. “That’s great. I
will check the accommodation on Monday. I can’t wait to meet everyone
there”, Melody said. “Me too” Jim said.
10
Int. J. Human-Computer Studies 117 (2018) 4–19
Fig. 3. Emailing stored cues about a memory (left) and text messaging (right).
Fig. 3 shows how Jim’s event data is transferred to email (left) and
text message (right). The designs are created for TBI survivors to back
up their memory or share memories (e.g., with their caregivers). In the
email, MyMemory inserts the data in such a way that it mirrors the ver-
bal descriptions caregivers typically use. In the text messages, MyMem-
ory prepares the data as a note similar to that which TBI survivors were
observed to make on paper materials such as a diary or post-its described
in Section 4.
5.6. Training tools
The training tools of MyMemory are based on rehearsal and are ex-
ecuted in two options: memory training flashcards and memory-ticker
widgets (the post-it equivalent for Android devices). Human memory
copes well with five to nine items for short-term memory, and as
MyMemory is specialised for TBI survivors, we have set a limit of five
events for the training tools.
5.6.1. Memory training flashcards
The next morning, Jim had his breakfast with Melody at home. He
turned on MyMemory and found Easter Holiday Plan at the top of the
event list. He selected the ‘FlashCard’ function and practised with the
Easter Holiday Plan. After breakfast, they went to the supermarket for
the weekly grocery shopping. They ran into Alex and his girlfriend –
Kelly. Alex mentioned the Easter holiday plan and Jim told him “I re-
member you and Kelly will join us”. Alex was surprised by Jim’s mem-
ory, which had been refreshed through his practice with the flashcards.
The concept of the FlashCard training uses a memory pattern to cre-
ate two questions and answers for each event. The elements of the pat-
tern include People, Location, Activity and Title from the event data to
produce two types of questions with answers, as follows:
Question 1: Where did you meet People?
Answer 1: I met People at Location.
Question 2: What did you do with People at Location?
Answer 2: I was Activity with People for Title.
For the ‘FlashCards’ created from Jim’s example, this pattern is used
to produce these question and answer pairs: (1) Where did you meet
Mum? / I met Mum at Dad’s house, and (2) What did you do with Mum
at Dad’s house? / I was talking with Mum for Easter Holiday Plan. Fig. 4
shows one example of the flashcard questions and answer pairs that were
created for Jim.
C. Chang et al.
Fig. 4. FlashCard examples: question (left) and answer (right).
5.6.2. Memory-ticker widgets
On Sunday night, when Jim turned to his phone to set the alarm, he
noticed the memory-ticker showing the Easter Holiday Plan event on his
phone’s home screen. Seeing the event data again refreshed his memory
of his dad’s birthday and made him think about the travel plans again.
On Monday, Jim saw the memory-ticker several times during the
day. At the end of the day he contacted his travel agent and booked six
flight tickets to Christchurch on Easter Friday. Jim felt happy because
he remembered the information without checking again. He decided to
leave the event in the ticker a bit longer so that he would later recall his
dad’s birthday celebration better.
The concept of the memory-ticker widget is derived from the post-it
notes we observed people with TBI using and which were one of the
more popular features from the initial mockup (4). From our interface
user study, concerns emerged about the privacy of digital post-its on
the phone’s home screen. Our solution was to use memory factors in a
widget instead of a question-answer pattern. The widget similarly ful-
fils the repetition need. Fig. 5 shows the example widget for Jim, with
information about the event (Easter Holiday Plan), people (Mum) and
location (Dad’s house) to trigger recall. These three factors were the
most common elements that TBI survivors recorded for their own mem-
ory on their physical post-it notes. The widget acts as the digital version
of the post-its on the main screen of the mobile device.
6. Effectiveness of MyMemory
We conducted a user study to evaluate the effectiveness of MyMem-
ory. The study had two aims: (1) to determine whether MyMemory could
help TBI survivors to improve their autobiographical memory functions
and (2) to find if using MyMemory would improve the well-being of the
TBI survivors (and if the care-givers corroborated these observations)
and the well-being of the caregiver by reducing their burden.
6.1. Study design
The study was designed using an A-B-A-B approach (see study plan
in Table 1). This method represents the measuring of an initial baseline
(the first ‘A’, further referred to as A1) followed by the introduction of a
treatment, which in this case is the use of the MyMemory app (first phase
‘B’, called B1). This enables a comparison between the use of the treat-
ment against the initial baseline. Then the treatment is removed (phase
11
Int. J. Human-Computer Studies 117 (2018) 4–19
Fig. 5. Memory-ticker Widget.
A2) to determine if there are measurable lasting effects from the use of
the treatment (if the measures are different from the initial baseline) and
finally the treatment is reintroduced (the final ‘B’ phase, now called B2).
This approach is a standard in use in psychological research (Ferguson
et al., 2015; Stapleton et al., 2007; Svoboda and Richards, 2009).
For our study, the baseline phase A consisted of participants complet-
ing tasks with their own memory aids and phase B denotes the active
intervention phase where participants complete tasks with the support
of the MyMemory app. Comparing the results of the A and B phases al-
lows exploration of the differences in the participants’ outcomes with,
and without, MyMemory. Each phase lasted two weeks (with an overall
study period of 8 weeks) and the researcher conducting the study met
participants at the beginning and the end of each phase.
The study received approval from the School of Psychology Research
and Ethics Committee, University of Waikato. At the start of the study,
each participant was informed about the study plan and asked to give
their consent for participation and data use. TBI participants were asked
to complete a questionnaire which was used for assessing their memory
functions and evaluating the amount of assistance provided by care-
givers. They were provided with an ‘assigned task sheet’ which lists 15
everyday tasks, such as adding an item to a shopping list or planning a
trip to visit a person. From this list, the participants chose two items and
they also selected two further self-assigned tasks from their daily life.
This was to ensure a minimum number of suitable ‘events to remember’
during each phase of the study. At the end of each phase the partici-
pant and researcher met to discuss their progress with the assigned tasks
and how well their normal memory aids or MyMemory (depending on
which phase they had just completed) assisted them with remembering
the tasks. Caregivers were asked to keep a memory logbook throughout
the study and to record any observations about the TBI-participant’s
accomplishing the tasks. Each participant (being either TBI participant
or caregiver) also completed three questionnaires: (1) the WHO Well-
being index (Heun et al., 2001), (2) the TBI or caregiver version of the
memory functioning scale (Hardy et al., 2006), and (3) the “Autobio-
graphical Memory Questionnaire” (Fitzgerald and Broadbridge, 2013)
(TBI participants only) or the “Support Burden Inventory” (Novak and
Guest, 1989) (caregiver participants only). At the end of each of the in-
tervention phases (B1 and B2), TBI participants also completed an eval-
C. Chang et al.
Table 1
User Study Plan: specific material indicated by T (TBI survivor) and C (caregiver).
uation questionnaire to provide qualitative feedback on the MyMemory
app.
WHO Well-being Index: This index focuses on the detection of de-
pression (Heun et al., 2001). It can be used to assess the quality
of life of TBI participants and was used specifically in our study
to monitor changes in the mood of the TBI participants during
the different phases of the study. The WHO Well-being index in-
cludes five questions which are answered using a six-point Likert
scale, ranging from 0 (never) to 5 (all the time), about how the
participant felt over the previous two weeks. The questions ask
for estimates of the participant’s well-being, for example, ‘I have
felt cheerful and in good spirits’ or ‘I wake up feeling fresh and
rested’.
Memory functioning scale: This questionnaire explores the partic-
ipants’ memory ability. There are two versions, one for TBI par-
ticipants to rate themselves and the other for caregiver partici-
pants to rate their TBI relatives’ memory ability based on their
observations. Each version has 13 questions and each question
requires a rating from 0 (never) to 4 (nearly always). The ques-
tions are based on a series of everyday situations where a person
might need to use his/her memory to manage in that situation,
e.g. ‘You have made an appointment and need to remember to
go along’ or ‘You have promised to do something later in the day
and need to remember to do it at the right time’ (Hardy et al.,
2006). The caregiver version of the questionnaire requires them
to rate the TBI participant they care for, and therefore enables
a comparison of the outcomes from the viewpoints of both TBI
participants and caregiver participants. A higher score indicates
better memory functions.
Autobiographical memory questionnaire: This questionnaire
aims to evaluate the TBI participants’ autobiographical memory
function. The researcher and participants would choose one task
that the participant had completed during the previous two weeks
and the details of this were used for the questionnaire. The partic-
ipants were first asked to describe the task and then complete the
questionnaire (Fitzgerald and Broadbridge, 2013) which consists
of 18 questions which are answered using an eight-point Likert
scale, ranging from 0 (not at all) to 7 (as much as any memory/as
12
Int. J. Human-Computer Studies 117 (2018) 4–19
clearly as if it were happening right now). A higher score indi-
cates better autobiographical memory functions.
Support burden inventory: In order to investigate the care-
giver participants’ burden when assisting TBI participants with
the their memory impairments, the “Support Burden Inven-
tory” (Novak and Guest, 1989) was used. This investigates five
factors of the burden on the caregiver using 24 questions. Over-
all a lower score indicates a lighter burden. The factors are time-
dependence burden, developmental burden, physical burden, so-
cial burden and emotional burden. The caregiver also kept a
memory log in a provided booklet to record their observations of
the TBI participants” behaviours when accomplishing the tasks.
MyMemory evaluation questionnaire: This questionnaire was de-
signed specifically to evaluate the effectiveness and impact of
MyMemory from the perspective of the TBI participants. It also
provided an opportunity for these participants to give feedback
and suggestions on the design of the app. Participants were asked
to rate each of the functions, as well as the app overall, using a
five level scale, ranging from 0 (not at all helpful) to 4 (very help-
ful), and then answer an open-ended question asking for any sug-
gestions. The app itself has a built in log function which allows
us to analyse how, and how often, the participants used the app.
Analysis of these logs was conducted in conjunction with analy-
sis of the questionnaires to further understand the participants”
responses.
6.2. Participants
The study participants were selected from the group of partici-
pants in our earlier interview studies (conducted in the first stages
of the research project, see Section 4). Nine participants volunteered
for the 8-week long study: five TBI participants, one participant with
memory problems associated with an underlying medical condition
(multiple sclerosis) and three care-givers. For the purpose of the
study the participant with multiple sclerosis was treated in the same
way as the TBI participants with the focus on memory impairment
solely. Table 2 shows a summary of the six TBI participants” demograph-
ics (T1–T6). The average age of the participants was 45 year (median
42). The average range of years post-injury was 20 years. T2, T4 and
C. Chang et al.
Table 2
TBI participants demographics. (P: Part-time job of 20 h/week).
Table 3
Caregiver participants demographics. (P: Part-time job of 20 h/week).
T6 have part-time jobs, T1 has a voluntary job for 10 h per week, T5
is a student and T3 is diagnosed as unable to work. T1, T2 and T5 rely
on their caregivers to remind them of daily events. T1, T2 and T6 regu-
larly recorded their schedule in a paper diary. T3, T4 and T5 relied on
their mobile device’s calendar with the alarm to remind them of their
schedule. T6 also uses Facebook and to-do lists as a memory aid. Writ-
ing down their memories is a common way for all TBI participants to
remember.
Table 3 presents a summary of the caregiver participants’ demo-
graphics. Each is a relative of a TBI survivor (C1 – husband, C2 – son
and C3 – mother). All of them help their TBI-relatives in the health and
medical area, work and household chores. Verbal reminders are the most
common strategy they use to help their relatives to remember.
13
Int. J. Human-Computer Studies 117 (2018) 4–19
6.3. User study results
The first goal of the study was to determine whether MyMemory,
and the approach taken to augment autobiographical memory, actually
helps TBI survivors to improve their memory ability. The second goal
was to determine if MyMemory could improve the quality of life of TBI
survivors and people around them. This section describes the findings
of the study and relates these to the goals.
6.3.1. TBI participants’ observations
TBI participants reported improvements when they used MyMemory.
The improvements were seen in their well-being, memory function and
autobiographical memory during the study. We discuss each of these in
turn.
C. Chang et al. Int. J. Human-Computer Studies 117 (2018) 4–19

Fig. 6. Aggregated results of Well-being index (left) and memory function (right).

Well-being. TBI participants reported positive changes in their well-

being (see Fig. 6, left). Overall, the scores for their well-being (using
the WHO index) during the baseline phases, A1 and A2, were lower
(mean of 14.8 and 15.8, respectively) than the scores during the phases
with MyMemory, B1 and B2 (mean of 16.8 and 17.8, respectively). Ad-
ditionally, the results after the first use of MyMemory (B1) were lower
than the second time (B2). One possible explanation for this is that TBI
participants required time to learn to use MyMemory and adapt to the
changes required to use it as their memory aid. This was supported by
observations from the caregivers. According to TBI participants’ reports,
they felt more confident about their memory abilities after first using
MyMemory (B1 phase), and this positive influence continued until the
next phase (A2 phase: without MyMemory). Some of them also reported
that their quality of sleep was better and that they had found their fa- Fig. 7. Accumulated results of autobiographical memory questionnaire.
tigue levels decreased after they used MyMemory.
One unexpected finding was how quickly some of the TBI partici-
pants came to rely on MyMemory in the short time of the first phase of experienced some changes in her personal life around child-care which
use (two weeks). Participants described how they felt anxious when they caused considerable stress and changed her memory needs. According
were asked to stop using MyMemory and return to their own memory to her interview, she relied on the reminders of her existing memory
aids. This negative emotion also reflected on their well-being scores in aids to track her own and her son’s schedules. These changes were re-
A2. Even so, most participants presented well-being scores in A2 which flected in the results of her memory function scores, the highest result
were higher than A1. Overall, the findings show that TBI participants was with her own memory aid (in A2), not with MyMemory. She said,
were happier after they had used MyMemory and at the end of the study. “I do believe MyMemory could improve my memory ability, but now I
We might interpret this to mean that the positive influences produced need a reminder with an alarm to track my busy schedule.” This need
after they used MyMemory accumulated and remained even after they for alarm may also indicate a useful future extension of MyMemory.
had stopped using the app.
Autobiographical memory. Participants used MyMemory to record mem-
ory cues and review the recorded memory data as part of the training
Memory ability. Our findings suggest that the use of MyMemory im-
process. Their aggregated scores for the autobiographical memory ques-
proved TBI participants’ memory function capacities during the study
tionnaire are shown in Fig. 7. Overall, the scores of all participants dur-
(see Fig. 6, right). MyMemory allowed participants to record memory
ing phases B1 and B2 (using MyMemory) were higher (mean of 88.2 and
cues related to events and train their memory with the flashcards and the
90.5, respectively) than for phases A1 and A2 (mean of 79.7 and 78.7,
widget. The scores from the memory function scale during the baseline
respectively). In addition, the results of participants in the B2 phase (sec-
phases, A1 and A2, were lower (mean of 24.3 and 26.8, respectively)
ond time with MyMemory) were highest in the study. Most participants
than the phases with MyMemory, B1 and B2 (mean of 25.8 and 29.5,
reported that their autobiographical memory was improved when they
respectively). Additionally, most participants reported that they felt that
first used MyMemory, however, the effects of MyMemory ceased when
their memory function had improved after using MyMemory (T1, T2, T3,
participants returned to their own memory aid. These results are similar
and T5). Comparing the two with-treatment phases (B1 and B2) where
to the results from the memory functioning scale. It again suggests that
participants were using MyMemory, most participants found their mem-
improvements stop when TBI participants do not use MyMemory. Again
ory functions were better in the B2 phase than in the B1 phase, the scores
this differs from the progress of well-being where the effects appear to
for memory function after the B2 phase were the highest in the study.
be accumulative and persist across subsequent phases of the study.
Unlike the well-being scores, improvement in memory ability stopped
when participants returned to using their own memory aids. This sug-
gests the effect of MyMemory in improving memory function requires 6.3.2. Caregiver participant’s observations
TBI survivors to use it long term. Caregiver participants reported their observations on their relative’s
Several participants (T2, T3, T5, and T6) reported that after having use of MyMemory. The observations further contain self-reporting of any
used MyMemory for phase B1, they then copied the concept of factors burden amelioration and their observations about the improvements in
of memory and used it to record events with their own memory aids in the memory functioning of their TBI relatives.
phase A2. This may be one possible reason that some results in the A2
phase were higher than the A1 phase. T4, however, had a lower result Reduction of caregiver participants’ burden. One aim of MyMemory is to
in the B2 phase than in B1 or A2. She reported that during the study she reduce the caregiver’s burden. The results from the “Support Burden

14
C. Chang et al.
Fig. 8. Aggregated results of Support Burden Inventory (lower scores are bet-
ter).
Inventory” do not give any conclusive indication that caregivers’ bur-
den decreased when TBI participants used MyMemory, see Fig. 8 (note
that here lower scores are better). However there were only three care-
giver participants in the study and each of them are involved in different
roles with their TBI relatives who require different levels of support and
time. Participant C1 provides 24-h-a-day comprehensive assistance for
his wife every day of the week. It was difficult for him to detect the ef-
fects of changes to his wife’s memory in their daily life. Occasionally, he
found his wife remembered to feed the cat or discussed with him her last
phone call. While these changes were observed after she used MyMem-
ory, his overall score for the support burden remained fairly flat across
all four phases of the study. In contrast, C2 who supports his mother in
various ways for a limited amount of time found it easier to note his
mother’s improvement during the with-treatment phases of the study
(B1 and B2). For example, his mother remembered to pay a bill with-
out him reminding her. The scores from C2’s burden inventory show
marked differences between phases, with both B1 and B2 indicating a
reduced burden. Finally C3 showed a reduced burden between the A1
and B1 phases, but this was not repeated in the second half of the study,
with results after B2 showing a burden score similar to the start of the
study. We infer from these results that burden amelioration is relative
to the amount of burden experienced overall (therefore it is easier to see
improvements when the burden is lower). Beyond that, a larger study
of caregivers is required to fully understand the effects of this.
6.3.3. Evaluation questionnaire and log analysis
This section reports on the TBI participants’ activities and records
with MyMemory based on the MyMemory log files. These results pro-
vide some information about what kind of memories TBI participants
want to use MyMemory for recording and retrieving and how they in-
teract with the app. In addition it provides the results of the qualitative
questionnaire and feedback provided by the participants which give fur-
ther information about how well they think the app supports their needs
and how it might be improved. Each of the main functions of the app
are considered in turn.
Capturing memory cues. There are two ways in which users can start
the process of recording memory cues: (1) selecting the ‘new memory’
option from the main menu and entering textual information, and (2)
by taking a photograph as the starting point of recording memory cues.
From the log files we saw that most participants started a new memory
using the menu and text (all used this method at least once) and one
participant never started from a photo. Fig. 9 (1st and 2nd block from
left) show the feedback participants gave for the usefulness of each of
the two options of creating memory cues.11 Overall most participants
11
Empty bars mean that some participants did not answer this question.
15
Int. J. Human-Computer Studies 117 (2018) 4–19
felt that both options would be very useful (receiving the maximum 4
points on the 0-to-4 scale). However, most participants did not attach
photos to their event records.
These results suggest that use of photos is a question of personal pref-
erence for TBI survivors, and they do not see it as an essential part of
their memory record. Some TBI participants gave an explicit account of
their reactions to taking photos when they used MyMemory. For exam-
ple, T2 reported she felt embarrassed taking some photos, such as taking
a picture of a new acquaintance for recording a memory. Nevertheless,
T3 found her memory could be retrieved more quickly by reading the
memory cues with both a textual description and a photo in MyMem-
ory. She was the only participant who attached a photo to each memory
record in the study.
Colour labels and memory record structure. MyMemory can also cat-
egorise memory records using different label colours. The ability to
change label colour to categorise memory records was given high scores
(1 × 3 and 3 × 4) by all participants after the second phase using MyMem-
ory (B2), see Fig. 9, 3rd block. All of them used colours to categorise
memory records and were able to remember the colour that represents
each category. MyMemory offers six colours for the labels, and each
participant used at least four colours. During the interviews, the partic-
ipants precisely described what each colour represented in terms of the
category. Some participants also reported that the colour label is better
than using a folder to organise documents (on a computer) for organis-
ing their memories. From these results we conclude that colour coding
is a very important factor for supporting TBI survivors’ memory.
All six participants found the method of presenting and reviewing the
memory data helpful (scores 3–4), see 4th block in Fig. 9. T2 additionally
asked for more opinions for the activities, such as meetings. T4 disliked
the time field options and would have preferred exact times, such as
10.00 a.m. (instead of ‘morning’).
Backup and sharing. Backup and sharing of memory data can be carried
out using email or text messages. In the feedback questionnaire, most
TBI participants gave high scores for both mechanisms for backup and
sharing of data (see Fig. 10, left). However, T5 did not find either backup
function useful as she felt she did not need these.
Finally, upon reviewing the users’ activities from the logs, we found
that TBI participants used the email function more than the text message
option.
Training tools. The training tools are the Memory FlashCards and Ticker
Widget, which are designed for training and improving the user’s mem-
ory ability. In the feedback participants gave both tools a high score
(see Fig. 10, right). The Ticker Widget got a very positive feedback from
all TBI participants after phase B2. All of them reported that they felt
the widget was a tool that could improve their memory. According to
the log files, all participants had used the widget, with the most use
being observed for T3 (39 memories). However, we noted marked dif-
ferences for the Memory Flashcards: the log files revealed that only half
of the participants had used the FlashCard function. Some participants
explained that because the Ticker Widget is located on the phone’s main
screen, they can use it easily after they turn on their cell phone. By con-
trast, the FlashCard memory training is located within MyMemory, and
so the user has to access MyMemory first to use the training tool. It is
convenient for them to check once they are using the app, but not to
access in general.
The participants also reported on reviewing the stored memory cues
without the FlashCards or Ticker Widget. For example, T3 reported that
she reviewed the stored memory cues of the previous week at the end
of the week by working through the list of memories, looking at the cue
details for each in turn. Similarly T5 reported reviewing her stored mem-
ory cues in the same way (via the memory overview on the MyMemory
main page) at the end of each day before going to bed. As T5 had also
C. Chang et al. Int. J. Human-Computer Studies 117 (2018) 4–19

Fig. 9. MyMemory evaluation questionnaire: Capturing and management.

Fig. 10. MyMemory evaluation questionnaire: Backup and Training.

Fig. 11. MyMemory evaluation questionnaire: Sorting memory records.

captured a photo for each memory, this effectively led to them review-
ing each memory twice (during photo selection and in the evening). In
contrast, T6 reported reviewing memory cues only when prompted by
someone (e.g., when asked about a particular event).
Sorting memory records. In addition to reviewing memory records using
their default presentation (ordered by time) on the main page, memory
cues can be searched for by ordering the lists using either ‘Sort by Person’
and ‘Sort by Location’. Most participants initially did not know how to
use these sorting tools when they first used MyMemory. However, after
an explanation and demonstration they understood how these two sort-
ing functions worked and how these might help them review or search
for memories (see Fig. 11). According to the log files, most participants
did not use either of the two sorting functions. During the interview,
we asked all participants about the low usage of these searching ap-
proaches. Most reported that the number of memory records stored was
not large enough for them to need to do a search.
In contrast, T5 reported that because they reviewed each memory
record at the end of the day, they did not need to go back later and
search for records as they remembered. They reported that they found
sort by location more useful than sort by person (as it was a detail that
was always provided). We cannot, therefore, conclude anything specific
yet about the usefulness of these functions and this will require further
study involving more participants with larger data sets over time.
Memory records of events. In our initial interview user study, we iden-
tified three particular memory problems that TBI survivors find most
distressing: ‘Remembering New Acquaintances’, ‘Remembering Ad-hoc
Events’ and ‘Remembering Instructions’. In this study, most of the
records of all participants in MyMemory related to ‘Remembering Ad-
hoc Events’. T3 and T4 had one record each related to ‘Remembering

16
C. Chang et al.
New Acquaintances’ respectively and T1 had one record related to ‘Re-
membering Instructions’. The total number of memory records was 71
for four TBI participants for B1 and B2 phases. The maximum number
of memory records was five for one day. These numbers suggest the TBI
participants might not record every event they experienced.
Each memory recorded usually involved the participant themselves
and sometimes other people. However, a number of records showed
events only related to the participants themselves: T1 had two out of
five that only related to themselves, T2 had three out of 29, T3 had
seven out of 29 and T4 had two out of eight. We observed that most of
these records concerned personal achievements with important mean-
ings for themselves. For example, T3 recorded clearing her room, which
was the first time she had cleared her lounge since she had a traumatic
brain injury. She said “I don’t need to tell other people about it but I
want to remember it”.
From this initial study it appears that in addition to the particu-
lar memory problems TBI survivors can identify, they are also keen to
support themselves in other ways, such as reinforcing self-value and
achievements by remembering personal experiences which reflect this.
Although this was not one of the design goals for MyMemory all partic-
ipants used it in this way.
Recording and editing memory details. Analysis of memory records and
log reports showed that most TBI participants provided full details of
each memory in MyMemory. T1 and T3 completed all their records with
full details. However, T2 had seven records (out of 29) without full de-
tails and T4 had two records (out of eight). Furthermore, from checking
the log report we could see that T1 was the only participant who had
never edited a memory record after initially recording it, all others had
experience of editing their records more than once. T2 had used the edit
function once, T4 twice, and T3 three times.
The fact that most participants were able to enter full details the
first time they entered an event suggests that the five factors of memory
approach support the way their short-term memory stores information
and can be used to subsequently retrieve the memories.
7. Discussion
Our work can be categorised within the wider research interest in
providing technological interventions as health solutions and in-home
support and care for the sick and elderly. However, one of the things that
was made clear by our initial interview studies and subsequent app eval-
uations is that TBI survivors have a particular set of needs and technol-
ogy use requirements that means most general solutions will not work
for them. Some evidence of this is provided by the fact that none of
our participants made regular use of any existing technology solutions;
even online calendars were not seen as particularly helpful, apart from
the use of inbuilt alarms which could be set to act as reminders. Not
only do we therefore need to consider TBI survivors as a distinct group
when creating technological solutions for memory aids, but as we have
also seen, we need to be mindful of the impact of involving them in
evaluation studies. While these are crucial in ensuring we tailor the so-
lutions appropriately, they also have the potential to add to the mental
burden of TBI participants which may be detrimental to their overall
well-being.
Related to this is the need to tailor interviews, observational studies
and evaluation studies in ways that make them easier for TBI partic-
ipants to participate. We discovered early in our first interview study
that we could not just set up appointments with the participants and
assume they would turn up or be available at designated times. Rather,
for each of the studies we conducted, we would have to follow up each
participant several times prior to meeting to have a better chance of
the appointment taking place. We also “lost” several participants at dif-
ferent stages due to losing contact with them. Given the prevalence of
TBI, particularly among younger (working-age) people, the importance
of supporting them in ways which enable them to more fully engage in
17
Int. J. Human-Computer Studies 117 (2018) 4–19
a “normal” lifestyle is magnified. Despite the challenges in setting up
and running appropriate studies, it appears to us that this is worthy of
ongoing research.
Our initial interviews elicited that many participants were self-
conscious about their condition and were not willing to use technologi-
cal solutions if these felt they would draw attention to their condition.
The prevalence of the use of mobile phones in almost all social circum-
stances is helpful in this regard, as it is not unusual to see someone
interacting with their phone even if they are in company. However it
does mean that some strategies that might appear to be intrinsically
useful in recording and storing memories (for example taking photos
of new acquaintances to support the need to better remember people
you meet and their names) are not appropriate or acceptable for the
majority of TBI survivors. It also suggests that the app is more likely to
be acceptable to younger people with TBI (who are in the demographic
of high mobile and app use) and that older people with TBI may take
longer to adjust to its use or to easily incorporate it into their daily
life.
When we consider the results from the final evaluation study for
the MyMemory app, we need to bear in mind a number of limita-
tions. This includes the length of time of the study (8 weeks) and
the number of participants. The study design meant that the TBI par-
ticipants changed their memory aids every fortnight. This is particu-
larly challenging for TBI survivors who rely on consistency in their ap-
proaches to support their memory, taking part in the study therefore
had the potential to increase the amount of stress in their daily life.
This switching between memory aids may have an effect on the results
and the feelings of participants towards using the app. A better solu-
tion for future studies may be to investigate the use of longer, single-
person studies, to try and measure the effects of using the app consis-
tently over longer periods of time. We already have one of the partic-
ipants from our study who has continued to use MyMemory beyond
the study and has incorporated it into their everyday memory support
systems.
We must also be clear that our results do not provide any clinical
proof that using the app has actually improved the memory capabilities
of the TBI participants. The memory questionnaires completed by the
participants are their reported perceptions of their memory and auto-
biographical memory capabilities. However, we feel that if the partic-
ipants perceive that their memory has improved this is more likely to
contribute to their overall well-being (which was reflected in the well-
being scores) which is itself a worthwhile contribution.
It has become apparent from the way in which participants used
the app in our user study that the inbuilt training methods (FlashCards
and Ticker Widget) did not align particularly well with the participants’
daily life. The Ticker Widget worked better than the FlashCard training
because it was available on the phone’s home screen (i.e., participants
did not have to remember to use it). However, what seemed to work
best was a regular revision of captured memory cues. Participant T5 en-
gaged in regular daily revision while participant T3 engaged in weekly
revision. In both cases the revisions were based on recent entries in
MyMemory, which is a process not directly supported in MyMemory to
the same extent as the other two revision/training tools. One of the rea-
sons cited for engaging with MyMemory in such a revision mode (and
not via the training tools provided) was that the participants wished to
use the photo as a memory cue.
For supporting such a revision and rehearsal of the memory records
the users have suggested that it might be appropriate to incorpo-
rate a (daily) review feature into the MyMemory functionality. How-
ever, as with the FlashCard feature, it will only be beneficial if the
users are willing to engage in the functionality offered. Reminder
functions (or even gamification of the reviewing process) to trig-
ger the user’s engagement may have to be incorporated. Alterna-
tively, incorporating the user-generated photographs into the two ex-
isting training tools was suggested by participants and should also be
explored.
C. Chang et al.
8. Summary and future work
This article introduced MyMemory, a mobile augmented memory
system that aims to assist TBI survivors in coping with their memory
impairments. We identified factors that may help TBI survivors with
their memory retrieval and used these to record cues for people’s (au-
tobiographical) memories. Thus MyMemory helps people record mem-
ory cues about events in their lives, which may be looked up later to
trigger remembering. MyMemory also offers memory training based on
these memory cues, using the concept of memory training flashcards and
memory-ticker widgets. Our MyMemory prototype was evaluated in a
user study involving both TBI survivors and caregivers. The participants
with TBI reported improvements to their well-being, memory function
and autobiographical memory. Their caregivers confirmed these obser-
vations but did not necessarily feel a reduction of their caregiver burden.
Plans for future work in this research involve larger-scale stud-
ies.This will include the development of the MyMemory app for other
platforms to extend the pool of possible participants in future studies.
We also wish to further examine the longer term effects of using the
app via single-person case studies (such as T5 for example) to under-
stand how we can better tailor the app to support this. These additional
studies with a larger group of participants will allow us to get a bet-
ter understanding of both the short- and long-term effects of the use of
MyMemory on both autobiographical memory and well-being. We have
already started exploring gamification of memory training for TBI sur-
vivors using location-based augmented reality card games (Gilbert et al.,
2017) as well as further exploration of community-based support for TBI
survivors through a memory app in which several people can contribute
to the store of memory cues (Khong, 2016).
Acknowledgments
The authors wish to gratefully acknowledge and thank the study
participants for their involvement and cooperation, and particularly for
the considerable time and personal engagement they contributed to this
study.
References
Acampora, G., Cook, D.J., Rashidi, P., Vasilakos, A.V., 2013. A survey on ambient intelli-
gence in healthcare. Proc. IEEE 101 (12), 2470–2494.
Adam, H., Soh, P., 2016. Development of a health monitoring vest for the elderly. In: 2016
IEEE MTT-S International Conference on Numerical Electromagnetic and Multiphysics
Modeling and Optimization (NEMO). IEEE, pp. 1–2.
Adami, A.M., Pavel, M., Hayes, T.L., Singer, C.M., 2010. Detection of movement in bed us-
ing unobtrusive load cell sensors. IEEE Trans. Inf. Technol. Biomed. 14 (2), 481–490.
Andreadis, S., Stavropoulos, T.G., Meditskos, G., Kompatsiaris, I., 2016. Dem@ home: am-
bient intelligence for clinical support of people living with dementia. In: International
Semantic Web Conference. Springer, pp. 357–368.
Barker-Collo, S., Wilde, N., Feigin, V., 2008. Trends in head injury incidence in new
zealand: a hospital-based study from 1997/1998 to 2003/2004. Neuroepidemiology
32 (1), 32–39.
Barthel, R., Mackley, K.L., Hudson-Smith, A., Karpovich, A., de Jode, M., Speed, C., 2013.
An internet of old things as an augmented memory system. Pers Ubiquitous Comput.
17 (2), 321–333.
Beer, J.M., Smarr, C.-A., Chen, T.L., Prakash, A., Mitzner, T.L., Kemp, C.C., Rogers, W.A.,
2012. The domesticated robot: design guidelines for assisting older adults to age in
place. In: Proceedings of the Seventh Annual ACM/IEEE International Conference on
Human–Robot Interaction. ACM, pp. 335–342.
BrainLine, 2013. Life-changing mobile apps for people with brain injury. http://www.
brainline.org/content/2013/12/life-changing-iphone-and-ipad-apps-for-people-with-
brain.html.
Bright Focus Foundation, 2011. Alzheimer’s Disease research.
http://www.brightfocus.org/alzheimers/. Last accessed January 2012. [Date
accessed: 2011].
Broekens, J., Heerink, M., Rosendal, H., 2009. Assistive social robots in elderly care: a
review. Gerontechnology 8 (2), 94–103.
Browne, G., Berry, E., Kapur, N., Hodges, S., Smyth, G., Watson, P., Wood, K., 2011.
Sensecam improves memory for recent events and quality of life in a patient with
memory retrieval difficulties. Memory 19 (7), 713–722.
Canzian, L., Musolesi, M., 2015. Trajectories of depression: unobtrusive monitoring of de-
pressive states by means of smartphone mobility traces analysis. In: Proceedings of the
2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing.
ACM, pp. 1293–1304.
18
Int. J. Human-Computer Studies 117 (2018) 4–19
Cardinale, A. M., 2016. Traumatic brain injury: the hidden epidemic nobody wants
to talk about. Psych Central online article at https://psychcentral.com/blog/
archives/2016/03/28/traumatic-brain-injury-the-hidden-epidemic-nobody-wants-to-
talk-about.
Carroll, L., Cassidy, J.D., Peloso, P., Borg, J., Von Holst, H., Holm, L., Paniak, C., Pépin, M.,
2004. Prognosis for mild traumatic brain injury: results of the who collaborating cen-
tre task force on mild traumatic brain injury. J. Rehabil. Med. 36 (0), 84–105.
Cavallo, M.M., Key, T., 2005. The Family System. In: Jonathan M, S., McAllister, T.W.,
Yudofsky, S.C. (Eds.), Textbook of Traumatic Brain Injury. American Psychiatric As-
sociation Publishing, Washington, DC, pp. 533–558.
Chang, C.C., 2011. The Dictionary of Psychology. Taiwan Tunghua, Taiwan.
Chang, C.S.-P., 2017. A Mobile Augmented Memory Aid for People with Traumatic Brain
Injury. Ph.D. thesis. University of Waikato, Hamilton, New Zealand.
Chen, T.L., Ciocarlie, M., Cousins, S., Grice, P.M., Hawkins, K., Hsiao, K., Kemp, C.C.,
King, C.-H., Lazewatsky, D.A., Leeper, A.E., et al., 2013. Robots for humanity: using
assistive robotics to empower people with disabilities. IEEE Robot. Autom. Mag. 20
(1), 30–39.
Chen, T.L., King, C.-H., Thomaz, A.L., Kemp, C.C., 2011. Touched by a robot: an inves-
tigation of subjective responses to robot-initiated touch. In: Proceedings of the 6th
ACM/IEEE International Conference on Human–Robot Interaction (HRI), 2011. IEEE,
pp. 457–464.
Costa, Â., Castillo, J.C., Novais, P., Fernández-Caballero, A., Simoes, R., 2012. Sensor–
driven agenda for intelligent home care of the elderly. Expert Syst. Appl. 39 (15),
12192–12204.
De Leo, G., Brivio, E., Sautter, S.W., 2011. Supporting autobiographical memory in pa-
tients with Alzheimer’s disease using smart phones. Appl. Neuropsychol. 18 (1),
69–76.
Dobbins, C., Merabti, M., Fergus, P., Llewellyn-Jones, D., 2014. The big data obstacle of
lifelogging. In: Proceedings of the 2014 28th International Conference on Advanced
Information Networking and Applications Workshops (WAINA). IEEE, pp. 922–926.
Doherty, A.R., Pauly-Takacs, K., Caprani, N., Gurrin, C., Moulin, C.J.A., O’Connor, N.E.,
Smeaton, A.F., 2012. Experiences of aiding autobiographical memory using the sense-
cam. Hum. Comput. Interact. 27 (1–2), 151–174.
Dritschel, B., Kogan, L., Burton, A., Burton, E., Goddard, L., 1998. Everyday planning dif-
ficulties following traumatic brain injury: a role for autobiographical memory. Brain
Inj. 12 (10), 875–886.
Ebling, M.R., 2016. Can cognitive assistants disappear? IEEE Pervasive Comput. 15 (3),
4–6.
Feigin, V., Barker-Collo, S., Krishnamurthi, R., Theadom, A., Starkey, N., 2010. Epidemiol-
ogy of ischaemic stroke and traumatic brain injury. Best Pract. Res. Clin. Anaesthesiol.
24 (4), 485–494.
Feigin, V.L., Theadom, A., Barker-Collo, S., Starkey, N.J., McPherson, K., Kahan, M., Dow-
ell, A., Brown, P., Parag, V., Kydd, R., Jones, K., Jones, A., Ameratunga, S., 2012.
Incidence of traumatic brain injury in new zealand: a population-based study. Lancet
Neurol. 12 (1), 53–64.
Feng, V., 2014. Mild Traumatic Brain Injury Burden in New Zealand: A Population-based
Incidence and Short Term Outcomes Study. Ph.D. thesis. ResearchSpace@ Auckland.
Ferguson, S., Friedland, D., Woodberry, E., 2015. Smartphone technology: gentle re-
minders of everyday tasks for those with prospective memory difficulties post-brain
injury. Brain Inj. 29 (5), 1–9.
Fitzgerald, J.M., Broadbridge, C.L., 2013. Latent constructs of the autobiographical mem-
ory questionnaire: a recollection-belief model of autobiographical experience. Mem-
ory 21 (2), 230.
Frank LoPresti, E., Mihailidis, A., Kirsch, N., 2004. Assistive technology for cognitive re-
habilitation: state of the art. Neuropsychol. Rehabil. 14 (1–2), 5–39.
Gilbert, L., Hinze, A., Bowen, J., 2017. Augmented reality game for people with traumatic
brain injury: concept and prototypical exploration. In: Proceedings of the ICCAE Con-
ference, pp. 51–55.
Gravenhorst, F., Muaremi, A., Bardram, J., Grünerbl, A., Mayora, O., Wurzer, G., Frost, M.,
Osmani, V., Arnrich, B., Lukowicz, P., et al., 2015. Mobile phones as medical devices
in mental disorder treatment: an overview. Pers. Ubiquitous Comput. 19 (2), 335–353.
Grünerbl, A., Muaremi, A., Osmani, V., Bahle, G., Oehler, S., Tröster, G., Mayora, O., Har-
ing, C., Lukowicz, P., 2015. Smartphone-based recognition of states and state changes
in bipolar disorder patients. IEEE J. Biomed. Health Inform. 19 (1), 140–148.
Gurrin, C., Smeaton, A.F., Doherty, A.R., et al., 2014. Lifelogging: Personal Big Data. Foun-
dations and Trends® in Information Retrieval 8 (1), 1–125.
Hao, Y., Foster, R., 2008. Wireless body sensor networks for health-monitoring applica-
tions. Physiol. Meas. 29 (11), R27.
Harbouche, A., Erradi, M., Kobbane, A., 2013. A flexible wireless body sensor network
system for health monitoring. In: 2013 IEEE 22nd International Workshop on Enabling
Technologies: Infrastructure for Collaborative Enterprises (WETICE). IEEE, pp. 44–49.
Hardy, R.M., Oyebode, J.R., Clare, L., 2006. Measuring awareness in people with mild to
moderate Alzheimer’s disease: development of the memory awareness rating scale-ad-
justed. Neuropsychol. Rehabil. 16 (2), 178–193.
Haux, R., Koch, S., Lovell, N., Marschollek, M., Nakashima, N., Wolf, K., et al., 2016.
Health-enabling and ambient assistive technologies: past, present, future. IMIA Yearb.
Health Care POV, 2012. Speaking of apps: Introducing memory strategies.
http://community.advanceweb.com/blogs/sp_6/archive/2012/10/15/introducing
-memory-strategies.aspx.
Heffner, C.L., 2004. All Psych Online.
Heun, R., Bonsignore, M., Barkow, K., Jessen, F., 2001. Validity of the five-item WHO
well-being index (WHO-5) in an elderly population. Eur. Arch. Psychiatry Clin. Neu-
rosci. 251 (2), 27–31.
Hewitt, J., Evans, J.J., Dritschel, B., 2006. Theory driven rehabilitation of executive func-
tioning: improving planning skills in people with traumatic brain injury through the
C. Chang et al.
use of an autobiographical episodic memory cueing procedure. Neuropsychologia 44
(8), 1468–1474.
Hodges, S., Berry, E., Wood, K., 2011. Sensecam: a wearable camera that stimulates and
rehabilitates autobiographical memory. Memory 19 (7), 685–696.
van den Hoven, E., Eggen, B., 2008. Informing augmented memory system design through
autobiographical memory theory. Pers. Ubiquitous Comput. 12 (6), 433–443.
van den Hoven, E., Eggen, B., 2015. The cue is key. Z. Psychol.
Iwamura, M., Kunze, K., Kato, Y., Utsumi, Y., Kise, K., 2014. Haven’t we met before?: a
realistic memory assistance system to remind you of the person in front of you. In:
Proceedings of the 5th Augmented Human International Conference. ACM, p. 32.
Jahr, D., 2017. Campaign to “fight the hidden epidemic” for the millions with
TBI, concussions. Neuro-Laser Foundation online article at http://www.tbi.
care/2017/02/campaign-fight-hidden-epidemic-millions-tbi-concussions.
Khong, E.B.-Y., 2016. Exploring Community-Based Health Care Using Mobile Applications.
Masters thesis. University of Waikato.
Knight, R.G., O’Hagan, K., 2009. Autobiographical memory in long-term survivors of se-
vere traumatic brain injury. J. Clin. Exp. Neuropsychol. 31 (5), 575–583.
Lee, M., Dey, A., 2006. Capturing and reviewing context in memory aids. Workshop on
Designing Technology for People with Cognitive Impairments.
Lee, M., Dey, A., 2007. Providing good memory cues for people with episodic memory
impairment. In: ACM SIGACCESS Conference on Computers and Accessibility. ACM,
pp. 131–138.
Lee, M., Dey, A., 2008. Lifelogging memory appliance for people with episodic memory
impairment. In: Conference on Ubiquitous computing UbiComp ’08. ACM, pp. 44–53.
Masel, B.E., DeWitt, D.S., 2010. Traumatic brain injury: a disease process, not an event.
J. Neurotrauma 27 (8).
McCullagh, S., Feinstein, A., 2011. Cognitive Changes, 2. American Psychiatric Press,
pp. 279–294.
Mordoch, E., Osterreicher, A., Guse, L., Roger, K., Thompson, G., 2013. Use of social com-
mitment robots in the care of elderly people with dementia: a literature review. Ma-
turitas 74 (1), 14–20.
Nelson, L.A., MacDonald, M., Stall, C., Pazdan, R., 2013. Effects of interactive metronome
therapy on cognitive functioning after blast-related brain injury: a randomized con-
trolled pilot trial. Neuropsychology 27 (6), 666.
Neumann, S., Banks, R., Dörk, M., 2017. Memory dialogue: exploring artefact-based mem-
ory sharing. In: Proceedings of the 2017 CHI Conference Extended Abstracts on Hu-
man Factors in Computing Systems. ACM, pp. 884–895.
Nielsen, J., 2012. Thinking aloud: The first usability tool. http://www.nngroup.
com/articles/thinking-aloud-the-1-usability-tool/. [Date accessed: 2015].
Novak, M., Guest, C., 1989. Application of a multidimensional caregiver burden inventory.
Gerontologist 29 (6), 798–803.
Pantelopoulos, A., Bourbakis, N.G., 2010. A survey on wearable sensor-based systems for
health monitoring and prognosis. IEEE Trans. Syst., Man, Cybern., Part C (Appl. Rev.)
40 (1), 1–12.
Petrelli, D., Whittaker, S., 2010. Family memories in the home: contrasting physical and
digital mementos. Pers. Ubiquitous Comput. 14 (2), 153–169.
Pollack, M.E., Brown, L., Colbry, D., McCarthy, C.E., Orosz, C., Peintner, B., Ramakrish-
nan, S., Tsamardinos, I., 2003. Autominder: an intelligent cognitive orthotic system
for people with memory impairment. Rob. Auton. Syst. 44 (3), 273–282.
Rantz, M.J., Skubic, M., Koopman, R.J., Phillips, L., Alexander, G.L., Miller, S.J., Gue-
vara, R.D., 2011. Using sensor networks to detect urinary tract infections in older
adults. In: 2011 13th IEEE International Conference one-Health Networking Applica-
tions and Services (Healthcom). IEEE, pp. 142–149.
19
Int. J. Human-Computer Studies 117 (2018) 4–19
Rashidi, P., Cook, D.J., 2009. Keeping the resident in the loop: adapting the smart home
to the user. IEEE Trans. Syst., Man, Cybern. -Part A: Syst. Hum. 39 (5), 949–959.
Rashidi, P., Mihailidis, A., 2013. A survey on ambient-assisted living tools for older adults.
IEEE J. Biomed. Health Inform. 17 (3), 579–590.
Richmond, J., 2012. Digital Parrot on Android. COMP590 Report. The University of
Waikato, Hamilton, New Zealand.
Rodriguez-Sanchez, M., Moreno-Alvarez, M., Martin, E., Borromeo, S., Hernandez-Ta-
mames, J., 2014. Accessible smartphones for blind users: a case study for a wayfinding
system. Expert Syst. Appl. 41 (16), 7210–7222.
Schweer, A., 2011. Augmenting Autobiographical Memory: An Approach Based on Cog-
nitive Psychology. Ph.D. thesis. University of Waikato, Hamilton, New Zealand.
Schweer, A., Jones, S., Hinze, A., 2009. Trails of experiences: navigating personal memo-
ries. In: Proceedings of the CHINZ. ACM, pp. 105–106.
Sellen, A.J., Whittaker, S., 2010. Beyond total capture: a constructive critique of lifelog-
ging. Commun. ACM 53, 70–77.
Shipstead, Z., Hicks, K.L., Engle, R.W., 2012. Working memory training remains a work
in progress. J. Appl. Res. Mem. Cogn. 1 (3), 217–219.
Smarr, C.-A., Fausset, C.B., Rogers, W.A., 2011. Understanding the potential for robot as-
sistance for older adults in the home environment. Technical Report. Georgia Institute
of Technology.
Stapleton, S., Adams, M., Atterton, L., 2007. A mobile phone as a memory aid for in-
dividuals with traumatic brain injury: a preliminary investigation. Brain Inj. 21 (4),
401–411.
Starkey, N.J., 2013. Traumatic brain injury in new zealand: a silent epidemic? Psychol.
Aotearoa 5 (1), 18–21.
Stevens, R., 2017. The hidden epidemic of traumatic brain injury.
John Hopkins Brain Science Institure; Brain Talk podcast, online at
http://www.brainscienceinstitute.org/brain_talk/the_hidden_epidemic_of_traumatic_
brain_injury.
Svoboda, E., Richards, B., 2009. Compensating for anterograde amnesia: a new training
method that capitalizes on emerging smartphone technologies. J. Int. Neuropsychol.
Soc. 15 (4), 629–638.
Te Ao, B., Brown, P., Tobias, M., Ameratunga, S., Barker-Collo, S., Theadom, A., McPher-
son, K., Starkey, N., Dowell, A., Jones, K., et al., 2014. Cost of traumatic brain injury in
new zealand evidence from a population-based study. Neurology 83 (18), 1645–1652.
Thompson, C.P., Skowronski, J.J., Larsen, S.F., Betz, A.L., 2013. Autobiographical Mem-
ory: Remembering What and Remembering When. Psychology Press.
Van Erp, J.B., Toet, A., 2015. Social touch in human–computer interaction. Front. Digit.
Hum. 2, 2.
Vemuri, S., Bender, W., 2004. Next-generation personal memory aids. BT Technol. J. 22
(4), 125–138.
Wada, K., Ikeda, Y., Inoue, K., Uehara, R., 2010. Development and preliminary evaluation
of a caregiver’s manual for robot therapy using the therapeutic seal robot Paro. In:
RO-MAN, 2010 IEEE. IEEE, pp. 533–538.
Wagenaar, W.A., 1986. My memory: a study of autobiographical memory over six years.
Cogn. Psychol. 18 (2), 225–252.
Wood, A., 2015. Energy-and Context-aware Approaches to Aiding Human Memory. Ph.D.
thesis. University of Southampton.
Woodberry, E., Browne, G., Hodges, S., Watson, P., Kapur, N., Woodberry, K., 2015.
The use of a wearable camera improves autobiographical memory in patients with
Alzheimer’s disease. Memory 23 (3), 340–349.