International Journal of Human-Computer Studies 142 (2020) 102462

Contents lists available at ScienceDirect

International Journal of Human-Computer Studies

journal homepage: www.elsevier.com/locate/ijhcs

The evaluation of a mobile user interface for people on the autism spectrum: T
An eye movement study
Mortaza Rezaea,b, , Nigel Chene, David McMeekina,b, Tele Tana,c, Aneesh Krishnad, Hoe Leea,e
⁎

a
Cooperative Research Centre for Living with Autism, Australia
b
School of Earth and Planetary Sciences, Curtin University, Perth, Western Australia, Australia
c
School of Civil and Mechanical Engineering, Curtin University, Perth, Western Australia, Australia
d
School of Electrical Engineering, Computing and Mathematical Sciences, Curtin University, Perth, Western Australia, Australia
e
School of Occupational Therapy, Social Work and Speech Pathology, Curtin University, Perth, Western Australia, Australia

ARTICLE INFO ABSTRACT

Keywords: Software solutions are increasingly used to facilitate independence for people on the autism spectrum. However,
User interface design guidelines to make software user interfaces easy to use have not been clearly defined. Here we show,
Mobile app through eye movement analyses, that autistic users interact differently with user interfaces than non-autistic
Autism spectrum users and make recommendations to improve user interface design. We found that autistic users process icons
Eye tracking
and images faster than texts. Using both icons and texts was also effective in facilitating comprehension.
However, we discovered that non-complementary icons and texts confuse users. Our results illustrate that users
on the autism spectrum process user interfaces differently than non-autistic users. We anticipate our findings to
improve user interface design for individuals on the autism spectrum.

1. Introduction
Autism is a diverse group of neurological conditions commonly
characterised by persistent deficits in social communication and social
interaction and restricted, repetitive patterns of behaviour
(American Psychiatric Association, 2013). Individuals on the autism
spectrum are also commonly sensitive to sensory stimuli including
sound, light, and touch (Miles, 2011; Putnam and Chong, 2008). These
characteristics realise significant lifelong challenges. For example, ev-
eryday tasks such as personal care may require individuals such as
caretakers to attend to Ayres et al. (2013). These characteristics also,
more broadly, restrict opportunities for community participation and
independence including education and employment (Hendricks, 2010;
Shattuck et al., 2012). Software applications are increasingly utilised to
facilitate learning and independence in individuals on the autism
spectrum (Chien et al., 2015; De Leo et al., 2011; Hatfield et al., 2017;
Hourcade et al., 2013; Kagohara et al., 2013). However, principles to
make these solutions easy to use have not been evaluated and user
interface (UI) design guidelines, tailored for users on the autism spec-
trum, have not been clearly defined.
Accessible user interface (UI) design has often been overlooked in
the software development process. This setback can be attributed to the
additional cost and effort required to research, inform and evaluate
accessible UI design (Pavlov, 2014). UI is the window that enables users
to communicate with a software application (Foster, 2014). Accessible
UIs allow individuals with widely different abilities to easily operate a
software and reap its benefits (Darejeh and Singh, 2013;
Kamaruzaman et al., 2016). Complex and difficult to use interfaces, on
the contrary, can confuse users causing them to abandon the software
despite its many benefits (Darejeh and Singh, 2013; Pavlov, 2014).
According to the software company Apple, a software is accessible
when every UI element (e.g., buttons, text, images, and navigational
items) can be perceived, understood, and interacted with by all users
(Apple Inc., 2012).
The literature on designing user interfaces through empirical evi-
dence is limited. For example, a review of software accessibility
guidelines for individuals on the autism spectrum reported that none of
the guidelines included in the review is based on empirical findings
(Cristina et al., 2016). Another study reported that existing UI design
guidelines are based on approaches that match autism characteristic to
potential accessibility barrier (Yaneva et al., 2019).
To make web pages more accessible for autistic individuals, the
Academic Autism Spectrum Partnership in Research and Education
(AASPIRE) Web Accessibility Guidelines recommend some design
principles, including:

⁎
Corresponding author at: School of Earth and Planetary Sciences, Kent Street, Curtin University, Bentley, 6102, Western Australia, Australia
E-mail address: Mortaza.rezae@curtin.edu.au (M. Rezae).

https://doi.org/10.1016/j.ijhcs.2020.102462
Received 27 July 2019; Received in revised form 17 April 2020; Accepted 30 April 2020
Available online 12 May 2020
1071-5819/ © 2020 Elsevier Ltd. All rights reserved.
M. Rezae, et al.
• Use simple interfaces to ease understanding
• Complement texts with images and icons
• Provide concrete examples to help with understanding abstraction
and generalisation.
• Minimise scrolling so that users do not assume what information
might be on the page
• Make content as short as possible without sacrificing precision and
specificity
• Be specific and precise in language use
• Provide simple and consistent navigation to increase usability
• Use plain sans-serif font to ease readability
To inform empirical-based guidelines to design mobile application
user interfaces, our goal in this study was threefold:
• First, to evaluate the difference between individuals on the autism
spectrum and non-autistic individuals (control group) in how they
interact with OrienTrip. We do this through eye movement analyses
and measuring parameters such as the number of fixations, total
fixation time, and first fixation latency (the time it took users to
fixate on an area of interest for the first time) when the participants
are using OrienTrip.
• Second, to understand what the individuals on the autism spectrum
and the control group think about the usability of OrienTrip. We do
this by collecting direct feedback through two surveys administered
before and after the eye-tracking exercise.
• Third, to improve mobile application UI design for individuals on
the autism spectrum. We use the findings from the previous steps to
make several evidence-based recommendations to design more ac-
cessible UIs.
Based on findings from the literature, we hypothesise that the au-
tistic group would be drawn to icons and images over texts. We also
hypothesise that the autism group would take longer to process and
understand long textual information compared to the control group.
2. Related research
This section summarises findings from the literature on user inter-
face design for people with varying skills and abilities, including chil-
dren, elderly people, and individuals with disabilities. Furthermore, we
summarise the guidelines from the Web Accessibility Initiative (WAI).
We then briefly discuss past studies that examine how information, in
the general context, should be presented to individuals on the autism
spectrum. Finally, we discuss the benefits of eye-tracking to assess UIs
and how it can be used to improve UI design for individuals on the
autism spectrum.
2.1. User interface design for people with varying abilities
Diverse groups with different abilities, (e.g., elderly, children, and
people with disabilities, including those with visual and cognitive im-
pairments), use software applications differently. For example, it has
been demonstrated that some elderly people have difficulty using UIs
effectively due to their limited experience with computer-related
technologies (Xie, 2003). As such, recommended design principles for
this group include enhanced linguistic support through constant feed-
back and a shorter chain of operations or steps required to complete a
task (Wirtz et al., 2009). Minimising technical terminologies and en-
hancing descriptive texts and instructions, such as FAQs, have also been
shown to improve the user experience for this group (Sayagoa and
Blata, 2010).
Children, on the other hand, require different UI design guidelines.
For example, the following principles have been shown to improve UI
accessibility for children: a) minimal use of text, b) increased use of
visual menus and icons, c) increased use of the mouse, d) larger buttons,
2
International Journal of Human-Computer Studies 142 (2020) 102462
and e) increased integration of interactive elements such as animations
and sounds (Darejeh and Singh, 2013; Grammenos et al., 2000).
Similarly, persons with disabilities require different UI design
guidelines. Audio functionality, for example, must be used to support
users with visual impairments. Colour-blind users, on the other hand,
must be provided with a combination of colours. Customisable UI (e.g.,
customisable fonts, text sizes, and window sizes) should be considered
to aid those with partial sight (Darejeh and Singh, 2013).
Software applications designed for persons with cognitive dis-
abilities are required to be simple with minimal functionalities. The UI
must be easily understood, learned, and contain few elements (e.g.,
buttons and labels) (Darejeh and Singh, 2013; Dawe, 2006).
2.2. The web accessibility initiative
The web accessibility initiative (WAI) is an effort by the World Wide
Web Consortium (W3C) that defines principles to make the web ac-
cessible for people with diverse disabilities. Their recommendations are
(The World Wide Web Consortium, 2018):
Presentation
• Use both text and image to convey a message
• Present headings, lists, table, and other structures in a clear manner
• Foreground and background colours should provide contrast
• Use soft colours
• Use clear fonts
• Use enlarged custom text sizes
Navigation
• All functionality that is available by mouse should be available by
keyboard
• Avoid pop-ups and interruptions
• Avoid implementing flashing content
• Pages should have clear and descriptive titles
• User should know their current location within a set of web pages
Understandable information
• Use simple and easy to understand language
• Navigation items that are repeated on multiple pages should appear
in the same place
• Use descriptive error messages
• Implement “help” for complex functionalities
2.3. User interfaces for individuals on the autism spectrum
UI guidelines for users on the autism spectrum must consider the
characteristic of autism. Autistic individuals have cognitive differences
including executive dysfunction, weak central coherence, and slow
processing speed (Happé et al., 2006; Mintz, 2013; Ozonoff et al.,
1994). A person with executive dysfunction, for example, has difficul-
ties in abilities to initiate a task, identify steps needed to execute a
desired goal, and focus on the activity at hand (Grynszpan et al., 2008;
Mintz, 2013). Similarly, a person with weak central coherence, one of
the theories postulated to be the central cause of autism traits, may
overly focus on the details of a setting thus unable to understand the
general meaning (Booth and Happé, 2010; Mintz, 2013). These cogni-
tive differences can influence how users on the spectrum interact with
software applications.
Some studies have investigated ways information should be pre-
sented, in the general context, to facilitate understanding in individuals
on the autism spectrum. For example, visual cues have been demon-
strated to be effective in facilitating communication and learning in
persons on the autism spectrum (Grynszpan et al., 2008;
Kamaruzaman et al., 2016; Pavlov, 2014; Yaneva et al., 2015). Rich
M. Rezae, et al.
multimedia interfaces, however, should be avoided as it causes further
challenges in usability (Grynszpan et al., 2008). To support reading
comprehension, sentences should be as short as possible, expressed in
easy or common language (Kamaruzaman et al., 2016). This is because
individuals on the autism spectrum may have difficulty understanding
long sentences, figurative language, or abstract words (Yaneva et al.,
2015). Concentration and focus can be facilitated by avoiding bright
colours and pop-up distractions (Hussain et al., 2016; Pavlov, 2014).
Finally, user interfaces that contain few elements are much easier to use
and appreciated by these users (Pavlov, 2014).
2.4. Eye tracking
Eye tracking is a technique used to measure a person's eye move-
ments while interacting with a stimulus (Mat Zain et al., 2011;
Sun et al., 2016). This allows researchers to know where a person is
looking at a given time and the sequence in which their eyes move from
one point to another (Mat Zain et al., 2011). As such, eye movement
measurements provide implicit data about the individual's behaviour
and how they process given stimuli (Granka et al., 2004).
Although, eye trackers are regarded as useful tools, they present
some challenges for researchers. First professional eye trackers are ex-
pensive which can make them inaccessible to low-budget laboratories
(Hauser et al., 2018). Second, eye tracking methodology lacks con-
sistent terminology and standardisation. To demonstrate this,
Hauser et al. (2018) reports that “eye tracking” is written in three
different ways in three different studies: “eye-tracking”, “eye tracking”,
and “eyetracking”. This lack of consistency and standardisation is also
true for the metrics and methods used in eye tracking, which can make
it difficult for researchers to plan and replicate studies. Finally, it has
been shown that analysing eye tracking data can be time-consuming
and can take months to complete (Hauser et al., 2018; Nivala et al.,
2016).
Despite the limitations of eye tracking, it is commonly used in
human-computer interaction (HCI) research to evaluate the usability of
software user interfaces and inform design decisions to improve ease of
use. Attributes such as fixation count and fixation time are commonly
measured in eye tracking (Cowen et al., 2002; Joachims et al., 2005).
These measurements can be used to evaluate and improve factors such
as visibility, meaningfulness, and placement of user interface elements
(Poole and Ball, 2006). Fixations, for example, are stable gazes that
represent information acquisition and processing (Cutrell and
Guan, 2007; Pan et al., 2004; Sun et al., 2016). A long fixation on a
button, in a scenario where the user is asked to find a button, can in-
dicate that the button is confusing and should be considered for a re-
design.
Eye tracking is also increasingly used in autism research. This can be
because; a) improved accessibility to unobtrusive eye tracking tech-
nologies (Guillon et al., 2014), and b) eye tracking does not require
advanced communication skills, commonly limited in individuals on
the autism spectrum (Falck-Ytter et al., 2013).
Some studies have used eye movement analyses to evaluate how
individuals on the autism spectrum view web pages. These studies have
shown that individuals on the autism spectrum interact with web page
user interfaces differently than non-autistic individuals. For example,
Yaneva et al. (2018) showed that autistic users take longer to complete
tasks on a website than non-autistic individuals (Yaneva et al., 2018).
The authors suggested that individuals on the autism spectrum can be
less efficient in the way they interact with web pages. Furthermore,
they reported that autistic individuals look at more elements on web
pages and make more transitions between user interface elements in-
dicating higher cognitive effort in processing the web page
(Yaneva et al., 2018).
Another study used eye movement analyses to assess autistic in-
dividuals’ ability to search for information on web pages (Eraslan et al.,
2017). The authors reported that participants on the autism spectrum
3
International Journal of Human-Computer Studies 142 (2020) 102462
are less successful in locating the required information on web pages
compared to non-autistic individuals. Furthermore, the authors ex-
pressed that autistic individuals are easily distracted by irrelevant ele-
ments on web pages. Interestingly, significant variance between the
participants on the autism spectrum was found which means that every
autistic individual interacts differently with web pages.
Finally, another study to evaluate the performance of people on the
autism spectrum in web search tasks under time constraints found that
autistic individuals are less successful in finding the correct information
(Eraslan et al., 2019). The authors reported that autistic individuals
have significantly longer eye gaze scan paths than the non-autistic
group indicating that they focus more on non-relevant elements
(Eraslan et al., 2019). Similarly, the authors discussed that autistic in-
dividuals make significantly more transitions between elements com-
pared to the non-autistic group. According to the eye tracking litera-
ture, more transitions between elements can indicate confusion or
uncertainty about a decision (Ehmke and Wilson, 2007).
3. Methods
3.1. Overview of OrienTrip
OrienTrip is a public transport trip planning mobile application
built for individuals on the autism spectrum. It has been co-designed by
people on the autism spectrum to make public transport experiences
less stressful. Public transport presents many challenges and obstacles
for autistic travellers. For example, more than 50% of adults on the
autism spectrum have difficulty planning public transport trips
(Deka et al., 2016) and more than 40% are worried about social in-
teractions and the friendliness of other passengers (Deka et al., 2016).
The ability to travel freely and independently is directly correlated
with improved quality of life including increased employment, educa-
tion, and social outcomes (Angell and Solomon, 2018; Feeley, 2010;
Lindsay, 2017; Mechling and O'Brien, 2010). In fact, a one-point im-
provement in independence can result in a 10% increase in the like-
lihood of securing paid employment (Chan et al., 2017). Concretely,
individuals on the autism spectrum who can travel independently are
five times more likely to be employed than their peers who rely on
others for transportation (Zalewska et al., 2016).
OrienTrip, in its current version (v1.0), provides the following
functionalities:
• Users can save favourite trips
• Favourite trips can be configured to remind users what to pack for
the trip
• Users can check the crowdedness of services before a trip
• Users can check their location, updated in real-time, in reference to
the stops in the journey. This facilitates awareness to inform when
the user needs to disembark.
• Users can designate a person from their contact list as their primary
caretaker
• Users can call their primary caretaker, when required, without
leaving the app (particularly useful in high anxiety situations)
• Users can share their location with their primary caretaker
• Users can review evidence-based anxiety management and sensory
overload strategies, specially tailored for public transport use
• Users can communicate with other people through a customisable
virtual card (designed for those with speech impairments)
3.2. Participants
Overall, 39 participants were recruited through internal email lists
and the Curtin Specialist Mentoring Program, a peer support mentoring
program specifically designed for students on the autism spectrum at
Curtin University.
The autism group consisted of twenty-one participants (17 male
M. Rezae, et al.
participants and four female participants) on the autism spectrum
(median age = 19.5; mean age = 19.9; standard deviation = 3.24).
Most of the autistic participants were university students studying an
undergraduate course. Two autistic participants were graduating high
school students. All participants were verbal and were able to com-
municate clearly with the study coordinator. Autism diagnosis for all
participants was self-reported. Due to calibration issues with the eye
tracker on one participant, complete eye movement data were available
from twenty autistic participants.
To assess the level of technological proficiency in the autistic par-
ticipants, we sought to understand how often they utilise common
technologies including social media and online shopping. All autistic
participants reported that they own a mobile phone. Most participants
on the autism spectrum reported that they use social media through a
browser or a mobile application at least once a week. Twelve in-
dividuals on the autism spectrum reported that they have shopped
online at least once in the past year. While nine individuals reported
that have do not shop online.
Similarly, we sought to understand the experiences of the partici-
pants with public transport and public transport applications. 15 in-
dividuals on the autism spectrum reported that they use public trans-
port at least once a week. Furthermore, only seven of these individuals
expressed that they use a public transport mobile application to plan
their trips. This suggests that most autistic participants were familiar
with the public transport systems and public transport applications.
The control group consisted of eighteen (seven male and 11 female)
non-autistic participants (median age = 22.5; mean age = 24.2;
standard deviation = 6.34). Most control group participants were
university students studying an undergraduate course, in addition to
one professional, and 5 postgraduate students.
Similarly, we sought to understand the control group's proficiency
with common technologies including online shopping and social media.
All participants reported that they own a mobile phone. 16 participants
expressed that they shop online at least once a year. All 18 participants
reported that they use social media at least once a week. Furthermore,
13 control group participants reported that they use public transport at
least once a week, while only seven of these participants expressed that
they use a mobile application to plan their journeys.
Participants in both groups were given a $20 gift card as a token of
our appreciation for their time. A two-group comparison was employed
so that the control group can be used as a baseline on which the re-
action of the autism group can be measured.
3.3. Ethics
Ethical approval was obtained from the Human Research Ethics
Committee (HREC) at Curtin University (HRE2016-0086) in Perth,
Western Australia.
Participants were provided with an information sheet describing
their role in the research. They were informed that they could withdraw
at any time without any negative consequences. Participants then
provided digital consent confirming their participation. All study data
were confidentially stored and maintained in line with the Western
Australian University Sector Disposal Authority.
3.4. Experimental hardware
The experimental task was delivered using SMI Experiment Centre
v3.0 and presented on a widescreen 24-inch monitor. Participants’ eye
movements were recorded with an SMI Red eye tracker at 120 Hz. Five
points of calibration were used.
3.5. Experimental task
The study consisted of eight tasks and 25 screens with varying visual
complexity (Appendix A shows the screens of the tasks). These eight
4
International Journal of Human-Computer Studies 142 (2020) 102462
tasks were instructions to work with the application in a specific way.
For example, task two was “Find the sensory overload strategies for
bright lights”. This required participants to click on the “Assistance”
button followed by the “Sensory overload” and “Bright Lights” options.
There were two reasons for using these eight tasks. First, to expose
the participants to the core functionalities of OrienTrip. These func-
tionalities are the ones we would expect users to use if they used the
application in the real world. Second, to introduce the participants to
OrienTrip's core UI elements and assess how they would interact with
these elements to complete the given tasks. Performance measures such
as the number of fixations, the time it took to have the first fixation, the
time it took to click on a UI element after the first fixation, and the total
fixation time, were collected.
Users were required to interact with one or more screens of
OrienTrip to complete a task. For data analysis, each screen consisted of
one or more Areas of Interest (AOI). The AOIs on each screen were
methodically selected. Specifically, elements that users had to interact
with or look at (i.e., button, input field, label, etc.) to obtain some in-
formation or to complete the task were selected as target AOIs.
The tasks were as follows:
1 Set “Mary Smith” as your caretaker in the app
2 Before you start planning your trip, you want to explore the sensory
overload tips, so you can be better prepared for the trip. Find the tip
for “bright lights”.
3 Go back to the home screen and plan a trip from Current Location to
Museum Street, departing at 3:00pm
4 Select the route with the least number of changes and start trip.
a How many routes have been suggested to you?
b How many changes does each route have?
c What is the crowdedness of each route?
d Find the route with the least number of interchanges
i What time does this route depart?
ii Can you describe this route?
e Select this route and start the trip
5 How many stops are there in your trip including the last stop.
6 Now imagine you're on the bus and the bus is moving, but you're
starting to feel a little anxious about missing your stop. Find the
anxiety management tip on “Automatic negative thoughts”.
7 Find your current location in the journey map
8 Imagine, you have arrived at your destination. Share your location
with your caretaker.
3.6. Procedure
The participants were asked to sit at a computer desk. The eye
tracker was placed at the base of the participant's monitor on the desk.
A keyboard and mouse were also placed on the desk for the participants
to use during the experiment. The experiment coordinator commu-
nicated instructions and made observations from another desk, per-
pendicular to the participant (Fig. 1a and b).
First, the purpose of the study and the procedure was described to
the participants. Furthermore, the participants were given a detailed
account of the idea and functionalities of OrienTrip. The participants
were encouraged to ask questions until they understood what was in-
volved. However, no screens of OrienTrip was shown to the participants
at this stage. Finally, they were requested to complete a participation
consent form, or an assent form and a parent consent form if they were
under 18 years old.
Second, the participants were asked to complete a pre-study ques-
tionnaire. The purpose of this survey was to form an idea of the par-
ticipants’ background and experiences with common technologies.
Upon completing the pre-study questionnaire, participants were
seated approximately 80 cm from the display monitor. Following cali-
bration, participants were informed that the application will appear on
the screen and they were requested to complete the instructed tasks.
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

thanked.

3.7. Data analysis

Analyses were performed using IBM SPSS Statistics 25, with the
significance level set at p < 0.05. Participants’ eye gaze data were
entered for analyses of variance (ANOVA) to test for main effects and
interaction effects of the dependent variables. For all ANOVAs con-
ducted, F statistics have been derived using a multivariate approach,
Pillai's trace, as this approach is generally considered to be robust
against potential departures from assumptions. Paired t-tests were
performed to follow up any significant main effects or interaction ef-
fects, with Bonferroni corrections applied to the significance level,
where necessary.
Participants’ post-study data, consisting of rankings based on a 5-
point Likert scale, was entered for Mann-Whitney U Test to test for
differences in sample means.

4. Results

4.1. Eye gaze data

For brevity, this section only discusses findings that were found to
be significant (p < 0.05). For an overview of the results, see Table 1.
The measured dependent variables are defined as follows by the
BeGaze Manual (v.3.7):

• Number of fixations is the count of all fixations by a participant on

an AOI
• First fixation latency is how long it took a participant to fixate on an
AOI for the first time
• First fixation to mouse click is how long it took for a participant to
click on an AOI after first fixation on the said AOI.
• Total fixation time is the sum of fixation duration on an AOI.
Fig. 1. (a). Visual overview of the experiment setup. (b) - Experiment setup. 4.1.1. Task 1
Task One was “Set ‘Mary Smith’ as your caretaker in the app”
A test of repeated measures ANOVA was performed to examine
The participants were requested to look straight the monitor in front of
whether the number of fixations, across pages and between the two
them at all times to reduce the impact of head movements. The tasks
groups, differed. A statistically significant difference in the number of
were read out by the experiment coordinator in a clear and easy to
fixations was evident across pages of task one, F(3,30)=12.16, p <
understand manner. The task descriptions had a very clear structure
0.001. There was no overall group differences F(1,32)=3.977, p=.055,
and avoided the use of metaphors or indirect references. If the parti-
however, a statistically significant interaction effect, pages by group,
cipant had any questions, they were asked to communicate their
was found, F(3.0,30)=4.005, p=.016. Pairwise comparisons revealed
questions while looking at the screen in front of them to minimize the
that the mean difference in the number of fixations between the autistic
impact of head movements.
group (I) and the control group (J) was statistically significant for page
OrienTrip's screens were delivered to the wide 24” screen as
one, mean difference (I-J) = 2.326 and p = 0.01, and page two, mean
screenshots due to eye tracking limitations on mobile devices. First, the
difference (I-J) = 2.049 and p=.028. The autistic group had a higher
task was communicated to the participant. After the participant clearly
mean number of fixations. This trend was, however, not evident for
understood the task, the experiment coordinator manually triggered,
page three, p=.582, and page four, p=.068 as this can be seen in Figs. 2
from a different computer, the first screen of OrienTrip required for the
and 3.
task, to be displayed on the participants’ monitor (Fig. 1). Subsequent
Similarly, a test of repeated measure ANOVA revealed statistically
screens of the task were triggered and displayed on the participants’
significant differences in the first fixation latency across the pages of
monitor when the participant was able to correctly identify and click
task one, F(2,30)=11.916, p < 0.001. No interaction effect, pages by
the required UI element (e.g., button, input field, label, etc.). The idea
group, was observed. Similarly, no group differences were evident in
of this approach was to manually simulate the flow of the application
the first fixation latency, p=.852. See Fig. 4.
through screenshots on a wider screen.
To assess whether there were differences, across the pages and be-
During the experiment, no assistance was provided to the partici-
tween the two groups, in the time it took to click on an AOI from the
pants except for clarification of the tasks. All participants completed the
first fixation, a repeated measures ANOVA was conducted. A statisti-
experiment in one sitting. Participants were informed that they can
cally significant difference was found across pages of task one, F(3,27)
complete the experiment in multiple sessions and take breaks between
=16.155, p < 0.001. No interaction effect was observed, F(3,27)
sessions; however, no one took this up. Each sitting, on average, took
=2.471, p=.083. No group differences in the mouse click from the first
about 30 minutes from start to finish.
fixation time, p=.833, were observed.
After the study, participants were requested to complete a post-
study questionnaire to provide direct feedback about the UI and the
4.1.2. Task 4a
user experience (UX) of the app. Finally, they were debriefed and
Task 4a was to find “how many routes have been suggested to you”.

5
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Table 1
Overview of eye tracking data analyses
Task Dependent Variable 1 Dependent Variable 2 Dependent Variable 3

One Number of fixations First fixation latency First fixation to mouse click
Pages Pages * group Pages Pages * group Pages Pages * group
F(3,30)=12.16, p=0.000 F(3,30)=4.005, p=.016 F(2,30)=11.916, F(2,30)=1.333, F(3,27)=16.155, p= F(3,27)=2.471, p=.083
p=0.000 p=0.082 0.000
Two Number of fixations First fixation latency First fixation to mouse click
Pages Pages * group Pages Pages * group Pages Pages * group
F(3,29)=7.611, p=.001 F(3,29)=.204, p=.893 F(1,31)=25.964, p=.000 F(1,31)=1.521, p=.227 F(3,23)=15.763, p=.000 F(3,23)=.333, p=.801
Three AOI: ‘Current location marker’
num_fixations * group first_fixation_to_mouse_click * group total_fixation_time * group
F(1,34)=.343, p=.562 F(1,26)=3.468, p=.074 F(1,32)=1.763, p=.194
Three AOI: ‘From field’
num_fixations * group first_fixation_to_mouse_click * group total_fixation_time * group
F(1,34)=.919, p=.345 F(1,26)=3.468, p=.074 F(1,34)=1.984, p=.168
Three AOI: ‘To Field
num_fixations * group first_fixation_to_mouse_click * group total_fixation_time * group
F(1,33)=8.954, p=.005 F(1,34)=3.291, p=.078 F(1,33)=4.887, p=.034
Three AOI: ‘Search’
num_fixations * group first_fixation_to_mouse_click * group total_fixation_time * group
F(1,30)=.632, p=.433 F(1,29)=2.346, p=.136 F(1,30)=.863, p=.360
Four (a) Number of fixations Total fixation time
Routes Routes * group Routes Routes * group
F(2,31)=17.651, p=.000 F(2,31)=.051, p=.951 F(2,32)=19.163, p=.000 F(2,32)=.991, p=.382
Four (b) Number of fixations Total fixation time
Routes Routes * group Routes Routes * group
F(2,30)=3,486, p=.044 F(2,30)=.162, p=.851 F(2,30)=4.442, p=.020 F(2,30)=7.074, p=.003
Four (c) Number of fixations Total fixation time
Routes Routes * group Routes Routes * group
F(2,31)=2,269, p=.120 F(2,31)=1.624, p=.213 F(2,31)=2.556, p=.094 F(2,31)=2.579, p=.092
Four (d) AOI: ‘Time’
num_fixations * group total_fixation_time * group
F(1,33)=9.089, p=.005 F(1,33)=4.788, p=.036
Four (d) AOI: ‘Description’
num_fixations * group total_fixation_time * group
F(1,33)=4.085, p=.051 F(1,33)=3.998, p=.054
Four (e) num_fixations * group first_fixation_to_mouse_click * group total_fixation * group
F(1,28)=.783, p=.384 F(1,26)=.287, p=.579 F(1,28)=2.041, p=.164
Five Number of fixations Total fixation time
Stops Stops * group Stops Stops * group
F(6,28)=12.095, p=.000 F(6,28)=1.039, p=.422 F(6,29)=9.923, p=.000 F(6,29)=.792, p=.583
Six Number of fixations First fixation latency First fixation to mouse click
Pages Pages * group Pages Pages * group Pages Pages * group
F(2,30)=3.364, p=.048 F(2,30)=4.603, p=.018 F(1,34)=11.976, p=.001 F(1,34)=.177, p=.677 F(2,25)=32.997, p=.000 F(2,25)=.301, p=.743
Seven num_fixations * group first_fixation_latency * group total_fixation_time * group
F(1,31)=.064, p=.802 F(1,31)=.819, p=.372 F(1,31)=,905, p=.349
Eight Number of fixations First fixation to mouse click
Pages Pages * group Pages Pages * group
F(1,30)=.167, p=.686 F(1,30)=.094, p=.761 F(1,30)=50.078, p=.000 F(1,30)=4.190,
p=0.049

These public transport routes were generated based on the actions from
the previous tasks.
A repeated measures ANOVA was conducted to analyse the differ-
ences in the number of fixations for task four(a). A statistically sig-
nificant difference was detected across the three presented routes, F
(2,31)=17.651, p < 0.001. No route by group interaction effect was
found, F(2,31)=.051, p=.951. No group differences were observed,
p=.305. See Fig. 5.
Similarly, to assess the differences in the total fixation time across
the routes, a repeated measure ANOVA was performed. Pillai's Trace
found a statistically significant difference across the three routes, F
(2,32)=19.163, p < 0.001. No route by group interaction was evident,
F(2,32)=.991, p=.382. Similarly, no group differences were observed,
p=.338. See Fig. 6.
4.1.3. Task 4b
Task 4b was to find “how many changes does each route have”.
To assess the difference in the number of fixations on the “changes”
AOI, repeated measures ANOVA found statistically significant
differences across the three routes, F(2,30)=3.486, p=.044. No route
by group interaction was found, F(2,30)=.162, p=.851. Similarly, no
group differences were observed, p=.910. See Fig. 7.
A test of repeated measure ANOVA detected a statistically sig-
nificant difference in total fixation time across the routes, F(2,30)
=4.442, p=.020. Interestingly, Pillai's Trace found a statistically sig-
nificant route by group interaction effect, F(2,30)=7.074, p=.003.
Pairwise comparison, however, found no differences between the
groups in the three pages. Test of between-subjects’ effects also ob-
served no group differences, p=.742 (Fig. 8).
4.1.4. Task 4d
Task 4d was to “find the route with the least number of inter-
changes” and describe “what time does this route start” and “describe
this route”.
One-way ANOVA observed statistically significant group differences
for the number of fixations on the “time” AOI, F(1,33)=9.089, p=.005.
The autism group had a higher number of fixations. Similarly, there
were statistically significant group differences in the total fixation time

6
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Fig. 2. Autism group on average showed greater number of fixations in task one Fig. 4. Autism group showed higher number of fixations compared to the
(Set “Mary Smith” as your caretaker in the app) compared to the control group. control group for routes of task four(a) (How many routes has been suggested to
you?).

Fig. 3. Autism group on average showed higher first fixation latency (in msec)
for pages of task one compared to the control group. Fig. 5. Autism group showed higher total fixation time (in msec) compared to
the control group for routes of task four(a).

4.2. Questionnaire data

on the aforementioned AOI, F(1,33)=4.788, p=.036. See Figs. 9 and
10. Similarly, the autism group had a higher total fixation time on the
The analyses presented in this section describe the findings from the
“time” AOI.
questionnaires conducted before the eye tracking study and after the
In contrast, one-way ANOVA found no statistically significant group
eye tracking study.
difference for the number of fixation on the “route description” AOI, F
To understand if the groups used public transport, Mann-Whitney U
(1,33)=4.085, p=.051, and total fixation time, F(1,33)=3.998,
test was performed. Mean ranks in the autistic and control group were
p=.054. See Figs. 11 and 12.
19.25 and 19.78; no significant differences were evident in the two
groups (Mann-Whitney U = 185.0, n (autistic group) = 20, n (control
group) = 18, p=.897 two-sided). See Fig. 13.

7
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Fig. 6. Number of fixations were on average the same for both the autism group
Fig. 8. Autism group had higher number of fixations on the “time” AOI com-
and the control group for task four(b) (How many changes does each route
pared to the control group for task four(d) (Find the route with the least number
have?).
of interchanges, (i) what time does this route depart? (ii) can you describe this
route?).

Fig. 7. Interestingly, control group showed on average higher total fixation

time (in msec) compared to the autism group for routes of task four(b).
Fig. 9. Autism group had higher total fixation time on the “time” AOI compared
to the control group for task four(d)
To learn how often the two groups used public transport, Mann-
Whitney U test found that the mean ranks in the autistic and control
control group = 17.42; no significant difference was observed in the
groups were 16.53 and 12.15; no significant differences were observed
groups (Mann-Whitney U = 142.5, n (autistic group) = 21, n(control
in the two groups (Mann-Whitney U = 67.0, n (autistic group) = 15, n
group) = 18, p =.276 two-sided). The higher mean rank in the autistic
(control group) = 13, p = .170 two-sided). The higher mean rank in
group suggests that they disagreed more with the statement (Fig. 15).
the autistic groups suggests that the autistic group used public transport
Similarly, to find if the participants found the flow of the app logical
more frequently than the non-autistic group. See Fig. 14.
(i.e., if the steps required to complete a task made sense), Mann-
To understand if the participants thought the text and icons in the
Whitney U test was performed. The mean ranks in the autistic and
app matched, Mann-Whitney U test was performed on the post-study
control group were 21.75 and 17.0; no significant difference was evi-
questionnaire. The mean ranks in the groups were autism = 21.38 and
dent in the groups (Mann-Whitney U = 135.0, n(autistic) = 20, n

8
M. Rezae, et al.
Fig. 10. Control group had higher number of fixations on the “description” AOI
compared to the autism group for task four(d).
Fig. 11. Control group had higher total fixation time (in msec) on the “de-
scription” AOI compared to the autism group for task four(d).
(control group)=18, and p = .196 two-sided). The higher mean rank in
the autistic group suggests that their view differed from the statement
(Fig. 16).
5. Discussion and conclusion
In this paper, we have shown that users on the autism spectrum
process information differently than non-autistic individuals. Previous
studies have demonstrated that multimedia interfaces facilitate learning
and improve comprehension in people on the autism spectrum
(Grynszpan et al., 2008; Kamaruzaman et al., 2016; Pavlov, 2014;
9
International Journal of Human-Computer Studies 142 (2020) 102462
Fig. 12. Autism group and the control group had similar mean rank of whether
they used public transport.
Fig. 13. Autism group used public transport more frequently in a week than the
control group.
Yaneva et al., 2015). Our study also found that participants on the
autism spectrum favour icons more than long texts. We also observed
that ambiguous or uncommon icons (icons that are not commonly used
in mobile apps) can confuse users. Similarly, this study validated that
long texts are often ignored by autistic users. Furthermore, when asked
to extract information from long texts, the participants on the autism
spectrum required more efforts compared to the control group.
5.1. Main results
Overall, we found differences in whether the two groups used public
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Fig. 14. The autism group disagreed more with the statement, “the texts and
icons matched correctly”.

transport. Analyses showed that participants on the autism spectrum

used public transport more frequently than their non-autistic counter-
parts. This suggested that the two groups were both familiar with public
transport services and knew what is expected from a trip planning ap-
plication.
In task one, which consists of four screens, users were asked to
“configure your caretaker” (Figs. A.1, A.2, A.3, and A.4). Red boxes
highlight the Area of Interest (AOI). The task required them to select a
person from the contact list as their primary carer. This action will then
activate other functionalities of OrienTrip, namely the ability to call the Fig. 16. Heat map of the control group in four(a) (how many routes has been
suggested to you?) showing that the fixation time on subsequent routes de-
designated person or share their location with them without leaving the
creased after they found the first route.
app.

Fig. 15. The autism group disagreed more with the statement, “the flow of the app was logical and consistent”.

10
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

The autism group showed a greater number of fixations on AOIs of differences were observed, in-depth analyses found that the autism
screen one and screen two for task one. The AOI on the screen one was a group had greater and longer fixations on each route than the control
“menu” icon and the AOI on the screen two a “bell” icon coupled with group. According to the eye tracking literature, this suggests difficulty
the text “Set up Carer”. According to eye tracking literature, repeated in extracting relevant information (Ehmke and Wilson, 2007). Inter-
fixations signify lack of meaningfulness or visibility (Ehmke and estingly, the number of fixations and fixation time decreased with each
Wilson, 2007). Specifically, this can suggest that the autism group took route, in both groups, as evident in the heat map in Figs. 16 and 17. The
longer to understand the highlighted AOIs compared to the control differences between the two groups, however, persisted. The autistic
group. It can also be argued that the higher number of fixations re- group, also, focused more on irrelevant elements than the control
present an interest in the AOI. However, the first fixation to mouse click group. Previous studies have also shown that individuals on the autism
(FFMC) data showed that the autism group also took longer to click on spectrum focus more on irrelevant elements on a webpage than non-
the AOI after the first fixation compared to the control group. This autistic individuals (Cristina et al., 2016; Eraslan et al., 2017;
finding supports the conclusion that the autism group took longer to Yaneva et al., 2019).
understand the highlighted AOIs. In subsequent tasks, task four(b) and task four(c), where partici-
To further understand how the icons and texts were perceived, pants were asked to identify specific information about each of the
participants were asked to rank the statement, “I thought the text and three routes (e.g., crowdedness status of services and the number of
illustration/icons matched”, on a 5-point Likert scale (1=strongly changes required), no differences were observed between the two
agree, 5=strongly disagree). In-depth analyses found that the autistic groups. In fact, in task four(b), the number of fixations and total fixa-
group disagreed more with the statement than the control group. tion time between the two groups were almost the same.
Comments from the autistic group included, “the texts that were asso- The greater number of fixations and longer fixation time in task four
ciated with an illustration should be separated a bit”, “[icons] possess (a), exhibited by the autism group, may indicate that they took longer
different styles”, and “the setting button was a bit hidden”. This can to process the UI and find the required information (Ehmke and
additionally suggest that the autistic group had difficulty perceiving the Wilson, 2007). Once understood, however, they were able to perform at
intended meaning of the AOIs or the UI elements on screens one and the same level as the control group as evident by the number of fixa-
two. tions and the total fixation time from tasks four(b) and four(c). It is
One explanation for the confusion on the screen one can be that the
“menu” icon was not the standard menu icon commonly used in mobile
apps. For example, the menu button is very often represented by three
parallel bars; whereas, our icon slightly deviated from this design.
Previous studies on UI design for autistic individuals have suggested
that icons should be easily recognisable (Hussain et al., 2016;
Raymaker et al., 2019). Another reason for the confusion can be that a
similar menu option, “Assistance”, was displayed on the same row as
the target menu button. This may have confused the users in choosing
the correct option.
The discrepancy in the number of fixations on screen two was
backed by another dependent variable: first fixation to mouse click
(FFMC). The autism group took longer to click on the AOI after the first
fixation compared to the control group. This can again suggest that the
participants on the spectrum took longer to conclude if the AOI was the
correct option (Ehmke and Wilson, 2007). One possible explanation can
be that the icon did not strongly resemble the meaning of the text
(Groba et al., 2018; Kamaruzaman et al., 2016; Pavlov, 2014). For
example, the text read “configure caretaker” while the icon showed a
bell that commonly resembles alarm or notification. A “gear” icon,
frequently, associated with “settings”, can be more suitable for this
purpose. These minute details are important because individuals on the
autism spectrum have difficulty with abstraction and generalisation
(Yaneva et al., 2015).
When asked to rank the statement “the flow of the app was logical
and made sense” (i.e., if the steps taken to complete a task were clear
and expected), participants on the autism spectrum disagreed more
with the statement than non-autistic individuals. Comments by the
autistic group revealed that they, at first glance, were not able to dif-
ferentiate between the “Assistance” button and the “menu” icon. This
supports our earlier eye movement analyses finding. One user com-
mented that they found it confusing that the “configure your carer”
option was located under a different menu than the “contact carer” and
“share location” options. Several suggestions were made to combine the
menu and the assistance buttons into one button. Previous studies have
also suggested a similar UI design guideline: related functionalities
should be presented on similar screens (Kamaruzaman et al., 2016;
Pavlov, 2014; Raymaker et al., 2019; Yaneva et al., 2018).
In task four(a), participants were asked to identify how many public Fig. 17. Heat map of the autism group in task four(a) (how many routes has
transport routes were shown on the screen. This screen was the result of been suggested to you?) showing that the fixation time on subsequent routes
previous tasks where participants planned a trip from their current lo- decreased after they found the first route. However, they looked at more irre-
cation to “Murray street, Perth”. Although no significant statistical levant elements than the control group.

11
M. Rezae, et al.
interesting to note that the AOIs in task four(a) consisted of many texts
and icons. While the AOIs in tasks four(b) and four(c) were only high-
contrast icons. This insight can indicate what previous studies have
shown: visual stimuli are more effective in facilitating comprehension
in individuals on the autism spectrum (Cristina et al., 2016;
Grynszpan et al., 2008; Kamaruzaman et al., 2016; Yaneva et al., 2019,
2018). Furthermore, bright colours have also been reported to quickly
attract attention (Pavlov, 2014).
Another interesting pattern was observed in task four(d). The pur-
pose of this task, composed of two AOIs, was to assess whether parti-
cipants could identify detailed information about the fastest route
(Fig. A.14). The “time” AOI, consisting of just text, showed the overall
departure and arrival times. The “description” AOI, consisting of both
text and icon, showed detailed transit information about the route (e.g.,
modes of transportation, service numbers, and departure times for each
mode).
The autism group exhibited a significantly greater number of fixa-
tions and total fixation time on the “time” AOI, suggesting lack of
meaningfulness in AOI and difficulty in understanding the information
(Ehmke and Wilson, 2007; Yaneva et al., 2019). Interestingly, this trend
changed for the second AOI, “description”. In-depth analyses revealed
that the autism group had a lower number of fixations and total fixation
Fig. 18. Heat map of control group for task four(d) (find the route with the least
number of interchanges) shows they spent less time on the ‘time’ AOI and
longer on “description” AOI reading all texts
12
International Journal of Human-Computer Studies 142 (2020) 102462
time than the control group. Heat map of total fixation time (Fig. 19),
reveals that the autism group spent longer fixating on the “time”. This
may be because: a) the text was small and difficult to read (Pavlov et al.,
2018), and b) there were no icons for them to rely on to infer the
meaning of the text (Sofian et al., 2018; Yaneva et al., 2015). In con-
trast, the autism group only took a brief look at “description” AOI
which consisted of many icons that are thought to ease understanding
(Eraslan et al., 2017; Raymaker et al., 2019; Yaneva et al., 2015). The
reverse was exhibited by the control group as evident in Fig. 18. They
fixated longer on the “description” AOI showing that they read all of the
small texts and information despite the icons.
5.2. Recommendations
The findings of this study justify recommendations for researchers
and software developers to design more accessible user interfaces for
individuals on the autism spectrum. These recommendations are pri-
marily targeted at public transport applications.
Recommendations are:
• Visual stimuli, icons or images, should accurately describe the given
context (Cristina et al., 2016; Groba et al., 2018;
Fig. 19. Heat map of autism group for task four(d) (find the route with the least
number of interchanges) shows that they fixated longer on the “time” AOI and
significantly less on the “description” AOI
M. Rezae, et al.
Kamaruzaman et al., 2016; Pavlov, 2014; Pavlov et al., 2018;
Raymaker et al., 2019; Sofian et al., 2018; Yaneva et al., 2019,
2015).
• Common icons, frequently seen in other mobile applications, should
be favoured over new designs to reduce uncertainty
(Kamaruzaman et al., 2016; Raymaker et al., 2019; Sofian et al.,
2018).
• Icons and texts, when used together, should communicate the same
message. In other words, icons should be an accurate interpretation
of the referent text and vice versa (Raymaker et al., 2019;
Yaneva et al., 2015).
• Texts should be brief and concise (Cristina et al., 2016;
Hussain et al., 2016; Pavlov, 2014; Raymaker et al., 2019;
Sofian et al., 2018; Yaneva et al., 2015).
• Screens should not be cluttered. Each screen should display as few
elements (e.g., buttons, icons, texts) as possible (Cristina et al.,
2016; Eraslan et al., 2019, 2017; Fletcher-Watson et al., 2016;
Hussain et al., 2016; Kamaruzaman et al., 2016; Pavlov et al., 2018;
Raymaker et al., 2019; Yaneva et al., 2018).
• UI design should be consistent (Cristina et al., 2016;
Kamaruzaman et al., 2016; Pavlov et al., 2018; Raymaker et al.,
2019; Sofian et al., 2018). For example, similar functionalities
should be grouped under one menu. The same terminology should
be used throughout.
Given the present findings that demonstrate that users on the autism
spectrum process user interfaces differently compared to non-autistic
users, future studies may benefit from similar empirical evaluations.
Researchers can use eye movement analyses to evaluate the UI of other
applications to improve their usability and use the findings to add to
existing UI design guidelines for users on the autism spectrum.
5.3. Limitations
A limitation related to the sample size of the autism group could be
that the participants may not be representative of the whole population.
This is because every individual on the autism spectrum is different,
therefore each may respond to stimuli differently. In addition, autism
diagnoses in the autistic group were entirely self-reported. It is unlikely
that the participants included people who did not fulfil the autism
spectrum diagnosis criteria; however, it cannot be ruled out. Similarly,
a limitation of the sample size of the control group could be that they
may not be representative of the whole population. Furthermore, con-
trols were not assessed for an autism diagnosis. Therefore, the occur-
rence of control participants with autism diagnosis which were not
reported cannot be accounted for. Another limitation of the study
concerns the discrepancy between the autism and control participants.
The autism group had a younger mean age and a larger proportion of
male participants relative to the control group. As such this remains a
limitation.
13
International Journal of Human-Computer Studies 142 (2020) 102462
Another possible limitation of this study could be the way OrienTrip
was displayed to the participants. The screenshots of OrienTrip was
displayed on a 24-inch widescreen monitor and measured the partici-
pants’ eye movements. This meant that the UI elements appeared larger
than they would on a mobile screen, and it may have made it easier to
interact with and use OrienTrip. Findings may be different if eye
movements were measured on a significantly smaller screen such as an
iPhone. At the time of this writing, unfortunately, accurate eye tracking
technology is not available for mobile devices.
Author statement
MR conceived of the study, participated in the design and co-
ordination of the study, performed analyses, and drafted the manu-
script. DM and TT conceived of the study, participated in its design,
coordination, and analyses, and revised the manuscript. HL participated
in the coordination. NC participated in the analyses. AK participated in
the design and revised the manuscript. All authors read and approved
the final manuscript.
Ethical approval
All procedures performed in studies involving human participants
were in accordance with Ethical approval was obtained from Human
Research Ethics Committee (HREC) at Curtin University (HRE2016-
0086) in Perth, WA, Australia.
Informed consent
Informed consent was obtained from all individual participants in-
cluded in the study.
Funding
This research was supported by the Cooperative Research Centre for
Living with Autism (Autism CRC).
Declaration of Competing Interest
The authors declare that they have no conflict of interest.
Acknowledgements
The authors would like to thank all individuals who took part in the
study. The authors also sincerely thank Professor Torbjorn Falkmer for
his comments and feedback and Ms Julia Tang for assisting with data
collection and participant recruitment. The authors also acknowledge
the support of Cooperative Research Centre for Living with Autism
(Autism CRC).
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Appendix

Appendix A. . Mobile app screens

Task 1 - Set “Mary Smith” as your care taker in the app.

Figs. A.1, A.2, A.3, A.4

Fig. A.1. Menu button on home screen.

14
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Fig. A.2. Configure carer option.

15
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Fig. A.3. Option to set up Mary Smith as carer.

16
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Fig. A.4. Done button confirming successful configuration.

17
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Task 2 - Before you start planning your trip, you want to explore the sensory overload tips, so you can be better prepared for the trip. Find the tip
for bright lights.
Figs. A.5, A.6, A.7

Fig. A.5. Assistance button to view assistance options.

18
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Fig. A.6. Sensory overload option for sensory overload help.

19
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Fig. A.7. Sensory overload strategies for bright lights option.

20
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Task 3 - Plan a trip from Current Location to Museum Street, departing at 3:00pm.
Figs. A.8, A.9, A.10

Fig. A.8. From field to enter their source location.

21
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Fig. A.9. To field to enter their destination.

22
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Fig. A.10. Search button to search for routes.

23
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Task 4 - Select the route with the least number of changes and start trip.
Task 4a - How many routes has been suggested to you?
Fig. A.11

Fig. A.11. App suggests public transport routes to destination.

24
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Task 4b - How many changes does each route have?

Fig. A.12

Fig. A.12. Number of changes each route requires

25
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Task 4c - What is the crowdedness of each route?

Fig. A.13

Fig. A.13. Expected crowdedness of each route.

26
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Task 4d - Find the route with the least number of interchanges, (i) What time does this route depart? (ii) Can you describe this route?
Fig. A.14

Fig. A.14. Detailed description of route two.

27
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Task 5 - How many stops are there in your trip including the last stop.
Fig. A.15

Fig. A.15. Stops along the user's journey. Blue dot marks users current location.

28
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Task 6 - Now imagine you're on the bus and the bus is moving, but you're starting to feel a little anxious about missing your stop. Find the anxiety
management tip on ‘Automatic negative thoughts’.
Figs. A.16, A.17, A.18

Fig. A.16. Assistance button visible during the journey.

29
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Fig. A.17. anxiety management option when assistance button is pressed.

30
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Fig. A.18. mindful breathing exercise to calm automatic negative thoughts.

31
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Task 7 - Find your current location in the journey map.

Fig. A.19

Fig. A.19. current location marker showing user is half-way through the journey.

32
M. Rezae, et al. International Journal of Human-Computer Studies 142 (2020) 102462

Task 8 - Imagine, you have arrived at your destination. Share your location with your care taker.
Figs. A.20, A.21,

Fig. A.20. Assistance option.

33
M. Rezae, et al.
Fig. A.21. Share location option to share current location with the configured carer.
References
American Psychiatric Association, 2013. Diagnostic and Statistical Manual of Mental
Disorders DSM-5.
Angell, A.M., Solomon, O., 2018. Understanding parents’ concerns about their children
with autism taking public school transportation in Los Angeles County. Autism 22,
401–413. https://doi.org/10.1177/1362361316680182.
Apple Inc., 2012. Making Your iOS App Accessible [WWW Document]. URLhttps://
developer.apple.com/library/content/documentation/UserExperience/Conceptual/
iPhoneAccessibility/Making_Application_Accessible/Making_Application_Accessible.
html(accessed 4.13.18).
Ayres, K.M., Mechling, L., Sansosti, F.J., 2013. The use of mobile technologies to assist
with life skills/independence of students with moderate/severe intellectual disability
and/or autism spectrum disorders: considerations for the future of school psychology.
Psychol. Sch. 50, 259–271. https://doi.org/10.1002/pits.21673.
Booth, R., Happé, F., 2010. Hunting with a knife and ... fork”: examining central co-
herence in autism, attention deficit/hyperactivity disorder, and typical development
with a linguistic task. J. Exp. Child Psychol. 107, 377–393. https://doi.org/10.1016/
j.jecp.2010.06.003.
Chan, W., Smith, L.E., Hong, J., Greenberg, J.S., Lounds Taylor, J., Mailick, M.R., 2017.
Factors Associated with Sustained Community Employment Among Adults with
Autism and Co-occurring Intellectual Disability. Autism 136236131770376. 10.
1177/1362361317703760.
Chien, M.-E., Jheng, C.-M., Lin, N.-M., Tang, H.-H., Taele, P., Tseng, W.-S., Chen, M.Y.,
2015. iCAN: a tablet-based pedagogical system for improving communication skills of
children with autism. Int. J. Hum. Comput. Stud. 73, 79–90. https://doi.org/10.
1016/j.ijhcs.2014.06.001.
Cowen, L., Ball, L.J, Delin, J., 2002. An eye movement analysis of web page usability.
People and Computers XVI - Memorable Yet Invisible. Springer London, London, pp.
317–335. https://doi.org/10.1007/978-1-4471-0105-5_19.
Cristina, T., Britto, P., Pizzolato, E.B., 2016. Towards web accessibility guidelines of
34
International Journal of Human-Computer Studies 142 (2020) 102462
interaction and interface design for people with autism spectrum disorder. In: Ninth
Int. Conf. Adv. Comput. Interact, pp. 138–144.
Cutrell, E., Guan, Z., 2007. What are you looking for? An eye-tracking study of in-
formation usage in Web search. In: CHI 2007 Proceedings, pp. 407–416.
Darejeh, A., Singh, D., 2013. A review on user interface design principles to increase
software usability for users with less computer literacy. J. Comput. Sci. 9,
1443–1450. https://doi.org/10.3844/jcssp.2013.1443.1450.
Dawe, M., 2006. Desperately seeking simplicity: how young adults with cognitive dis-
abilities and their families adopt assistive technologies. In: CHI 2006, dl.acm.org.
De Leo, G., Gonzales, C.H., Battagiri, P., Leroy, G., 2011. A smart-phone application and a
companion website for the improvement of the communication skills of children with
autism: clinical rationale, technical development and preliminary results. J. Med.
Syst. 35, 703–711. https://doi.org/10.1007/s10916-009-9407-1.
Deka, D., Feeley, C., Lubin, A., 2016. Travel patterns, needs, and barriers of adults with
autism spectrum disorder. Transp. Res. Rec. J. Transp. Res. Board 2542, 9–16.
https://doi.org/10.3141/2542-02.
Ehmke, C., Wilson, S., 2007. Identifying web usability problems from eye-tracking data.
In: BCS-HCI ’07 Proceedings of the 21st British HCI Group Annual Conference on
People and Computers: HCI...but Not as We Know It. pp. 119–128.
Eraslan, S., Yaneva, V., Yesilada, Y., Harper, S., 2019. Web users with autism: eye tracking
evidence for differences. Behav. Inf. Technol. 38, 678–700. https://doi.org/10.1080/
0144929X.2018.1551933.
Eraslan, S., Yaneva, V., Yesilada, Y., Harper, S., 2017. Do web users with autism ex-
perience barriers when searching for information within web pages? In: Proceedings
of the 14th Web for All Conference, W4A 2017. Association for Computing
Machinery, Inc. https://doi.org/10.1145/3058555.3058566.
Falck-Ytter, T., Bölte, S., Gredebäck, G., 2013. Eye tracking in early autism research. J.
Neurodev. Disord. 5, 28. https://doi.org/10.1186/1866-1955-5-28.
Feeley, C., 2010. Evaluating the transportation needs and accessibility issues for adults on
the autism spectrum in New Jersey. In: 89th Annu. Meet. Transp. Res. Board, pp.
10–14.
Fletcher-Watson, S., Pain, H., Hammond, S., Humphry, A., Mcconachie, H., 2016.
M. Rezae, et al.
Designing for young children with autism spectrum disorder: a case study of an iPad
app. Int. J. Child-Comput. Interact. 7, 1–14. https://doi.org/10.1016/j.ijcci.2016.03.
002.
Foster, E.C., 2014. User interface design. Software Engineering: a Methodical Approach.
Apress, Berkeley, CA, pp. 187–205. https://doi.org/10.1007/978-1-4842-0847-2_11.
Grammenos, D., Paramythis, A., Stephanidis, C., 2000. Designing the User Interface of an
Interactive Software Environment for Children. researchgate.net1–22.
Granka, L.A., Joachims, T., Gay, G., 2004. Eye-tracking analysis of user behavior in WWW
search. In: Proceedings of the 27th Annual International Conference on Research and
Development in Information Retrieval - SIGIR ’04, pp. 478. https://doi.org/10.1145/
1008992.1009079.
Groba, B., Canosa, N., Concheiro-Moscoso, P., 2018. Guidelines to support graphical user
interface design for children with autism spectrum disorder: an interdisciplinary
approach. Proceedings 2, 1177. https://doi.org/10.3390/proceedings2181177.
Grynszpan, O., Martin, J.-C., Nadel, J., 2008. Multimedia Interfaces for Users with High
Functioning Autism: an Empirical Investigation66, 628–639. 10.1016/j.ijhcs.2008.
04.001.
Guillon, Q., Hadjikhani, N., Baduel, S., Rogé, B., 2014. Neuroscience and Biobehavioral
Reviews Visual social attention in autism spectrum disorder : insights from eye
tracking studies. Neurosci. Biobehav. Rev. https://doi.org/10.1016/j.neubiorev.
2014.03.013.
Happé, F., Booth, R., Charlton, R., Hughes, C., 2006. Executive function deficits in autism
spectrum disorders and attention-deficit/hyperactivity disorder: examining profiles
across domains and ages. Brain Cogn. 61, 25–39. https://doi.org/10.1016/j.bandc.
2006.03.004.
Hatfield, M., Murray, N., Ciccarelli, M., Falkmer, T., Falkmer, M., 2017. Pilot of the
BOOST-ATM: an online transition planning program for adolescents with autism. Aust.
Occup. Ther. J. 64, 448–456. https://doi.org/10.1111/1440-1630.12410.
Hauser, F., Mottok, J., Gruber, H., 2018. Eye tracking metrics in software engineering. In:
ACM International Conference Proceeding Series. Association for Computing
Machinery, pp. 39–44. https://doi.org/10.1145/3209087.3209092.
Hendricks, D., 2010. Employment and adults with autism spectrum disorders: challenges
and strategies for success. J. Vocat. Rehabil. https://doi.org/10.3233/JVR-2010-
0502.
Hourcade, J.P., Williams, S.R., Miller, E.A., Huebner, K.E., Liang, L.J., 2013. Evaluation of
tablet apps to encourage social interaction in children with autism spectrum dis-
orders. In: Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems - CHI ’13. ACM Press, New York, , New York, USA, pp. 3197. https://doi.org/
10.1145/2470654.2466438.
Hussain, A., Abdullah, A., Husni, H., Mkpojiogu, E.O.C., 2016. Interaction design prin-
ciples for edutainment systems: enhancing the communication skills of children with
autism spectrum disorders. Rev. Tec. La Fac. Ing. Univ. Del Zuliahttps://doi.org/10.
21311/001.39.8.06.
Joachims, T., Granka, L., Pan, B., Hembrooke, H., Gay, G., 2005. Accurately interpreting
clickthrough data as implicit feedback. In: Proceedings of the 28th Annual
International ACM SIGIR Conference on Research and Development in Information
Retrieval - SIGIR ’05, pp. 154. https://doi.org/10.1145/1076034.1076063.
Kagohara, D.M., Van Der Meer, L., Ramdoss, S., O ’reilly, M.F., Lancioni, G.E., Davis, T.N.,
Rispoli, M., Lang, R., Marschik, P.B., Sutherland, D., Green, V.A., Sigafoos, J., 2013.
Using iPods 1 and iPads 1 in Teaching Programs for Individuals with Developmental
Disabilities: A Systematic Review. 10.1016/j.ridd.2012.07.027.
Kamaruzaman, M.F., Rani, N.M., Nor, H.M., Azahari, M.H.H., 2016. Developing user
interface design for children with autism. Procedia - Soc. Behav. Sci. 217, 887–894.
https://doi.org/10.1016/j.sbspro.2016.02.022.
Lindsay, S., 2017. Systematic review of factors affecting driving and motor vehicle
transportation among people with autism spectrum disorder. Disabil. Rehabil. 39,
837–846. https://doi.org/10.3109/09638288.2016.1161849.
Mat Zain, N.H., Abdul Razak, F.H., Jaafar, A., Zulkipli, M.F., 2011. Eye tracking in
educational games environment: Evaluating user interface design through eye
tracking patterns. In: Lecture Notes in Computer Science (Including Subseries Lecture
Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 64–73.
https://doi.org/10.1007/978-3-642-25200-6_7.
Mechling, L., O'Brien, E., 2010. Computer-based video instruction to teach students with
intellectual disabilities to use public bus transportation. Educ. Train. Autism Dev.
Disabil. 45, 230–241. https://doi.org/10.2307/23879809.
Miles, J.H., 2011. Autism spectrum disorders—a genetics review. Genet. Med. 13,
35
International Journal of Human-Computer Studies 142 (2020) 102462
278–294. https://doi.org/10.1097/GIM.0b013e3181ff67ba.
Mintz, J., 2013. Additional key factors mediating the use of a mobile technology tool
designed to develop social and life skills in children with Autism Spectrum Disorders:
Evaluation of the 2nd HANDS prototype. Comput. Educ. 63, 17–27. https://doi.org/
10.1016/j.compedu.2012.11.006.
Nivala, M., Hauser, F., Mottok, J., Gruber, H., 2016. Developing visual expertise in
software engineering: an eye tracking study. In: IEEE Global Engineering Education
Conference. EDUCON, pp. 613–620. https://doi.org/10.1109/EDUCON.2016.
7474614.
Ozonoff, S., Strayer, D.L., McMahon, W.M., Filloux, F., 1994. Executive function abilities
in autism and tourette syndrome: an information processing approach. J. Child
Psychol. Psychiatry 35, 1015–1032. https://doi.org/10.1111/j.1469-7610.1994.
tb01807.x.
Pan, B., Hembrooke, H.A., Gay, G.K., Granka, L.A., Feusner, M.K., Newman, J.K., 2004.
The determinants of web page viewing behavior. In: Dl.Acm.Org, pp. 147–154.
https://doi.org/10.1145/968363.968391.
Pavlov, N., 2014. User interface for people with autism spectrum disorders. J. Softw. Eng.
Appl. 07, 128–134. https://doi.org/10.4236/jsea.2014.72014.
Pavlov, N., Castro, M., Chukanska, Y., Molina, C., Mileva, N., Albetr, M.J., 2018. Mobile
graphical user interface with people with verbal communication disorders.
Colloquium in Information Science and Technology. CIST, pp. 391–395. https://doi.
org/10.1109/CIST.2018.8596478.
Poole, A., Ball, L.J., 2006. Eye tracking in HCI and usability research. Encyclopedia of
Human Computer Interaction. IGI Global, pp. 211–219. https://doi.org/10.4018/
978-1-59140-562-7.ch034.
Putnam, C., Chong, L., 2008. Software and technologies designed for people with autism.
In: Proceedings of the 10th International ACM SIGACCESS Conference on Computers
and Accessibility - Assets ’08. New York, USA. ACM Press, New York, pp. 3. https://
doi.org/10.1145/1414471.1414475.
Raymaker, D.M., Kapp, S.K., McDonald, K.E., Weiner, M., Ashkenazy, E., Nicolaidis, C.,
2019. Development of the AASPIRE web accessibility guidelines for autistic web
users. Autism Adulthood 1, 146–157. https://doi.org/10.1089/aut.2018.0020.
Sayagoa, S., Blata, J., 2010. Telling the story of older people e-mailing: an ethnographical
study. Int. J. Hum. Comput. Stud. 68, 105–120.
Shattuck, P.T., Narendorf, S.C., Cooper, B., Sterzing, P.R., Wagner, M., Taylor, J.L., 2012.
Postsecondary education and employment among youth with an autism spectrum
disorder. Pediatrics 129, 1042–1049. https://doi.org/10.1542/peds.2011-2864.
Sofian, N.M., Hashim, A.S., Ahmad, W.F.W., 2018. A review on usability guidelines for
designing mobile apps user interface for children with autism. In: AIP Conference
Proceedings, , 020094. https://doi.org/10.1063/1.5055496.
Sun, Q.(Chayn), Xia, J.(Cecilia), Nadarajah, N., Falkmer, T., Foster, J., Lee, H., 2016.
Assessing drivers’ visual-motor coordination using eye tracking, GNSS and GIS: a
spatial turn in driving psychology. J. Spat. Sci. 61, 299–316. https://doi.org/10.
1080/14498596.2016.1149116.
The World Wide Web Consortium, 2018. Web Accessibility Initiative (WAI) [WWW
Document]. URLhttps://www.w3.org/WAI/(accessed 10.5.18).
Wirtz, S., Jakobs, E., Ziefle, M., 2009. Age-specific usability issues of software interfaces.
In: Proc. 9th Int. Conf. Work With Comput. Syst. pp. 2–10.
Xie, B., 2003. Older adults, computers, and the internet: Future directions.
Gerontechnology 2, 290–304. https://doi.org/10.4017/gt.2003.02.04.002.00.
Yaneva, V., Ha, L.A., Eraslan, S., Yesilada, Y., 2019. Adults with High-functioning Autism
Process Web Pages With Similar Accuracy but Higher Cognitive Effort Compared to
Controls. In: Proceedings of the 16th Web For All 2019 Personalization -
Personalizing the Web - W4A ’19. ACM Press, New York, , New York, USA, pp. 1–4.
https://doi.org/10.1145/3315002.3317563.
Yaneva, V., Ha, L.A., Eraslan, S., Yesilada, Y., 2018. Autism and the web: using web-
searching tasks to detect autism and improve web accessibility. SIGACCESS Access.
Comput. https://doi.org/10.1145/3264631.3264633. 2:1–2:1.
Yaneva, V., Temnikova, I., Mitkov, R., 2015. Accessible Texts for Autism. In: Proceedings
of the 17th International ACM SIGACCESS Conference on Computers & Accessibility -
ASSETS ’15. ACM Press, New York, New York, USA, pp. 49–57. https://doi.org/10.
1145/2700648.2809852.
Zalewska, A., Migliore, A., Butterworth, J., 2016. Self-determination, social skills, job
search, and transportation: Is there a relationship with employment of young adults
with autism? J. Vocat. Rehabil. 45, 225–239. https://doi.org/10.3233/JVR-160825.