Usability evaluation of touchscreen phone in emergency context
Fernanda Pozza, Vanessa Roncalio, Cristiana Miranda, Maria Lucia Okimoto
Federal University of Paraná - UFPR Curitiba, Brazil firstname.lastname@example.org
Emergencies are always unexpected, causing changes in the emotional state of the individual, such as tension and stress. In this situation, a simple task such as making a phone call can prove difficult, considering that the user may not be familiar with the device. Also, interfaces are often not designed for emergency use. This paper reports on a pilot test of the usability of a touchscreen phone in an emergency context, which consisted of three phases: pre-test questionnaire, test and post-test questionnaire. Thus, it was possible to obtain information about the usability of the device and the user´s response in a simulated emergency situation. Keywords: emergency context, usability, touchscreen phone
According to Paulheim et al. (2009), in their study "Improving Usability of Integrated Emergency Response System," the usability in an emergency situation is gaining more attention from researchers. He also highlights that the unpreparedness of designers and users to this issue is due to the fact that most devices are not designed considering their use during a future emergency scenario. Consequently,
Over the past few years have been brought to market devices with different capabilities and features. These are the means by which it is possible to interact with the device. According to the Brazilian Standard NBR 9241-11 (2002) usability is the ability of a product to meet certain goals to specific users with effectiveness. in this case. Most of the phones interfaces are composed by visual elements on the screen alongside with pressing or sliding buttons. the users are not very familiar with a particular interface to respond in a very short time.. to initiate the call (Task 4).when emergencies occur. because according to the NBR 9241-11 (2002).1
METHODOLOGICAL PROCEDURES Description of the tasks
Seven participants (n = 7) were chosen for this study. Thereby. The tasks to be completed by the participants included: to unlock the phone screen (Task 1). considering that he or she is under stress and out of their normal responses to the environment (Paulheim et al. Each participant was instructed to ‘make an emergency call’ from a locked screen phone. during a simulated emergency context. considering the specific context of use. the usability can vary significantly in different usage situations. Santos and Maciel (2010) states that cell phones today are designed to not only make calls but to be "mobile multimedia terminals. the context is an important factor in usability studies. the interfaces play a key role in communication between human and machine. it is important that the interface is designed to optimize the product´s performance. storage and processing. In order to evaluate the usability of a touchscreen phone." Thus. can lead to serious consequences to human life. It is known that emergency situations increase the probability of human errors. The severity of effects. considering only that they did not own the same model of the phone being tested. Such devices involve increasingly complex tasks. Were considered in the experiment. requiring more cognitive ability of the user. The touch screen phones belong to this category of devices. the emotional and behavioral aspects of the user and the interface characteristics of the cell phone. 2009). this study is based on the usability test of a touchscreen phone in a simulated emergency setting. Thus. in order to contribute with usability in emergencies. to activate the phone's keypad (Task 2). Nokia model 5530.
. this article deals only with issues related to the interface of the device being tested.
2 2. they offer more features. Therefore. efficiency and satisfaction. Hence. to dial the given number (Task 3). as illustrated in Figure 1. this pilot test was set for the users ‘to make a call'. However.
In the first room (Room 1).Figure 1 Tasks to be performed on the phone´s interface during the test
2. Thus. there was an observer next to the participant counting the number of ‘touches’ and another observer further away marking the time for the completion of the task. and a post-test questionnaire in order to enquire the participant's impressions about the device and the test itself.2
Description of the room and equipment
Two rooms were prepared for the test isolated from one another. as Figure 2:
Figure 2 Phases of the test
. The experiment was performed in three steps. each participant had their heart rate measured (initial frequency). it was possible to compare the initial heart rate (before testing). one facing the participant's face and torso and the second directed to the phone screen to assist in the counting of ‘touches’ the heart rate monitor adjusted on each participant with the digital display positioned on the table. In this room the participant also answered a pre-test questionnaire about stress perception. intermediate (during test) and final (right after the experiment) of each individual. equipped with: a laptop connected to speakers and the phone (test instrument) on a table. two video cameras. Besides the equipment. The test was conducted in Room 2.
being this time set from the time of the expert (10 seconds). 2008). the time taken to perform it and the efficiency. Every minute is crucial. with ease or not (Tullis. Because there are ethical principles that limit the level of stress the test participant is submitted. 1993).
. the participant was informed of the task a few seconds before executing it. You need to act fast! Pick up the phone. In this case. Albert. This comprises the cognitive and physical aspects and it can be defined as the user´s actions or steps to complete the action. Instead of creating the anticipatory period reported in TSST the present experiment was set to be unpredictable for the participant. 0.8 Hz). 2008). For task success the authors mean its completeness. Suddenly. To simulate the emergency context in this study we opted for an approximation of the person´s emotional state in a real emergency situation. The best known test for induction of stress in the laboratory is the Trier Social Stress Test . We tried to induce some significant levels of stress by combining the following factors: psychological suggestion triggered by a story told to the participant. However it was not intended to replicate the TSST procedures in this study.
2. For the induction of stress in the laboratory is commonly used sensory stimuli such as noise or exposure to extreme temperatures or psychological stressors that include demanding cognitive tasks and social assessment (Bollini. In this study were considered the time each participant took from the start of the first task to the completion of the last task.4 Usability metrics
To evaluate the usability of the touchscreen phone were applied these metrics proposed by Tullis and Albert (2008): task success. Your only option is to use your friend´s touch screen phone. your friend passes out. The TSST has an anticipatory period in which the participant is preparing for a speech to be presented to two or three evaluators.9 decibels . designed to cause anxiety and whose protocol has been adapted to induce stress in other studies. unlock it and call 0000-0000’. (Tullis. by the need to perform the task in the shortest time and with the disturbance caused by a sudden and loud siren (71. besides other monitored tasks. the efficiency can be measured by both the time and effort. we took into account the number of ‘touches’ and the time of each participant in relation to the number of ‘touches’ and time of an expert. For this metric.TSST (Kirschbaum. The total time for performing the test should not exceed two minutes. Each individual heard the following story: ‘you and a friend are in an isolated location. Albert. Hellhammer. generating increased levels of stress. According to Albert and Tullis (2008). Walker.2. as would occur in a real emergency. You try to call for help using your own cell phone when you realize its battery is discharged.3
An emergency event causes an immediate response in any person. 2002). Pirke.
we consider ´touch´ any attempt to unlock the phone and make the call.considering only individuals who have succeeded in all tasks. Figure 3 highlights (in gray) such issues.
Figure 3 semantic questionnaire
In open-ended questions were asked the following questions: • How do you think it should be the process to unlock the screen of the phone tested? • What did you think of the emergency simulation? • What did you think of this touchscreen phone?
3 3. proposed on a semantic differential scale. Thus. were asked questions related to opinions of the individual in relation to the test and the device being tested.5
The pre-test questionnaire reported on the emotional aspects. In the post-test questionnaire.
2. pressing or sliding them over the phone or on the screen. therefore. whether giving quick touches with fingertips. the number of ´touches´ was counted until the individual accomplish the tasks. not to be covered in this article. Thus. For this study.1
This action demanded more time for all participants. as shown in Table 2.
Table 1 Comparison of initial. The task time for each participant. Among the participants there was only one woman.
Table 2 Total time and number of touches of each participant Participants Time to accomplish all tasks Number of touches
. as shown in Table 1. The verification of the influence of these elements can be seen by comparing the initial heart rate with the both the intermediate and final heart rate. by the loud siren during testing and by the urgency to complete the tasks in the shortest possible time. as well as the number of ‘touches’ made by them can be viewed and compared to the time and ‘touches’ of the expert. Therefore.3
Success and task time
Two participants (2 and 5) failed to complete the task "to unlock screen" because they did not find the unlocking button of the device. Samsung and Gradient. all other participants showed a higher heart rate during the test. we believe that these individuals were influenced by the emergency resources used . however they were different models of the device being tested. Four participants had Nokia cell phones.history. loud siren and timing. 40% between 22 and 25 and 60% between 32 and 49.The group aged between 22 and 49-years-old.
3. The other three participants had different brands: Foston. intermediate and final heart rates (beats per minute)
Participant 1 2 3 4 5 6 7 Initial 85 86 93 95 85 94 85 Intermediate 89 112 98 112 101 89 105 Final 76 97 95 99 94 90 103
It can be noted that with the exception of participant 6.2
Influence of emergency context
The emergency context in the proposed experiment was emphasized by these elements: a short story told to the participant.
16 0. as shown in Table 4.12 0. So in order to validate the pilot test were compared. first. the third participant was able to make the call under the expert time.
Table 3 Measured efficiency considering the time to complete all tasks. he used the auto redial instead of following the specified tasks.25
. as shown in Table 3. were considered the number of touches of the participants (PN) and the number of touches of the expert (EN).Expert 1 2 3 4 5 6 7
10’’ 60’’ 50’’ 36’’ 82’’ 36’’ 75’’ 31’’
11 38 42 8 49 38 43 14
It is noted that the lowest time achieved by the participants was 31 seconds while the expert took 10 seconds. In a second phase. with 14 touches.
Participants 1 4 6 Efficiency (effort) 0. the time for each participant (PT) at the time of the expert (ET). EN / PN. as suggested by Tullis and Albert (2008).28 0. However. In relation to the number of touches.13 0.32
Table 4 Measured efficiency considering the effort to complete all tasks.
Participants 1 3 4 6 7 Efficiency (time) 0.4
To measure the efficiency were taken only participants who were successful in all tasks.22 0. Therefore. considering ET / PT. the participant closest to the number of touches of the expert was number seven.
none of the subjects achieved the efficiency ratio taken as the parameter. The responses obtained with the semantic differential were coincident in some aspects and discordant in others. his results were not included in the calculation of the efficiency and effort.32 for time and 0. When questioned about how the screen unlocking process should be.
. as shown in Figure 4. the efficiency rate in both cases equals to 1. Participants did not consider the device too complex and most of them said it was easy to use it. with 0. Therefore.
Figure 4 Results of the semantic differential scale
By analysing the responses certain inconsistencies became evident.5
Results of the independent variables
The post-test questionnaire allowed the collection of additional data. Observing the Tables 3 and 4. one participant said it should be automatic. while most of them agreed partially that they need instructions to use it.
Considering that the third participant made the call using the auto redial instead of dialing the number. it can be argued that the participant 7 came closest to the index of the expert. All of them agreed that the figures on the initial screen are easy to understand. only two participants found the device easy to use in an emergency. Following the formulas described and considering the time and effort of the expert. Five participants found it difficult to identify the unlocking button of the device. They also agreed fully or partially that others will also have difficulty using the product.7
0. while the call was considered an easy task for most of them. we obtained the following: five participants suggested that the unlocking should be integrated into the touchscreen display. However.78 for effort.
it failed sometimes" “Good and practical" "An aesthetically beautiful device. a little concentration was required" "Interesting" "It enabled an experience that really could happen" “"I really got the impression of an emergency" "Very interesting" "Interesting" "Simple and practical"
Asked about their views on the mobile device tested. About the emergency simulation test.and another said it did not need to change at all. By comparing these responses to the results of time and effort it can be noted that there is an inconsistency between their performance and opinions. with the addition of elements in the environment such as a
. I've had worse gadgets" "Apart from the unlocking button the phone is apparently good" "Better than my own phone" "With minimal tools for the user"
The proposed method of usability in emergencies consisted of an exploratory study of the context of use. the opinions given by the participants are as follows (Table 5):
Table 5 Participants' opinions about the test.
Participants 1 2 3 4 5 6 7 Comments about the phone "The touch screen display is a bit weak.
Participants 1 2 3 4 5 6 7 Comments about the test "Well. It could be inferred that the emergency context altered the performance of users. but it took some time to figure out how to unlock it" "Okay. the participants pointed out that (Table 6):
Table 6 Participants' opinions about the device being tested.
But the manufactures does not consider that the same device might be used in an emergency by someone other than the owner of the phone. Paulheim. Pirke. and presenting usability metrics. 2008. 2012. Rio de Janeiro.L. Robson. 2009.parte 11 . 2010. K. NBR 9241-11: requisitos ergonômicos para trabalho de escritórios com computadores . From the techniques applied (questionnaires and test) and metrics used (success of tasks. Such difficulty was due to the lack of signaling and information about this action in the device. were obtained inconsistent responses. Tullis.. Hellhammer. Improving Usability of integrated emergency response systems: the SoKNOS approach.sudden and loud noise through siren and a timing device. analyzing. It is believed that this can be mitigated by asking more objective and specific questions.
Bollini.jasnh. 1993.com/c2.. Francimar. were gathered sufficient information to assess the usability of a touchscreen phone. Dirk H. PUC-Rio. a comparative study needs to be done in a context of stress free environment. Accessed February 5. From this.orientações sobre usabilidade.. Elaine F. knowing the importance that mobile phones have gained in our lives it is recommended that there should be more specific guidelines for usability considering different contexts of use. Maciel. Massachusetts: Morgan Kaufman. The perception of the influence of these factors can be attributed to the increased heart rate during and after the test. Estilos de interação em interfaces para dispositivos móveis. Associação Brasileira de Normas Técnicas . H. However. so the question is: are the products designed for an emergency situation? Thus.H. Santos. Measuring user experience: collecting... In Proceedings of GI Jahrestagung. http://www.htm. Kirchbaum. Anna M. Bill.T. and Ziegert. Clemens. The Trier Social Stress Test: a tool for investigating psychobiological stress responses in a laboratory setting. Rio de Janeiro: ABNT.. In the post-test questionnaire. 2002. considering the proposed tasks in an emergency context. Regarding the usability of the touchscreen phone tested. S. 1435-1349. Tom. one cannot state with certainty to in what degree the emergency context caused tension and stress in participants. 28:76-81. Karl-Marlin. To be able to access the features of the mobile phone can be decisive in an emergency. it is believed that the manufacturers assume that the phone users have read the manual and therefore know how to use all the phone features. Döweling. Walker. This was emphasized in the post-test questionnaire. Thus. Efficacy of a laboratory stressor: failure to replicate the Trier Social Stress Test. the main difficulty for the participants was to unlock the screen to access their features. F. time and efficiency).ABNT. Tso-Sutter. Neuropsychobiology. Probst. Ação Ergonômica 5: 21-27. Albert.