INF4820

2020

Assignment
2
number:
Assignment
797334
unique code:
Student number: 67685862
Last name &
SS GUMEDE
Initials

Statement of Originality
This is to certify that to the best of my knowledge; the content of this assignment is my own work.

I certify that the intellectual content of this assignment is the product of my own work and that all
the assistance received in preparing this assignment and sources have been acknowledged.

School of Computing
Question 1
Which of the following examples presents an "unmistakable usability issue"? Answer either Yes, or No, or
Not necessarily

a) Yes

b) No

c) No

d) No

e) Yes

Question 2
a) When doing the usability study, its best to choose one usability specialist when using usability
rating because: (://measuringu.com/severity-ratings/)

 The usability rating systems needs to be consistent as there a few different ones,
having multiple experts might delay the study and it will not provide ease in
comprehending the usability issues. Experts will need to reach an agreement as
there will be judgement.

 It will be difficult to get good ratings; each expert will find a different rating of the
usability study. There should be aggregation of ratings, discuss or get the average
before they can be reported.

b) 20

c) I would use the below analyses:

 Issues by Category: Identifies the focus of design improvements from tactical perspective
and only on certain areas of the system (categories are group). Presents the frequency of
usability issues grouped by categories and version/design iteration.

 Issues by Task: Identifies the number of tasks that should be focused on for the next
version / design iteration of the system and issues are analysed on task level.

 Frequency of Unique Issues: Analyses the frequency of unique usability issues that are
most useful in an iterative design process, its best when you want informative data about
issues that occur with each new version/design iteration.

 Frequency of Issues Per Participant: Analyses nonunique issues experienced by each

participant over different versions/design iterations. Its best for analysing average issues
experienced in two web-based system versions experienced by participants in iterations.

School of Computing
  Frequency of Participants: An analysis of important usability issues and identifies the
consistency between participants and designs. Useful metric when you need to focus on
whether there is improvement in certain elements of design.

Question 3
Participant T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Proportion
1 x x x x x x x 0.7
2 x x x x x 0.5
3 x x x x x x 0.6
4 x x x x 0.7
5 x x x x x x x 0.7
6 x x x x x x x 0.7
7 x x x x 0.4
8 x x x x x x x x x 0.9
9 x 0.1
10 x x x x x x x 0.7
Proportion 0.4 0.4 0.4 0.6 0.8 0.6 0.6 0.7 0.6 0.6 0.585

Sample 3

= 1−(1−P), N

=1−(1− 0.585), 3

=0.93 or about 93%, of the issues would be identified with a sample of 5 users from this study.

Sample 4

= 1−(1−P), N

=1−(1− 0.585), 4

=0.97 or about 97%, of the issues would be identified with a sample of 4 users from this study.

Sample 5

= 1−(1−P), N

=1−(1− 0.585), 5

=0.99 or about 99%, of the issues would be identified with a sample of 5 users from this study.

Question 4
Consider the following conditions and then state if you would use 5-10 or more than 10 users in a usability
test

School of Computing
 a) More than 10 users

b) 5 - 10 users

c) 5 – 10 users

Question 5
I would capture the reported data on emotional/task level and use likert scales (rating scales)

The below question would be asked “You will definitely download the application once its released” , the
below statement will need a user to provide an answer using the five -point rating scale , the following is
used to rate the statement for the user.

1 = Strongly disagree

2 = Disagree

3 = Neither agree nor disagree

4 = Agree

5 = Strongly agree

(P.T.O)

Question 6
Desktop system name: Iress Pro – Market data and Trading System

School of Computing
 Strongly Strongly
Disagree Agree

1. I think that I would like to use Iress √ Pro

frequently 4
1 2 3 4 5

2. I found Iress Pro unnecessarily √ complex

4
1 2 3 4 5

√
3. I thought Iress Pro was easy to use 2

1 2 3 4 5

√
4. I think that I would need the support of
a technical person to be able to user 1 2 3 4 5 2
Iress Pro
√

5. I found the various functions in Iress 1 2 3 4 5

Pro were well integrated 4
√

6. I thought there was too much 1 2 3 4 5

inconsistency in Iress Pro 1
√

7. I would imagine that most people 1 2 3 4 5

would learn to use Iress Pro quickly 1
√
8. I found Iress Pro very cumbersome to
1 2 3 4 5
use 4
√
9. I felt very confident using Iress Pro 3

1 2 3 4 5

10. I needed to learn a lot of things √ before 1

I could get going with Iress Pro

The scores were calculated as below :

1. 3-1 = 2

School of Computing
 2. 5-2 =3
3. 2-1 =1
4. 5-4 = 1
5. 3-1 = 2
6. 5-1 = 4
7. 2-1 = 1
8. 5-5 = 0
9. 2-1 = 1
10. 5-3 = 2

Total Score = 26

SUS Score = 26 * 2.5

= 65

According to Bangor and colleagues the above SUS score falls under the below range 50-70 and its
regarded as marginal score.

Question 7
a) The below are TWO suggested guidelines for preventing self-reported bias in usability testing data
collection:

 UX expert needs to collect the post-data in such a way that the moderator / UX expert or
the facilitator does not get to see the users’ score until the user who participate has left ,
one way to do this is to leave the room when the user completes the survey.
 Ask the users who participated in the usability test to complete a post-test survey when
they get back home or at the office , one way to do this is to send an email a user can click
on a URL that will redirect to an online survey.

b) Using 0 (Zero) in usability study allows UX experts to find stability by start numbering at 0.
This means rating scale starting at 0 allows 0 to correspond to 0% unlike to start numbering from 1
(assume its 1-5) it gives a correspondence percentage of 20%.

Question 8

a)

School of Computing
b)
DWELL TIME: Looks at the total amount of time participants have spent looking on a
certain AOI.

The longer the duration of looking at a certain AOI can indicate high level of interest and
information is well received, while less or short duration can indicate the total opposite
that one particular AOI of the homepage is more appealing than others. The level of
interest on a one AOI is conveyed.

School of Computing
 The below AOI will provide useful data as it’s the centre of the homepage. the following
data can be presented: entry time, dwell time , first fixation and fixation count.

 NUMBER OF FIXATIONS: The “number of fixations” with a certain AOI, this means the total
count of fixations.

Like dwell time, the reported dwell time on a certain AOI is recorded. The more the AOI
appeals the participants, the number of fixations will be high as opposed to an area with
less interest.

All AOI can provide useful data for each matric but the below is the logo of the team in the
homepage and number of fixations will be higher.

 FIXATION DURATION: Presents the average time for fixations lasted for, for participants on
how long the fixation lasted for.

More comparisons of AOI allows UX experts to confirm

which AOI had more focus that other AOI and what are
possible reasons. Average fixation durations combine
dwell time and number of fixations. This matric can provide useful data on the below AOI.

School of Computing
 SEQUENCE: Presents the sequence in which each AOI is first fixed

The order of attention in eye tracking reflects the participants interest and how each
elements of the website homepage stand out in terms of their colour, using the sequence

fixation on the homepage screen will provide useful data as in which AOI provides the first
interest and which one gets the last interest , example is as below :

School of Computing
 TIME TO FIRST FIXATION: Presents the amount of time it takes users to first notice a
certain element.

This matric can be used on the below AOI to indicate the amount of time it will take a
participant (or all participants on average) to for an element, in this case on the homepage

we can find how long it takes a participant to find the football club logo and the search
button.

 REVISITS: Presents the number of times that the eye fixates within an AOI

This matric can be use on the below AOI to find the number of revisits and this will provide
useful data about how many times a participant leaves the AOI and comes back to have a
look again , or does the participant fixate and never comes back or they find themselves
coming back to the AOI. The follow can be presented: Ratio, Re-visitors, Revisits and
Fixations.

School of Computing
 HIT RATIO: Presents the percentage of users who participated and had at least one fixation
with the AOI.

The hit ratio looks determines the information (in %) about how many of participants have
looked on a specific AOI (such as football club logo on the homepage, adverts or interesting
story) . This provides a useful order the actual percentage of people who have looked at
the specific AOI, the following data can be presented: time spent, ration and fixations.

This matric can be used in the below AOI:

School of Computing
Question 9
Three ways are presented on how to derive new usability metrics from the product I have choose and data.
I will be deriving new usability metrics from my data and there after I will combine metrics to measure
usability.

1. Combining Metrics Based on Target Goals

I will be combining different metrics data and check if when combined they meet the target goal,
there for each combination of these metrics, in this case it will be task completion and task time
that will be measured.

The target goal will consider data point of two metrics, the goal is to measure If each of the
following participants (8) managed to complete at least 85% of their tasks without issues and task
time is less or equal to 70 seconds on average time. The target will get only 1 if the criteria is met
and 0 if not.

Participant Task completion Task Time (Seconds) Target Met?

1 81% 50s 0

2 97% 70s 1

3 68% 80s 0

4 75% 90s 0

5 86% 69s 1

6 98% 50s 1

7 80% 78s 1

8 95% 57s 1

Average: 85% 68s 63%

The results show that 5 out 8 participants have met the criteria, where complete time is at least
85% and task time is less or equal 70 seconds. The average for task completion is 85% and for task
times is 68s, it shows the task target was met as the percentage is 62.5%.
The results are looking good and all participants did participate in the test which made the results
to be reasonable in any way.

School of Computing
 2. Combining Metrics Based on Percentages

The below technique will be trying to combine scores on different scales and convert each score to a
percentage and after average the scores, this allows different metrics to be combine when you do not have
any target goals.
Some of the columns in the table are; Time per task which is the average time to complete a task and
presented in seconds , task completed presents the number of all the task and total to 10.The last column
is the rating column as I will rate each task ( from zero to four) where four is regarded the best.

Participant # Time Per Task (Seconds) Task Completed Rating (0-4)

(out of 10)
1 55 9 3.2

2 60 10 3.8

3 49 7 2.2

4 78 5 1.3

5 59 9 2.6

6 80 6 3.3

7 75 8 3.1

8 89 7 2.7

9 73 7 1.2

10 28 4 1.1

The below second table is calculated by taking the manipulating data as follows from the first table:

School of Computing
For task completion, each score was divided by 10 to get a percentage, For rating score was divided by 4
and for the time column , firstly the difference between longest time 89 seconds and 28 seconds was
calculated then the difference of the longest time (89) and the score is further divided by the difference of
the longest and shortest time.

Participant # Time Per Task Task Completed Rating Average

(Seconds) (out of 10)
1 56% 90% 80% 75%

2 48% 100% 95% 81%

3 65% 70% 55% 63%

4 18% 50% 33% 33%

5 49% 90% 65% 68%

6 80% 60% 83% 74%

7 15% 80% 78% 58%

8 0% 70% 68% 46%

9 26% 70% 30% 42%

10 100% 40% 28% 56%

School of Computing