CONFIDENTIAL

SET B

UNIVERSITI TEKNOLOGI MALAYSIA

FINAL EXAMINATION SEMESTER I 2021 / 2022

SUBJECT CODE : SECV 2113

SUBJECT NAME : HUMAN-COMPUTER INTERACTION

SECTION : QIU

TIME : 7 DAYS (EXCLUDING WEEKENDS)

DATE OF DISSEMINATION : 23 JANUARY 2022 (10AM)

DATE OF SUBMISSION : 31 JANUARY 2022 (10PM)

VENUES : ONLINE (ALTERNATIVE ASSESSMENT)

INSTRUCTIONS:

1. Download and fill in the Integrity Declaration form. Upload the signed Integrity
Declaration form in the eLearning platform on 23rd January 2022, Sunday, by 5PM.
Failure to submit the signed Integrity Declaration form will cause your submission to
be not evaluated.
2. Answer all questions provided.
3. Follow the instructions as enclosed in the question to produce the report.
4. The interim submission is compulsory, which it must:
o be uploaded on e-learning
o contains at least 40% of work-in-progress,
o and submitted on the 4th day, 26th of January 2022, 5PM
5. The full and final submission is via e-learning. The full submission is on the 7th day,
31st January 2022.
6. Refer to the detailed instruction that is attached separately for the assigned case study.
This question paper consists of seven (7) printed pages excluding this page.
ALTERNATIVE ASSESSMENT [100 Marks]
This assessment focuses on heuristic evaluation in which each candidate must write and submit
an individual case study report.

QUESTION

You have been working in a design and innovation company for a few years, specializing in
research of the user interface (UI) and user experience (UX). As a valued member in your
team, you take pride in possessing the ability to empathize, observe/inspect, and analyze, all of
which are essential criteria of a UI/UX researcher.

As an alma-mater of UTM, you would like to contribute your expertise back to UTM by
improving UTM’s digital presence. You decided to offer a heuristic evaluation on UTM’s and
another institution’s existing websites for comparison purposes. The two websites’ URLs are
as follows:

UTM Alumni https://www.utm.my/alumni/

UM Alumni https://umalumni.um.edu.my/

The goal is to apply the Nielsen’s heuristics to critique the user interface design and to
discover the usability problems for both websites by performing a detailed heuristic
evaluation.

You will submit a Heuristic Evaluation Report based on the two websites. The content of
your report must demonstrate your understanding of the Heuristic Evaluation method, that
must consist of the following components:

1. Heuristics. List and explain ALL usability heuristics by Jakob Nielsen (1994).

2. Procedure. Describe how you conduct the heuristic evaluation and specify THREE (3)
suitable tasks to be applied on the websites. The same three tasks identified must be
performed on both websites.

3. Heuristic violation. Break down the problems found and categorize them based on
which heuristics it has violated. For each website, ensure that you find at least TEN (10)
or a maximum of FIFTEEN (15) violations. For both websites, in total, you would
 identify at least twenty (20), with the maximum thirty (30) violations. Provide a
thorough explanation for each problem.

4. Screenshots. Include the recorded screenshots (with annotations) and describe them to
support your discussion for Component (3). One violation should be supported by not
more than 2 screenshots. Include figure label and caption for the screenshot images and
these figures should be referred in your statement.

5. Severity rating. Describe the Nielsen’s severity rating system that you use alongside
your heuristic evaluation. Assess the severity of each problem which you identified from
both websites. Rate them using the system that you just described.

6. Recommendation. For each problem that you find, recommend potential solution for
improvement. Include images or illustrations of your recommendation, when needed.
Include figure label and caption for each of them and these figures should be referred in
your statement.

7. Comparison table. Construct a comparison table that describes and summarizes the
violations that you found for both websites. Analyze the results.

8. Conclusion. Conclude your report by reflecting your experience in doing the evaluation.

9. References. Provide at least 5 references. Cite them based on the Harvard reference style:
https://sps.utm.my/wp-content/uploads/2019/08/THESIS-GUIDELINES-070819.pdf
(page 33)

REPORT SUBMISSION

Consolidate all the above components into your own individual Heuristic Evaluation Report
with the following format:

Item 1 Cover page for Alternative Assessment submission

• Follow the format for cover page given in the next page

Item 2 Introduction section

• Write an Introduction to Heuristic Evaluation (HE) and why the need of
performing HE to this activity
 • Give a brief explanation on the websites involved and the aim of the
websites
• Provide some related works that describe and apply HE (you are
encouraged to get references from journal articles, conference
proceedings, books)
• End this section with a closing paragraph that highlight the structure or
the organization of your report
Item 3 Method section
• List and explain the heuristics (component 1)
• Describe how you conduct the Heuristic Evaluation (component 2)
• Write the suitable tasks selection for both websites (component 2)
Item 4 Results section
• List out the minimum of 20 violations for both websites, with
screenshots to illustrate the violations (component 3, 4)
• Assess the severity rating for all violations found (component 5)
• Hint: best to ensure all heuristics are presented at least once and
elaborate on your findings
Item 5 Discussion section
• Provide suggestion for improvement for each violation (component 6)
• Write a summary of the comparison result and analysis between the two
websites and support this with a table (component 7)
Item 6 Conclusion
• Close the report by providing reflection (component 8)
• Hint: What do you learn from the HE activity? What are the
contributions?
Item 7 References
• Provide a list of 5 academic reference journals/articles/etc. that you refer
to, with more than 50% of them are papers published less than 5 years
(component 9)
Item 8 Other formatting
• Use font type Times New Roman or Arial only, with font size 12.
• Use one and a half line (1.5) spacing.
• Ensure the maximum number of pages for the report is 30 (including
images)
 SESSION 2021/2022 SEMESTER 1

SECV2113
HUMAN-COMPUTER INTERACTION

ALTERNATIVE ASSESSMENT

HEURISTIC EVALUATION REPORT

CASE STUDY
<<https://www.utm.my/alumni/>> vs <<https://umalumni.um.edu.my/index.php>>

NAME
<<Danner Burhan Mohammed>>

MATRIC NO
<<QU202SECJ006>>

EMAIL
<<dbqiu200043@uniq.edu.iq>>

LECTURER’S NAME
<<Dr. Mohammed Shihab>>

SECTION
<< 3 >>
Introduction

Heuristic Evaluation (HE) is a method of evaluating usability for computer software, it

can be seen as a sort of inspection that helps the administrator of the software to identify
usability problem in the user interface design, in this method experts use rules of thumb to be
able to measure the usability of user interfaces based on the already established heuristics such
as Nielsen, Moich’s in order to reveal insights which can give the design team the ability to
improve their product’s usability from an early stage of development.
The aim of this activity is to perform Nielsen’s Heuristic Evaluation on the websites given in
order to find the violations and suggest ways to correct the violations in correct and appropriate
manner and then compare the two websites.

The given websites are websites which are made for universities and the aim of the given
websites is to offer information about their respective universities, the information should be in
a clear an accessible format in order to promote the best user interface experience possible for
the individuals who visit said website. Website number one represents the university of UTM
(University Teknologi Malaysia) while website number two represents the university of UM
(University MALAYA).

In this report paper we will be applying the Nielsen’s 10 usability heuristics in order to evaluate
the given websites, compare them and locate the violations made to the 10 heuristics as well as
providing a suggestion as to how the websites can be improved. Throughout the course of this
report paper, we will find a minimum of 15 violations in each of the website and tread
according to the given guidelines.
Methodology
The method used for conduction this research will be by using Jacob Nielsen’s 10
usability heuristics which were developed by Jacob Nielsen in the early 1990s and nowadays it
has been revised for current technology and is used to evaluate mobile devices, wearables and
virtual worlds, etc.…
This is the list of the 10 Usability heuristics,
1. Visibility of System status
The principle of this heuristic is that designs should be made to keep users informed
about whatever that is going on, through the appropriate and timely feedback.
meaning that the system should tell the user where they are in the system at the moment
and tell them the steps taken to reach the point that they are in at the moment in order to
reduce confusion.

2. Match between System and the Real World

In this principle of heuristics it states that the design made must be able to speak the
language that the user speaks, with the usage of words, phrases and concepts that the use
if familiar with rather than an unspoken and confusing language.

3. User Control and Freedom

This heuristic states that the users often make actions that they don’t mean by mistake
and that the design must be built in a way that lets the user correct their mistake with
ease and that it needs to have a clearly marked emergency exit to let go of the unwanted
action.

4. Consistency and Standards

In the principle of this heuristic it states that the design must be made in a way that there
must not be confusion among the users regarding to the usage of different words,
situations and actions and whether that they mean the same thing or not.

5. Error Prevention
This heuristic establishes that in order for a design to be considered a well-made design
is to prevent errors from happening in obvious places, every aspect of the design should
be tested beforehand so that if there is an error anywhere it should be prevented and
fixed as soon as possible.
6. Recognition rather than Recall
This heuristic establishes that the design must be made in a way that allows the user to
use very little of their memory and minimize their memory load, we can make this
happen by making elements, actions and options visible in a way that avoids making the
users remember information.

7. Flexibility and Efficiency of Use

This heuristic makes it clear that the design should implement elements which make it
easier for an expert user to use, for example shortcuts that may be hidden from the
regular user but are essential to make an expert’s interaction with the design faster and
more convenient.

8. Aesthetic and Minimalist Design

The main point of this heuristic is to avoid information which is irrelevant to the
interface because with each extra unit of information it is more likely that the main units
of information may be lost within a sea of non-important information, so the point here
is to avoid adding unnecessary information to our design and make the main points easy
to find and make the user’s interaction more convenient and faster.

9. Recognize, Diagnose and Recover from Errors

This heuristic establishes that in the design there must be appropriate error messages if
there is any, meaning that the error message should specify what the error indicates and
what it actually is rather than giving a code that the user will not be able to understand
which leads to further confusion, but none of this should matter if the design is
considered a good design because a good design shouldn’t run into errors and sending
error codes.

10. Help and Documentation

And in the last heuristic it says that sometimes it is necessary for the designer to add
helpful information in places of the design where it is needed to provide further
explanation all in order to improve user experience, however it is best if the design stays
away from this when it can avoid giving extra information because in some cases the
extra information may be seem as a violation to heuristic number 8 (Aesthetic and
Minimalist design).
 The Implementation of the evaluation is going to be based on the listed heuristics above, by
observing and checking every nook and cranny of the website and performing some basic
interaction with what the websites have to offer, it is going to be possible to look for
violations of the listed heuristics.
For the tasks we as the experts in User Interface (UI) User Experience (UX) will go through
the interface of both of the websites freely, the quality of the evaluation will be better if we
as the expert predefine a set of mock tests to perform in the inspection, these tasks are either
critical or typical tasks. Under normal circumstances the expert would stay in contact and
work with the client to make the set of tasks that will be implemented throughout the
inspection, but since we can't do that, we have to resort to different measures, the tasks will
mostly be what any typical user would try to do with the website, for example opening
specific parts of the website to get some information and seeing if the process goes without
any issues.

Results Section
Severity Rating
Severity rating is a rating system that helps give a rough indication to how severe the violation
of the heuristic is, if the severity ratings are many in numbers and severe violations exist then it
may not be a good idea to release the program, but if there isn’t that many violations and the
ones that exist have low severity rate then there might not be problems releasing the program.
In order to determine a severity rating you need to keep 3 factors in mind, they are Frequency,
Impact and Persistence.
And the Nielsen’s severity rating system is a number-based system ranging from 0 to 4 where 0
is (No problem), 1 is (Cosmetic problem), 2 is (Minor usability problem), 3 is (Major usability
problem) and 4 is (Usability catastrophe)

The website number 1 UTM Alumni https://www.utm.my/alumni/:

The problems:

1: At the home page scroll down a little and you can see that there is a list of pictures
with texts written over them in a big font which leads to the user not being able to see neither
the photo nor the text, this is a violation of the (Aesthetic and Minimalist Design) heuristic.
Recommendation: separating the two in a way that lets the user to see both of them clearly.
Severity rating: 3

2: When you click on the Facebook icon which is made to redirect the user to the
Facebook page it doesn’t work and instead the user is presented with a page saying that the
page is not available (Error Prevention)

Recommendation: Replace the current Facebook link which doesn’t work with an
updated link that redirects the user to the destination it originally was meant to go.
Severity Rating: 4
 3: In the alumni photo album you can see a batch of unorganized topics that make it
hard for the user to find the particular topic that they are looking for in the alumni album,
(Aesthetic and Minimalist Design).

Recommendation: To add a sorting option for the albums and give the users the option to sort
by either date, A to Z, Z to A and most viewed.
Severity Rating: 2

4: When you click on a newsletter in the newsletter section found in

Publications→Newsletter, it takes you to a new tab where it tells the user that the resource has
expired this is a violation of the (Error Prevention) heuristic.

Recommendation: To either fix the links on the newsletter sections so it takes the user to
correct source or delete the section as it serves no purpose.
Severity Rating: 4

5: In the Alumni Shop tab there is an option that says SHOP NOW on the page but
when the user clicks it nothing happens and it just sends the user back to the same place,
This is a violation of the (Consistency and Standards) Heuristic.
Recommendation: Fix the design or remove the button.
Severity Rating: 3

6: The sign in and sign-up button is in a rather unusual place and it will be hard for the
user to find as it is usually visible in either top right or top left of the home page, but instead it
is in a slide on the home page that you have to look for, this is a violation of the (Recognition
rather than Recall) heuristic.

Recommendation: it is always a good idea to follow the usual place where famous websites
such as amazon, twitter, Facebook place their login buttons which is usually on top of the home
page.
Severity Rating: 1
 7: The website almost never uses phrases words or icons that the user recognizes and is
familiar with and it doesn’t speak the user’s language which is a violation of the (Match
between System and the Real World).

Recommendation: to add more elements that the user is familiar with.

Severity Rating: 2

8: When you click on the Applications button on the top left of the webpage it redirects
the user to a page that says “No Results Found”. This is a violation of the (Consistency and
Standards) heuristic.

Recommendation: either remove the button and replace it with another function and in this
case a (SIGN UP) button would be most fitting, or make a functional link that redirects the user
to the correct destination.
Severity Rating: 4
 9: The colors used in the website are very bright and hard on the eyes, the user’s eyes
will be tired from the colors if they are looked at for a long while and this is a violation of the
(Aesthetics and Minimalist Design).

Recommendation: using different color schemes that don’t look as bright.

Severity Rating: 1

10: The website only allows the user to browse it using the preset language that the website
uses, this makes confusion for people who can not speak the written language, this is a
violation of the (Flexibility and efficiency of use) heuristic.

Recommendation: Making a language button which lets the user change the browser’s
language to their desired language.
Severity Rating: 3

4: The website doesn’t give the user an option to exit out of situations that the user
accidentally walks into which is a violation to the (User Control and Freedom) heuristic.

Recommendation: To add a return to homepage button in the end of every page.

Severity Rating: 1
The website number 2: UM Alumni https://umalumni.um.edu.my/

The problems:

1: On the home page the first noticeable thing is the colour of the page and the font and
that they are both bright shades, this makes the text almost unreadable. This is a violation of the
(Aesthetics and Minimalist Design).

Recommendation: to either darken the text or change it to completely to a darker colour.

Severity Rating: 1
2: Following the error on the last website I decided to see if this website’s Facebook link
works or now, turns out it has the same issue as the 1st website where it takes the user to a page
that says (This page isn’t available), and this is a (Error Prevention) heuristic Violation.
Recommendation: to fix the link and make it serve the purpose it was supposed to originally
serve.
Severity Rating: 4

3: On the main page, there are tabs that lead to the content they describe, if you hover
over them, it opens a list and some of the items on said lists do the same if you hover over
them, this is a violation of the (Aesthetics and Minimalist Design).

Recommendation: To either add more tabs for the lists or remove them and have them
accessible on other pages as tabs.
Severity Rating: 1
 4: When the user clicks on a topic in the COMMUNICATION tab it takes the user to a
page that says “To view this page ensure that Adobe Flash Player version 10.0.0 or greater is
installed”. This is a violation of the (Flexibility and Efficiency of use) heuristic

Recommendation: To move to another method of showing the content because Adobe

Flash Player is not functional and there needs to be another way to show the content.
Severity Rating: 3

5: You cannot click the images on the main page (Meaning the slides) in order to view
the information that the images show, this is a violation of the (Consistency and Standards)
heuristic.

Recommendation: clicking the photo should take user to a page with more info on the image
Severity Rating: 2
 6: When the user attempts to search for something on the search bar they end up with
an empty page, this is the result no matter what the user attempts to search for, this is a
violation to the (Consistency and standards) heuristic.

Recommendation: Fixing the search bar so it becomes functional or remove it because it

serves no purpose
Severity rating: 3

7: When pressing the SIGN UP button the system takes the user to a page that says
REGISTRATION on the top of the page and the difference of these 2 words makes a
confusion for the user.
Recommendation: Only use one of the words to prevent confusion.
Severity: 2

8: There is an event calendar section that doesn’t show any upcoming events,
(Consistency and Standards) violation.

Recommendation: Add a calendar that has upcoming events marked on the days of the
event.
Severity Rating: 2

9: The appearance of the Sign-up option in many places and even in the sign-up
menu itself which is a violation of the (Aesthetics and Minimalist Design).

Recommendation: to remove the sign up button when not needed.

Severity Rating: 1
10: The System fails in indicating the location of the user within the website and this is a
violation to the (Visibility of System status) heuristic

Recommendation: To highlight the part that the user is currently in by coloring the current tab
differently.
Severity rating: 1

Comparison table

Usability Heuristics 1st Website 2nd Website

Visibility of System status The website didn’t violate The website did violate

Match between System and The Website didn’t violate The website did violate
the Real World

User Control and Freedom The website did violate The website did not violate

Consistency and Standards The website did violate The website did violate
Error Prevention The website did not violate The website did not violate

Recognition rather than The website did not violate The website did not violate
Recall

Flexibility and Efficiency of The website did violate The website did violate
Use

Recognize, Diagnose and The website did not violate The website did not violate
Recover from Errors

Help and Documentation The website did not violate The website did not violate

Aesthetic and Minimalist The website did not violate The website did not violate
Design
Conclusion:
To conclude the test, we can say that heuristic evaluation is a rather costly and time-consuming
procedure, but you can also say that it pays off well in the end and that it was a useful
evaluation that uncovers most of the errors that a system has and by only using rules of thumb
these errors can be easily recognized, evaluated for severity and eventually terminized.
Having all this said it is strongly recommended to hire an expert to do this evaluation in order to
ensure that the system you are trying to release is safe to use and doesn’t have any severe issues
that could cause the system to not function correctly, because these experts have been given
special training in order to find these issues in any system.
Reference list:
https://www.interaction-design.org/literature/topics/heuristic-evaluation
https://www.theguardian.com/higher-education-network/higher-education-network-
blog/2012/sep/14/university-websites-branding-user-experience
https://media.nngroup.com/media/articles/attachments/Heuristic_Summary1-compressed.pdf
https://www.sfu.ca/~jebowes/assignment---heuristic.html
 Scoring Rubric

Report Item Absent Minimal Standard Good Excellent Weigh Score

No. (0) (1) (2) (3) (4) t

Item 1: None Provided, or Information All All information 0.25 /1

Cover Page provided information incomplete but information complete and
incomplete, follow the complete but follow the
and did not format did not follow format
follow the the format
format

Item 2: None Minimal Provide Provide Provide

Introduction provided introduction background background background 1 /4
provided for information on information on information on
the report and the topic of the the topic of the the topic of the
case study report report report

with acceptable with good with excellent

explanation, and explanation explanation and
with < 5 with 5 with 5
references references references and
highlight the
aim of the
study

Item 3: Incorrect Listed 3 Listed 6 Listed 10 Listed 10

Method answer or heuristic types heuristic types heuristic types heuristic types 1.25 /5
none with accurate with accurate with accurate with accurate
provided explanation explanation explanation and well-
Component 1 articulated
Nielsen With With acceptable With explanation
Heuristics acceptable explanation, but acceptable
explanation, with incomplete explanation,
but with Nielsen’s HE but with
incomplete elements incomplete
Nielsen’s HE Nielsen’s HE
elements elements

Provided Provided Provided clear Provided clear 1.25 /5

Component 2 None provided minimal adequate methodology methodology
How to methodology on methodology on on the topic of on the topic of
conduct HE the topic of the the topic of the the report with the report with
report report good excellent
explanation explanation

Component 2 No task Provided 3 Provided 3 tasks Created a well- Created a well- 1.5 /6
Specify 3 provided tasks that that are aligned rounded 3 tasks rounded 3 tasks
tasks vaguely with the websites that are clearly that are clearly
aligned with aligned with the and fully
the websites websites aligned with the
websites
Item 4: None Violations Total of 10-14 Total of 15-20 Total of 21-30 5 /20
Results provided found are too violations found violations violations found
minimal (10 or found (min. 10 (min. 10 from
less) from each site each site)
Component 3 may not be the
Heuristic and with good case)
violations explanation explanation
are with
insufficient with explanation
explanation reflect student’s
reflect student’s excellent
good understanding
understanding

Component 4 None Recorded Recorded and Recorded and Recorded and 2.5 /10
Screenshots provided several annotated annotated clearly
screenshots but screenshots for screenshots for annotated
with no each problem each problem screenshots for
annotation each problem
with description
and no for several of the with with excellent
description for accompanied description for description for
the screenshots all all
accompanied accompanied accompanied
screenshots screenshots screenshots

Component 5 None Provide a Provide a rating Provide a Provide a rating 1.25 /5

Severity provided rating system system to be rating system system to be
rating to be used used to be used used
description
but with no with adequate with good with excellent
description of description of its description of description of
its purpose purpose its purpose its purpose

Component 5 None Provided Provided Provided Provided 2.5 /10

Severity provided / minimal but sufficient but sufficient sufficient
rating usage incorrect vague / vague / descriptions descriptions
rating inaccurate / inaccurate / with accurate with clear,
usage unreasonable unreasonable and reasonable accurate and
ratings ratings ratings reasonable
ratings

Item 5: No solutions Recommend Recommend Recommend Recommend 2.5 /10

Discussion provided potential potential solutions potential potential
solutions to to all problems solutions to all solutions to all
Component 6 only some but are not based problems but problems and
Improvement problems on principles and only some are all are
suggestions guidelines based on correctly based
principles and on principles
guidelines and guidelines
 No summary Provided Provided Provided good Provided 2 /8
Component 7 of comparison minimal acceptable summary of excellent
Comparison provided summary of summary of comparison, with summary of
table comparison, comparison, with table of comparison, with
with no table of no table of violations table of
violations violations violations

Item 6: No conclusion Conclusion is Conclusion is Conclusion Conclusion 1.5 /6

Conclusion provided generic and/or adequately reflects reflects
non- reflective and student’s student’s
Component 8 reflective expressed in learning learning
his/her own experience and experience and
words good excellent
understanding, understanding,
expressed in expressed
his/her own articulately in
words his/her own
words

Item 7: No references Cited minimal Cited acceptable Cited sufficient Cited sufficient 1.5 /6
References provided references (2 or number (3 or number (4 or number (5or
less) or less) of relevant less) of recent more) of recent
Component 9 irrelevant references and relevant and relevant
references references references with
in text
citations

Item 8: Insufficiently - - Sufficiently Fully followed 1 /4

Formatting followed followed
Use font type
Use font Use font type Times New
types other Times New Roman or
than Times Roman or Arial
New Roman Arial
or Arial and
and
and/or use 1.5-line
use 1.5-line spacing
use line spacing
spacing other and
than 1.5 and
report did not
and/or report did not exceed 30
exceed 30 pages, and
report exceed pages overall is neat,
50 pages tidy and well-
organized

Tot 100
al