Assessment Task Number 10

Evaluation Techniques

1. Design an experiment to test whether adding color coding to an interface will improve
accuracy. Identify your hypothesis, participant group, dependent and independent variables,
experimental design, task and analysis approach.

Hypothesis: The accuracy will increase if color coding is implemented in the interface for a
Student Information System.
Participant Group: BSIT Third Year students of Our Lady of Fatima University
Dependent Variable: The accuracy of the interface design.
Independent Variable: The application of color coding into the interface.
Experimental Design: There are about 30 to 40 available participants for each section of the
participant group. The interface design that will be evaluated is the interface for a Student
Information System, so yes, the chosen participant group rightfully represents the user group.
Task and Analysis Approach:
Experimental Task and Questionnaires will be used. An interface with color-coded buttons,
and an interface with nonuniform colored buttons will both be used by the participant group.
To measure the accuracy, the participant group will be asked to perform tasks using the two
interfaces. The length of time that it took them to perform the tasks will be recorded and
compared. The participant group will answer a scalar questionnaire afterwards, to record
their feedback from their experience in using the two interface designs.
Since the dependent variable is continuous and the independent variable is discrete, the rank-
sum versions of ANOVA will be used as the statistical measure to analyze the gathered
results.

2. Choose an appropriate evaluation method for each of the following situations. In each
case identify:
(i) the participants
(ii) the technique used
(iii) representative tasks to be examined
(iv) measurements that would be appropriate
(v) an outline plan for carrying out the evaluation.

(a) You are at an early stage in the design of a spreadsheet package and you wish to test
what type of icons will be easiest to learn.

(i) Third year BS Accounting Students of Our Lady of Fatima University

(ii) Query Technique
(iii) Identifying icons
(iv) Rank-sum versions of ANOVA
(v) About 100 respondents from the chosen participant group will be given a
paper with pictures of different icons. The participant will identify the icon
based on their own perception, then write it down in the questionnaire.

(b) You have a prototype for a theatre booking system to be used by potential theatre-
goers to reduce queues at the box office.

(i) Members of the Theatre Club

(ii) Query Technique
(iii) Using the prototype to book a ticket
(vi) Rank-sum versions of ANOVA
(iv)About 100 respondents from the chosen participant group will use the theatre
booking system prototype to create an account and book a ticket. The usability
of the prototype will be measured through a scalar questionnaire, and
feedback or suggestions on what to improve in the prototype will be gathered
through an Open-ended Questionnaire.

(c) You have designed and implemented a new game system and want to evaluate it
before release.

(i) Members of a E-Gaming Club

(ii) Query Technique
(iii) Using the new game system when playing
(iv)Rank-sum versions of ANOVA
(v) About 120 respondents from the chosen participant group will use the new
gaming system. The respondents will play one round of game and through
their firsthand experience of the new gaming system, they will be asked to
evaluate it through a Scalar Questionnaire.
(d) You have developed a group decision support system for a solicitor’s office.
 (i) Solicitors that reside in Quezon City
(ii) Query Technique
(iii) Using the group decision support system
(iv)Rank-sum versions of ANOVA
(v) About 50 Solicitors from the chosen participant group will use the group
decision support system to formulate a solution or a conclusion for a case. As
this involves legal matter, constructive feedback from the user will be
obtained through an interview after they’re done using the system.

(e) You have been asked to develop a system to store and manage student exam results
and would like to test two different designs prior to implementation or prototyping.

(i) CCS Faculty of Our Lady of Fatima University

(ii) Think Aloud and Cooperative Technique
(iii) Evaluating the two interface designs
(iv)Rank-sum versions of ANOVA
(v) The whole CCS Faculty will evaluate the two designs’ affordance by
identifying the objects in the interface design. The participant group’s
understanding of how the objects in the interface functions will be measured
through their immediacy of response. There would be an evaluator who will
observe if the participant is confused or having a hard time understanding the
design.