Usability Testing Report

Indianapolis Public School
Te a m 2 S t eve n E n t e z a r i Huang Hai Dan Ja y W h e e l e r

Brief recall of the inspection result

Entezari, Huang, Wheeler - IPS

 

Most Salient Problems

Entezari, Huang, Wheeler - IPS

Method & Instruments

Entezari, Huang, Wheeler - IPS

Method
Participants Parents Teachers Community members


Method Pre-evaluation User profile Evaluation  Environment  Tasks  Post-task Question Post-evaluation Questionnaire

Entezari, Huang, Wheeler - IPS

Participant profile
Participants: 11 Age: 24~55 Computer Experience: Experience(6), Somewhat(3) Role: Teacher(2), Parent(4), Community(5) Internet familiarity: 20+hrs per week.
Steve Jay Gender Participant 1 male Participant 2 female Participant 3 female Participant 4 female Participant 5 male Participant 6 female Participant 7 female Participant 8 female Participant 9 female Participant 10 male Participant 11 female Age 56 44 29 53 49 55 31 26 24 27 21 Computer Experience Somewhat Somewhat Experienced Somewhat Experienced Somewhat Experienced Experienced Experienced Experienced Experienced Internet Usage 40 hrs 30hrs 30+ hrs 20 hrs 40+ hrs 20 hrs 30 hrs 40 hrs 40 hrs 40 hrs 20 hrs Participan t Type Parent Parent Teacher Community Parent Community Teacher Community Community Parent Community

Haidan

Steve

Entezari, Huang, Wheeler - IPS

Instruments
Script

Pre-evaluation Form  Evaluation Task List  Post-task question  Post-Evaluation Form 

Entezari, Huang, Wheeler - IPS

Tasks

1. Find Board Meeting Date 2. Find Technology Budget Plan 3. Find contact information for a College 4. Find a School Near Home 5. Locate the English Student application for a Magnet School 6. Find contact info for the payroll office (Teacher only)

As a parent you are growing concerned about the direction the IPS School board is moving towards concerning year round school. Another parent mentioned that there are school board meetings where you can go and voice your concerns. Assume today is December 1st and you want to go to the next briefing session. Please identify the date for this meeting.

As a parent you feel that your child’s ability to learn and use technology is very important. In the midst of growing concerns about budget cuts you are interested in knowing how much money IPS has allocated to implementing new technology. Find the budgeted amount IPS has allocated in its Technology Plan.

As a parent of a high school junior, another parent told you that IPS has resources for you and your child to start exploring colleges. Find contact information about Sawyer College in Merrillville.

As a parent you want to identify the schools that are nearest to your home where your child would attend middle school. Your house is located at 816 N. Audubon Road, Indianapolis, IN 46219. Please name the closest high school to your house and find the enrollment information for that school.

You noticed that a school located near your house was a magnet school. Please locate the 2011-2012 English student application for this magnet school.

As a new teacher you had an issue with you pay check this week. You contacted the front office at your school, however, because they do not directly deal with payroll they directed you to the corporation website to find the appropriate contact information and the location were you needed to go to sort everything out. Identify the telephone number and location of the payroll office

7. Locate a specifically Salary information (Teacher only)

As a teacher you wanted to review updates to the 2010 teacher salary information specifically Amendment 5233. Locate information about Amendment 5233.

Entezari, Huang, Wheeler - IPS

Synthesis of Results from inspection

Entezari, Huang, Wheeler - IPS

    

a. Content
Inspection Missing and inconsistent information Target User Misidentified Consistency of information and representation When was the news new? Unclear input requirement Unclear boundary map Vague enrollment information Usability Testing

b. Navigation
Inspection Undistinguishable links Too much navigation No indication when leaving the website Provide choices without decision information Technique problematic navigation bar block navigating within the topic
Entezari, Huang, Wheeler - IPS

Usability Testing Undistinguishable links (text, & banner) User missed navigation links (Users scrolled down to the section they Lack external website alerts lead wanted vs. clicking on the shortcuts) confusion School link with vague description

c. Information Architecture
Visualization of school proximity Problem finding school according to distance Too much information accommodating Users are confused and have difficulty too many people back from the Where am I? No clues of the current location Homepage without main content (Homeless homepage) Archiving Unrelated Navigation homepage provide weak navigation assistant into the website.(user fixate on the homepage, they are sure the information can be found there) Mismatch infor mation cate gorization between user and Problem of finding items from a long the website.(users categorize list (find schools from the long school information according the closeness list, arrange according to alphabetic is of their name, but the website group recommended) according the department while the users are not familiar with the organization of departments.)

Entezari, Huang, Wheeler - IPS

d. Presentation
Users don’t know where they are Inconsistent font sizes, weights & decoration (Hyperlinks) Mixed representation of information and linked files Inconsistent design of sub-domains Deemphasizing position of important menus Error recovery Mislabeled Pages Insufficient Information Banner Blindness. (The user Overlook the navigation bar on the top. In contrast, the users can notice the so call “deemphasized” menus on the right.) The Search Bar is too small to be noticed Un-salient links

Entezari, Huang, Wheeler - IPS

New problem: # The information architecture Does not match the user’s expectation
Phenomenon: (Task 2) The users have difficulty in finding the Technology Plan.
      

Analysis: Users categorize information according the familiarity of their name. However, the website groups them according to the department while the users are not familiar with the organization of departments.
 

Entezari, Huang, Wheeler - IPS

overlap problem: # homeless homepage

especially the dashboard area to navigate them throughout the website.

Phenomenon (Task 2): Users spend a long time looking at the homepage;

Banner Blindness

Users expect to find content in quick links area

Entezari, Huang, Wheeler - IPS

overlap problem: # visualization of school proximity

Results

Entezari, Huang, Wheeler - IPS

Time on Task Analysis
Task 1 – 229 seconds
 Perceived Ease – 2.9

Task 2 – 328 seconds
 Perceived Ease – 1.7

Task 3 – 143 seconds
 Perceived Ease – 2.6

Task 4 – 266 seconds
 Perceived Ease – 2.4

Task 5 – 212 seconds
 Perceived Ease – 2.9

Task 6 – 60 seconds
 Perceived Ease – 3

Task 7 – 120 seconds
 Perceived Ease – 2.5

Entezari, Huang, Wheeler - IPS

Levels of Success Analysis (Full/Partial/Fail)
Task 1 – 7 / 2 / 2


Task 2 – 3 / 4 / 4


Task 3 – 7 / 3 / 1


Task 4 – 5 / 1 / 5


Task 5 – 7 / 2 / 2


Task 6 – 2 / 0 / 0


Task 7 – 2 / 0 / 0

Entezari, Huang, Wheeler - IPS

Quantitative Summary

Entezari, Huang, Wheeler - IPS

Quantitative Correlations
[Lostness] x [Time on Task]

+0.92 -0.93 -0.95 -0.39 -0.68 +0.66

[Lostness] x [Success Rate]

[Time on Task] x [Success Rate]

  Lostness Time on Task Success Rate Ease

Lostn Time on ess Task   0 . 92            

Success Rate

- 0 . 93 - 0 . 95
   

Ease -0.39 -0.68 0.66  

[Lostness] x [Ease]

[Time on Task] x [Ease]

[Success Rate] x [Ease]

Entezari, Huang, Wheeler - IPS

Task 1
Participants failed to recognize the main navigation
 Banner Blindness
        
  

Participants felt the need to find the answer to the scenario in the homepage due to it’s construct Ambiguousness of the word “Calendar” and the function of calendars on the site proved to be confusing Participants wanted to find school board meetings in “Parents” section as opposed to “School Board” section Users wanted the list to be ordered according to today

“As a parent you are growing concerned about the direction the IPS School board is moving towards concerning year round school. Another parent mentioned that there are school board meetings where you can go and voice your concerns. Please identify the date for the next briefing session.”


Entezari, Huang, Wheeler - IPS

Task 2
Skipped navigation of page and continued to skip content before realizing navigation was more useful Mislabeled pages create much confusion Users didn’t immediately differentiate between “Technology” and “Information Technology” There were a multitude of different places that participants expected the budget to be. Navigation in PDF document was extremely difficult and time-consuming.
        
  

“As a parent you feel that your child’s ability to learn and use technology is very important. In the midst of growing concerns about budget cuts you are interested in knowing how much money IPS has allocated to implementing new technology. Find the budgeted amount IPS has allocated in its Technology Plan.”


Entezari, Huang, Wheeler - IPS

Task 3
This task had a much better outcome than we had expected. Usability Evaluation demonstrated the difficulties in constructs for this task. Mislabeling of pages still rose some difficulties for the participants Organization of site didn’t allow for a solid mental map to be developed.
       

“As a parent of a high school junior, another parent told you that IPS has resources for you and your child to start exploring colleges. Find contact information about Sawyer College in Merrillville. “

 

Entezari, Huang, Wheeler - IPS

Task 4
Participants noticed the banner was static on some pages and dynamic on others Logical paths that participants decided on were not reinforced when attempting to enact their planto-browse for resources “Boundary Map” was not a well-fitting label for many of the participants. They expected to find this information in the “Schools” section, but didn’t It is in a very noticeable place (first link on first page) however, the participants still didn’t notice it until much later
       
   

“As a parent you want to identify the schools that are nearest to your home where your child would attend middle school. Your house is located at 816 N. Audubon Road, Indianapolis, IN 46219. Please name the closest high school to your house and find the enrollment information for that school. “

 

Entezari, Huang, Wheeler - IPS

Task 5
Users expected magnet school applications to be in schools Some participants went as far as navigating each school to find an appropriate school. Magnet schools were categorized under the “other” category and participants weren’t immediately aware of this Page-specific navigation was ignored by participants. Before identifying this, participants would skim the page in it’s entirety to see if they could find the answer to the task question.
       
   

“You noticed that a school located near your house was a magnet school. Please locate the 2011-2012 English student application for this magnet school.”

 

Entezari, Huang, Wheeler - IPS

Tasks 6 & 7
Tasks were completed by domain experts Tasks were intended for only teachers Found that participants who had experience with site were very proficient with navigating site This does not mean that these tasks are not errorridden. Users may be habituated to poor-user design. Both users found the task goals efficiently.
      
    

“As a new teacher you had an issue with you pay check this week. You contacted the front office at your school, however, because they do not directly deal with payroll they directed you to the corporation website to find the appropriate contact information and the location were you needed to go to sort everything out. Identify the telephone number and location of the payroll office” “As a teacher you wanted to review updates to the 2010 teacher salary information specifically Amendment 5233. Locate information about Amendment 5233.”

 

 

Entezari, Huang, Wheeler - IPS

recommendations

Entezari, Huang, Wheeler - IPS

recommendation
CONTENT # Search feature: Move format of address closer to the Reference|| Unclear input requirement search box. # Boundary map: Make sure the boundary map provides information that the users need. Map Reference|| Unclear Boundary # Enrollment Information: Make Enrollment as a link and add a page providing enrollment information, Reference|| Vague information including how to enroll, current enrollment status, resource of forms, etc. IN UT

recommendation
NAVIGATION
# Examine the navigation bar to make sure the links Reference||pages withiny pr oblematic navigation cover all the Technicall the topic. bar block navigating within the topic # Use proper formatting to ensure consistency between text and links, as well as banner picture and Reference|| Undistinguishable Links (text & links banner) # Keep the most necessary navigation and get rid of the ones irrelevant to the page. Make sure the appearance and layout of the navigation are salient, consistent and easily noticed. Distinguish the navigation of different levels by different hierarchical Refer ence|| Too many navigations that user s presentation. ignor e one or two

IN

UT

recommendation
INFORMATION ARCHITECTURE
# Provide a sitemap of the website to help the user Reference|| Mismatch the logical architecture of gain an understanding of infor mation cate gorization between user s and the website. the webiste. # Design should include visual cues in the navigation (Usability Testing) that highlights • Severity: 3 their current location. Possible Reference|| W her e am I? User s ar e lost solutions are breadcrumb providing the track and because ther e ar e no clues of the cur r ent # Develop an area for main content to each page has a navigate within the hierarchy. Ensure be displayed to location the user. This should summarize the site and what the title matching the pages content. Refer ence|| Homepa ge without main content site represents. pr ovides weak navigation assistant into the website.

IN

UT

recommendation
PRESENTATION
# Make the navigation bar around the banner area Reference|| Banner Blindness is avoid using the color more salient. Possible solution Severity: 3 similar to the banner for the background of navigation # Make the links more visually salient. At the same bar. Make the appearance of the navigation bar more time, put them a button. outstanding position, such as touchable likeMixed r epr esentation of Reference|| in a more on the top of the page and avoid mixing within the infor mation and linked files, Non-salient links # Provide clear recovery instruction when the error body of the content. appears. Especially to the search feature, provide Refer ence|| Er r or r ecover y & User s more options for user to choose to let themdon’t r ecognize r ather thanfr om the sear ch er r or know how to r ecover r ecall. Provide format requirement and example near the input area.

IN

UT

Questions?

Entezari, Huang, Wheeler - IPS

Sign up to vote on this title
UsefulNot useful