rin Services SI 622: Evaluation of Systems and he at ,C ANN ARBOR.

COM USABILITY rg Ga a TESTING REPORT rim Ga Team Newsies: u, F Danlu Fu, Garima Garg lu an Catherine Le, Caroline Moran D April 2010 ©

,C Le e

ro a

M ine l

o

an r

• Who is …

©

D

nlu a

u, F

rim Ga

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

o

an r

©

D

nlu a

u, F

rim Ga

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

o

an r

Why Usability Testing?
• To understand real users
– How they interact with the system e, C L – How they feel about the systemne

ro a

M ine l

o

an r

at ,C • Experts know too much rg Ga a rim to fix problem after launch • Too expensive Ga u, F nlu Da Adapted From lecture slides ©

ri he

Usability Testing

rim METHODS Ga u, F nlu Da ©

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

o

an r

Participant Selection
• Target Participants
– Age range: 20 - 30 years old – Novice users

• Recruiting

– 5 participants – Our personalinetworks, our colleagues and graduate rm Ga students at other schools u, F – Incentives: $25 gift cards lu

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

o

an r

©

an D

Process
• When, Where, Who and What
– – – –

April 6 and April 7, 2010 ,C Le Ann Arbor.com office conference e in room er1 tester 1 moderator, 1 note-taker and th Ca Laptops with a trial version of Camtasia Studio ,

ro a

M ine l

o

an r

©

D

nlu a

u, F

rim Ga

rg Ga a

Process
• During the testing
– – – – Introduction Pre-questionnaire 5.5 Tasks Debriefing

©

D

nlu a

u, F

rim Ga

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

o

an r

Process
• Tasks

– Task 1: Register a new account ,C Le it on Facebook – Task 2: Find a specific article and e inshare er area – Task 3: Find a crime report tin your h
• 3a: [if user does not use the Neighborhoods section] Ca rg, the Neighborhoods section Find the crime report in a

ro a

M ine l

o

an r

G achoose a concert – Task 4: Find and rim promotions – Task 5: Find some Ga u, F nlu Da ©

Results and Analysis

Jakob Nielsen’s (1993) rating scale for usability ro Ca each key problems to rank the severitye, of L finding: ne

M ine l

o

an r

©

D

nlu a

u, F

rim Ga

rg Ga a

at ,C

ri he

Usability Testing

rim KEY FINDINGS Ga u, F nlu Da ©

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

o

an r

4

Finding 1
L e

• Neighborhoods Section does not display ro Ca correctly in all browsers e,

M ine l

o

an r

©

D

IE 8.0 browser

nlu a

u, F

rim Ga

rg Ga a

at ,C

rin he

Firefox 3.6.2 browser on the same computer

4

Finding 2

• Users have difficulty navigating within the ro Ca Events Calendar e,

M ine l

o

an r

L an event – 3 out of 5 users failed the task to e in find er – Most users do not notice that there is an actual th calendar Ca rg, – Most users did not navigate using the links at the top Ga section a of the Events Calendar rim an article about an event through – Users often found Ga the Entertainment section u, F nlu Da ©

Finding 2
,C Le e ro a

M ine l

o

an r

©

D

nlu a

u, F

rim Ga

rg Ga a

at ,C

rin he

3

Finding 3

• In the Neighborhoods Section, users are ro Ca unclear as to which geographice, region of Ann L Arbor is the most appropriate choice ne

M ine l

o

an r

i rNeighborhoods tab – All users who navigated to the he at paused and expressed confusion about the region ,C rg labels Ga a rim Ga u, F nlu Da ©

Finding 3

• It was equally intuitive for users to seek the ro Ca neighborhood crime information, under the Le News sub-section of the drop-down menu. ne

M ine l

o

an r

riArbor-wide crime – Unfortunately, it shows an Ann he at report rather than the neighborhood-specific crime ,C rg report. Ga a rim Ga u, F nlu Da ©

2

Finding 4

• Hard to differentiate and find Ann Arbor cityro Ca wide news versus neighborhood-specific e, L news ne

M ine l

o

an r

ri – While searching for neighborhood-specific crime news, he atArbor-wide crime report some users found the Ann ,C rg whereas others found a more neighborhood-specific Ga crime report. a m – There is noari to access the neighborhood-specific way ,G crime report from the Crime/News section and there is Futo access the Ann Arbor-wide crime report no lway n u each Neighborhoods section. a Dfrom ©

1

Finding 5

• Users have difficulty finding the Events ro Ca Calendar e,

M ine l

o

an r

L section first to – Users often went to the Entertainment ine look for local events such ashconcerts er at the right side of the – The link is actually located on ,C rg drop-down menu, but not very visible. Ga a rim Ga u, F nlu Da ©

Usability Testing

rim RECOMMENDATIONS Ga u, F nlu Da ©

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

o

an r

Recommendation 1
• Run browser compatibility testing ro a

M ine l

o

an r

– Further testing should be done to ensure that all ,C Le consistent across features and are compatible withe n and ri all common browsers and platforms. e

©

D

nlu a

u, F

rim Ga

rg Ga a

th Ca ,

Recommendation 2

• Increase visibility of the map of Ann o r Arbor Ca area for the Neighborhoods section e,

M ine l

o

an r

L corresponding – A map of Ann Arbor overlaid with the ine the Neighborhoods geographic labels is displayed once er th feedback should be section is reached, but more Ca provided at the onset. , rg Ga or map in the drop-down menu – Either a zip code query a would alleviate the confusion users face when trying to rim Ga determine which geographic label link is the most u, for them to click. F relevant nlu Da ©

Recommendation 3

• Include an Events Calendar link in the ro Ca Entertainment section e,

M ine l

o

an r

L association of – There seems to be a strong and ne i intuitive Events (such as concerts) with rthe Entertainment he at section ,C rg – One such example being Boston's Ga the Weekly a alternative news source, rim Dig (http://www.weeklydig.com) Ga u, F nlu Da ©

Recommendation 4

• To focus attention, organize eventsro the by Ca week's most highly featured recommendations e,

M ine l

o

an r

L of events are – In LA Weekly’s calendar, only a handful ine considered staff picks each week. er th Ca rg, Ga a rim Ga u, F nlu Da ©

Recommendation 5

• Add more cues to describe events at a glance ro

Cabut no legend – Icons to the left describe the event type, e, L is given. ine as "Concert”, – Describe the event type in tags, such er th who are not able to “Folk”, or “the Ark", for those Ca g, determine the eventrtype from titles. Ga a rim Ga u, F nlu Da ©

M ine l

o

an r

Recommendation 6

• Increase consistency of access to articles ro Ca across site e,

M ine l

o

an r

Lroute or another, – It is equally intuitive to choose one ne ior the Neighborhoods such as choosing the Crime her tab at tab when looking to find reports about neighborhood ,C crime. rg Ga – If articles are cross-linked in multiple related suba sections, initial mistakes in information-seeking can be rim Ga there will be a safety net of multiple minimized since u, routes to access the same information. F modes and nlu Da ©

Usability Testing

rim DISCUSSION Ga u, F nlu Da ©

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

o

an r

M ine • 5 participants who have never visited ol ar AnnArbor.com ,C Le – In real cases new users probably won’t use these ine very first visit specific functions in our tasks on their er th – During their navigation through news articles, they Ca related sections such as might notice some non-news g, rwhich might further indicate what the Events Calendar, GaAnnArbor.com. else they can doaon rim • Further Research Ga u, – Recruiting some regular users who never used F functions other than reading the news nlu Da ©

New Users

o

an r

Unfamiliarity with the city of oran M Ann Arbor ine l
• All of the participants came to Ann Arbor ,C Le within the past year e

ro a

rin familiar with the – 2 participants said that they are not he because they are area distribution of Ann Arbor at ,C newcomers here rg Ga a • Further Research rim Ga local young participants so as to – Recruit several u, the problem of area location for F evaluate nlu Neighborhoods section Da ©

Task Wording

• Many participants clicked Entertainment ro Ca concert instead of the Events Calendareto find , L events ne

M ine l

o

an r

ri of events which are – It might be wise to test othere kinds th menu so as to eliminate not related to any tab on a C top rg, as much as possible. the impact of otheraoptions aG rim • Further Research Ga u, tasks such as finding a workshop can be used F – Example nlu to replace the concert task Da ©

Usability Testing

rim CONCLUSION Ga u, F nlu Da ©

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

o

an r

• Findings:

ro – Participants have difficulty finding their neighborhood Ca within Ann Arbor. e, L – They also have difficulty with understanding whether ine er they should go to the Entertainment or the Events th Calendar section for concert information. Ca , rgEvents Calendar section didn’t – Navigation within the Ga a work as users expected. rim news is only in the main News – Ann Arbor-wide Ga category, while related neighborhood-specific news is u, F only available in Neighborhood sections. nlu Da

M ine l

o

an r

©

• Recommendations:

– Unambiguous geographic identification in o ar neighborhood sections. ,C Levisible to users – The Events Calendar should be more ine such as and integrated with other sections er Entertainment. th Ca within Events Calendar – Better and clearer navigation rg, so as to provide flexible should also be supported Ga paths to desiredaevents. rim – IncreasedGa consistency across the general News sections and Neighborhood sections should be considered for u, F future improvements. lu

M ine l

o

an r

©

an D

©

D

nlu a

,G Fu

,C L Thank you!e ine er Q &th aA ,C rg Ga a rim a

ro a

M ine l

o

an r

Sign up to vote on this title
UsefulNot useful