Talk structure: 3 techniques for better interface design

• Card Sorting – Not used to its full potential – We need an easier way to do and record data for card sorts • Persona Creation – The process for creating personas needs to be demystified. Period. • Usability Tests – Its time to stop thinking of this monolithic entity called “usability tests”. There are many types of usability tests with different levels of complexity. – Tests for competitive benchmarking or for iterative design process can borrow from experimental design techniques.

Cost-effective methods for rapid user research and usability testing

Rashmi Sinha
www.rashmisinha.com www.uzanto.com

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

Understanding your users
• Finding out what issues are relevant

Understanding the categorical structures of the mind
• Card sorting is one of a family of techniques to

to them • Exploring their mental models
• Methods
– – – –

tap into the user’s categorical structures. Animal Birds
(wings, flies, feathers) (has skin, can move, eats)

Fish
(swims, gills, fins)

Interview Observation Ethnographic Methods Card Sorting & other methods of exploring categorization – Persona Creation Focus on methods for information rich sites

Canary
(sings, yellow)

Ostrich
(tall, does not fly)

Salmon
(edible, pink)

Shark
(dangerous, gray)

• How semantic memory is organized
Semantic and categorical refer to similar things in present cont ext

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

1

Why you need to understand categorical structures?
• Semantic organization builds up user

Example of individual differences
Sushi Shark Aquarium

expectancies regarding how things are organized.
– Design in accordance to these expectancies for better information architecture – Violate these expectancies and you leave the user lost and confused

Fish

Diving

• Do people’s categorical structures differ

Fish
Shark
• Length of arrows implies semantic distance

Grilled Salmon

from each other?

– Yes, semantic networks are determined by one’s experience, learning, culture etc. – but there is enough consistency across people for effective design

• Semantic memory is a network. Points are connected in

multiple direct and indirect ways.

• Degree of closeness between items might vary between

people, but we recognize the idiosyncratic association others make
BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

Semantic Distance and Proximity / Similarity / Distance Matrix
• How similar are the two (on scale of 1-5)

Semantic distance as basis of categorization
• At root of all categorization techniques is

– salmon & shark:

1 1

2 2

3 3

4 4

5
Very similar

question: “How far is A from B?”

Not similar Not similar

– canary & ostrich: Self correlation
Salmon Shark Canary Ostrich Trout Tiger Robin Salmon Shark 5 3 4 5 2 5 5

• Proximity / similarity matrix can help generate

5
Very similar

the most complex semantic networks (by using cluster analysis and other statistical techniques).
Animal

Canary Ostrich Trout 5 4 3 2 3 2

Tiger

Robin
(has skin, can move, eats)

5 2 4 4 1

Birds

Fish
(swims, gills, fins)

5 4 4 2

(wings, flies, feathers)

5 5 4

5 5

Canary

Ostrich
(tall, does not fly)

Salmon
(edible, pink)

Shark
(dangerous, gray)

5

(sings, yellow)

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

2

Methods to tap into categorical structures
• Open Card Sorting for generating information

Card Sorting: Understanding user’s mental models
• Why: Helps you build site structure
-good for deciding how to group information

architecture architecture

• Closed Card Sorting for verifying information • Generating semantic associations • Identifying dimensions used in categorization

• How: Write down each topic on an index card
– Ask users to sort cards into piles – Have them name each category later

• Data obtained from card -sorting is indirect

enough that it avoids pitfalls of direct questioning

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

Kind of questions card sorting helps answer
• Do the users want to see the information grouped

Open and closed card sorting
• Open Card Sorting: users given bunch of items to

by: subject, process, business group, or type of information? main menu?

sort those into categories. They are not given name or number of categories.
– Useful for generating new or drastically redefining information architecture

• What are the most important items to put on the • How many menu items should there be, and how

• Closed Card Sorting:users given items to place

deep should it go?

into pre -defined categories.

• How similar or different are the needs of the

users throughout the organization?

– Useful for verifying pre -existing information architecture – Useful for verifying category labels

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

3

Conducting card-sorting exercises
• Rather involved method, requires a lot of setup.

Card sorting for designing an online travel guide
• Example: Designing an online

Sorts themselves are fun to conduct.

• Main problem is in getting data into any form

travel guide to help users plan their trips to various locations.
– Lots and lots of items. How to structure the information.

where one can easily look at it, or share information with design team.
– IBM EZ Sort – NIST WebCAT

• Purpose of card sort:

• Software for conducting card sorts.

• Items include
– lodging, entertainment, local information, When to Go, Travel by Car/Air/Bus, Music Events, Hiking, Day Trips, Skiing, Diving, Golf, Emergency Info.

• Alternative method: Using online survey software

to conduct card sorts

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

Conducting card sorting exercises
• Check to make sure that
– – – – each term is clear and unambiguous all the items that need to categorized were included set up items to be presented in random order create comprehensive instructions so that all participants have the same understanding of the process – Participants should be left alone during the sorting to avoid placing them under time pressure, but tester should remain available for answering questions

Card sorting using online survey software
Each item (card) occupies a row Each potential (unnamed category) is represented by a column User indicates category membership by checking box

• Conducting card sorting with online survey

software

– Most online survey software will do – Make sure it allowed you to present questions in matrix format, and allows downloading of each user’s data separately.
www.rashmisinha.com www.rashmisinha.com

BayCHI UE BOF, July 02

BayCHI UE BOF, July 02

4

Adding labels to categories
At end, ask users to assign names / labels to each category they created Some users like to do this along the way (noting labels on piece of paper)

Looking at card-sorting data
Cat-1 Cat-2 Cat-3 Cat-4 Cat-5 Look at data from Name Category Entertainment Outdoor Lodging Practical Culture one user at a time Info Before you go Before you go Fast Food At a glance you can Duties/Customs look down a Duties & Customs particular column, DIving DIving and see what items Nightclubs Nightclubs belong in it. ATMS's ATMS's Emergency info Emergency info You can also draw Etiquette Etiquette hierarchical Day Trips Day Trips structures showing Museums & user created Galleries Museums & GalleriesMuseums etc. categories Films Films Films & Movies Gay/Lesbian Hangouts Highlights Highlights Statistical analysis Highlights can Architectural Sites Architectural Sites create these

pictures for you across users.

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

Closed Card Sorts
Category labels have been provided

Closed card sorts: when & how
• Category names were provided. • Users simply asked to assign items to categories. • Results are not informative about whether

number and structuring of categories was appropriate. response.

• Generally users are asked to give a “Yes / No”
– One spin on this method is to ask users to rate “degree of certainty” in assigning items to categories. – This will help highlight problem categories / items. If users are very uncertain about assigning an item to a category, that should be probed further.

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

5

Rating “degree of certainty” in closed card sorts

Advantages of using online survey software for card sorts
• Easy to carry out, no software to install and

figure out. Can be done using online survey software that are easily available.

• Can be done remotely • Trivial to get user-created categories into Excel

(or your favorite application) for qualitative / quantitative analysis. sorts

• Can be used to conduct either open / closed card • Can also be adapted to individual / group

sessions

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

Other issues about card sorting
• Should each item be in one category • Should card sorts be done individually or with

Other methods to tap into categorical structures
• Open Card Sorting for generating information

architecture architecture

groups

• Closed Card Sorting for verifying information • Generating Semantic Associations • Identifying dimensions used in categorization

– group card sorts provides an energy and enthusiasm that makes light work of an otherwise tedious task – Can overshadow individual differences highlighting opinion of a few.

• Combining card sorting with importance ratings is

a good idea.

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

6

Generating semantic associations
• Helps understand what lies nearest to target items. Can be

Generating semantic associations…
• Category Member Association
– What member items would you expect to find in below categories – Music: _____ _____ _____ ______ _____ _____ _____ _____ _____ ______ _____ _____ – Diving: _____ _____ _____ ______ _____ _____ _____ _____ _____ ______ _____ _____

conducted in conjunction with or independently of card sorting.

• Helpful in understanding problem categories and items • Semantic Association – What words do you associate below target words with Music: _____ _____ _____ ______ _____ _____ Diving: _____ _____ _____ ______ _____ _____ • Category Association – What categories would you expect to find below words in Music: _____ _____ _____ ______ Diving: _____ _____ _____ ______
www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

Identifying dimensions for categorizing
• Present two random items and ask user what differentiates

Talk structure: Personas…
• Card Sorting – Not used to its full potential – We need an easier way to do card sorts

them in your domain context. Repeat question till dimensions start repeating frequently.

• What differentiates pair below. List one or more ways that

they are different.

• Persona Creation
– The process for creating personas needs to be demystified. Period.
• Usability Tests – Its time to stop thinking of this monolithic entity called “usability tests”. There are different types of “usability tests”. – Tests for competitive benchmarking or for iterative design process can borrow from experimental design techniques.

– Diving & Live Music Event: _______ ______ ______ – Shopping & Air Tickets: _______ ______ ______ – Motels & Restaurants: _______ ______ ______ • Good way to explore domain you are unfamiliar with • Some of the pairs will sound ridiculous!

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

7

Personas as “User Archetypes”
• Personas is designing for one person. • Personas is a “User Archetype” who
– Who should that one person be?

Creating personas
• Method – Conduct interviews with various stakeholders – Find patterns (inclusive cases) – Pick a nugget, interesting tidbit and build persona around it – Should represent large group in terms of goals, and personal characteristics • Problems with method: – Interviews are not economical way to find who representative users are – No tight coupling between user research & personas – Would two designers creating personas for same product create same personas? • Bottom line: Cooper’s persona creation method works well

represents needs and goals of many other users.

• According to Cooper
– Not the average user – Not a real user – Derived from background user research (interviews etc.)

when you have time on your hand, resources to interview many people, and skilled persona developers.
BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

What is needed (from the horse’s mouth)
“What is needed is a product definition tool that takes the guesswork out of the process—a tool that provides insight into what motivates people to use a product, so that well-grounded decisions can be made about features and how they are presented.”
– Cooper.com

Will market segmenting techniques serve any purpose?
• Market segmentation is used to identify

clusters of people product can appeal to

– Mostly on basis of demographics – Sometimes on the basis of psychological variables (also called psychographic variables).

• Techniques can forecast marketplace

acceptance of products and services. Can also help convince executives to build product.

• Techniques not helpful in defining product

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

8

Deconstructing marketing techniques
• Questions focus on like / dislike of product

What is needed is a method that will…
• Ground the personas in reality
– Common critique of personas is that they are based on the designer’s imagination

concept

– what do you think of vanilla coke or green Heinz ketchup?

• Focus on motives/needs of users • Be complementary to existing methods e.g.,

• Statistical techniques used in market

segmentation cluster users according to demographic variables, not according to user needs from products.

interview, observation

• Develop a concrete series of steps that any

designer can follow to develop personas

• Concern themselves with product at a high

• Be reliable. When used by other designers, at

conceptual level. For interaction design, one needs to focus on specifics: How will product satisfy needs / goals.
BayCHI UE BOF, July 02

other times will lead to similar personas.

www.rashmisinha.com

www.rashmisinha.com

BayCHI UE BOF, July 02

Challenge: How to reliably identify user archetypes in cost-effective manner
• Solution:
– Use survey techniques that are also used in market segmentation – Focus questions around user needs rather than what they simply like / dislike – Identify constellations of needs rather than clusters of users – Use this information as the kernel to build personas around

Designing personas for a Bay Area restaurant finder
• Goals of site: Be the premier site for Bay Area

residents & visitors to find restaurants of their choice. advisors) apart from conventional advanced search / browse options restaurants

• Design quick restaurant-finders (a la product

• Conducted survey on features are used in finding
– 45 respondents answered survey about importance of features in choosing restaurants – Food quality, décor, wine selection, cuisine, service

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

9

Steps in process
• Collect information via survey, interview,

Design of survey
• Broad and shallow look • Try to be comprehensive • Identify features and get their importance rated

observation

• Analyze survey data to identify underlying

factors, (types of needs)

on same scale (5 or 7 point Likert scales)

• Identify some user archetypes • Verify these user archetypes with your qualitative

– Example: How important are following restaurant features 1 2 3 4 5 – Service:
Not similar Very similar

data (information from interview, observation etc.). personas

• Continue with your usual process of creating

• Don’t frame question in abstract way, instead tie

it to actual behavior

• Assure respondents about privacy, ask if they will

participate in follow-up interview

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

The survey: Importance ratings of 25 restaurant features

Main results of survey (based on 39 respondents)
• Most respondents thinks quality of food, type of

cuisine, cleanliness is very important

• Few respondents think valet parking is important • Mixed feelings about features such buffet,

location, wine selection, waiting in lines etc. as a couple. Fewer with family. Even fewer alone.

• Most report going to restaurants with friends, or • Most report going 2 -3 times a week.

Next step: Identifying user archetypes based on patterns of needs…
www.rashmisinha.com
BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

10

What would marketing techniques do?
User ID
5240744 5240760 5241210 5241375 5241496 5241562 5243833 5243926 5243943 5245631 5253253 5261459 5263121 5264284 5264327 5264503 5264616 5264685 5264820 5264925 5265174

Design techniques need to identify cluster of needs
User ID Food Quality
3 5 4 5 5 5 5 5 5 5 5 5 5 3 5 4 3 5 4 5 5

Food Quality
3 5 4 5 5 5 5 5 5 5 5 5 5 3 5 4 3 5 4 5 5

Décor

R e s t a u r a n t F e a t u r e s Wine ChildOutdoor Romanti Selectio Buffet friendly seating c n
3 5 1 3 4 4 3 4 5 3 4 3 2 3 2 3 2 3 4 3 1 1 2 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 5 3 1 1 1 5 4 3 5 5 4 5 1 2 2 4 1 2 1 5 3 4 5 5 5 5 5 3 5 5 5 4 4 2 3 5 4 3 5 4 4 3 2 3 1 5 3 4 1 1 3 1 1 1 2 2 4 2 1 2 5 2 1

Service
1 1 1 3 1 1 3 2 1 3 1 2 2 2 1 1 2 1 1 2 1 1 2 1 2 1 1 4 3 5 5 1 3 3 1 1 1 1 1 1 1 1

Décor

Cluster A

Cluster B

5240744 5240760 5241210 5241375 5241496 5241562 5243833 5243926 5243943 5245631 5253253 5261459 5263121 5264284 5264327 5264503 5264616 5264685 5264820 5264925 5265174

R e s t a u r a n t F e a t u r e s Wine ChildOutdoor Romanti Selectio Buffet friendly seating c n
3 5 1 3 4 4 3 4 5 3 4 3 2 3 2 3 2 3 4 3 1 1 2 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 5 3 1 1 1 5 4 3 5 5 4 5 1 2 2 4 1 2 1 5 3 4 5 5 5 5 5 3 5 5 5 4 4 2 3 5 4 3 5 4 4 3 2 3 1 5 3 4 1 1 3 1 1 1 2 2 4 2 1 2 5 2 1

Service
1 1 1 3 1 1 3 2 1 3 1 2 2 2 1 1 2 1 1 2 1 1 2 1 2 1 1 4 3 5 5 1 3 3 1 1 1 1 1 1 1 1

Cluster A
www.rashmisinha.com
BayCHI UE BOF, July 02

Cluster B
BayCHI UE BOF, July 02

www.rashmisinha.com

From cluster of needs to user archetypes
• Features that are highly related to each other will

User Archetype 1: Romantic, relaxed, dining experience…
• Important Features – Good decor – Romantic setting – Relaxed service – Trendy – Caters to special Requirements (Low - fat, vegan etc.) – Type of crowd • Not Important Features – price, buffet-style – Many in 42- 51 age- group, spend more than $20, go out as a couple Food Quality, Type of Cuisine & Cleanliness does not appear in these archetypes because it was important to everyone.

form part of the same constellation, or group of needs groupings. type.

• Factor analysis techniques can identify such • Each grouping can be said to represent a need • These are groupings of features not of users.

What need types emerge from restaurant survey?

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

11

User Archetype 2: Buffet style, with kids in tow…
• Important Features
– – – – – – Buffet style dining Good for groups Child Friendly Relaxed Service Credit Card Portion Size

User Archetype 3: Meet for drinks, eat some food too…
• Important Features
– – – – – – Wine selection & bar Outdoor seating Easy access to public transport Live entertainment Type of crowd that visits place Good for groups

• Not Important Features
– views, outdoor dining, people- watching, bar

• Not Important Features
– long Lines, finding Parking nearby

• Many in age group 32 -41, dine mostly with

family, once a week or lesser

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

After establishing some user archetypes
• Use user archetypes as “candidate personas” • Verify archetypes with results of interviews,

Advantages of technique
• Inexpensive in terms of time and expertise • A reliable method, two designers using same

observations

method are likely to get similar results

• If survey is first step, then these can guide

interviews (interview examples of each user archetype) on primary and secondary personas according to your design goals survey as final word. Use them in conjunction with other information.
BayCHI UE BOF, July 02

• Retains creativity and fictional elements of

creating personas, while grounding them in real data generating personas.

• Generate sample scenarios, make final decision

• Is complementary to current process of

• Do not treat user archetypes generated through

www.rashmisinha.com

www.rashmisinha.com

BayCHI UE BOF, July 02

12

Talk structure: designing usability tests
• Card Sorting
– Not used to its full potential – We need an easier way to do card sorts

Usability tests for iterative design process & comparative benchmarking
• Both scenarios require testing of more than one

product

• Iterative Design requires multiple tests of same

software.

• Persona Creation
– The process for creating personas needs to be demystified. Period.

– It also calls for creation of norms, baselines to compare subsequent versions against.

• Competitive Benchmarking calls for comparing

one product with other similar ones

• Usability Tests – Tests for competitive benchmarking or for iterative design can borrow from experimental design techniques.
www.rashmisinha.com
BayCHI UE BOF, July 02

• When comparing one product with another, test

becomes complex enough to benefit from experimental design.

www.rashmisinha.com

BayCHI UE BOF, July 02

Not an experiment, has some features of it. Why?
• So that one can make generalizable conclusions • Thinking about study design insures that potential

6 steps to designing good usability tests

1: Identify the goal of the usability test; write it down 2: Identify factor you are most interested in (Independent Variable) 3: What changes do you expect in test results (Dependent Variabl es) 4: What random factors can influence results (Controlled Variables) 5: Decide on a within-user / between -users study design. Take care of learning & fatigue effects 6: Make sure test design is practical (rapid and cost -effective), and gives you the results you need. Else redesign test.

problems are identified beforehand, rather than later
• Study designs are adaptable, 3-4 study designs can

be adapted for most testing situations
• Helps make best use of limited time and resources • Helps reduce number of users

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

13

Example study: Examine usability of AskJeeves Digital Camera Advisor
• Step 1: Identify goal: – To find effectiveness of Jeeves digital camera advisor • Method:

Step 2: Identify Independent Variables
• is the factor that undergoes a change • “type of camera advisor” is Independent Variable with three

levels

Compare effectiveness of Jeeves camera advisor to two popular camera advisors

• MySimon, ActiveDecisions, & AskJeeves Independent Variable: Type of Camera Advisor Jeeves Camera Advisor MySimon Camera Advisor Active Decisions Camera Advisor

• In Iterative Design Process, “Version” is your Independent

Variable. Different versions (e.g., version 1, Version 1.5, Version 2) are your levels of Independent Variable

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

What if there is no Independent Variable?
If you are interested in the usability of a particular software only…

Step 3: Identify dependent variables and decide how to measure each
• These are the “things” you are measuring:

satisfaction, time to task, errors. The expectation is that they will change with changes in Independent Variable

Identify a relevant benchmark, even if benchmark is theoretical ideal (i.e., make one up)
– the ideal system – older system that current software is replacing – competing software / process in the market

• Possible Dependent Variables:
– Satisfaction: User comments, satisfaction questionnaire – Task success: time taken, number and type of errors

Comparison with a relevant benchmark makes tests results more compelling and easier to interpret.

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

14

Step 4: What controlled variables could inadvertently influence results
• Random factors that can effect results (need to control

Review types of variables: Independent, Dependent & Controlled
• Independent Variable: Is the factor that

them). E.g., network speed, testing conditions (light, monitor size), fatigue of subjects. of current interest

undergoes a change, the factor you are interested in.
– Two versions of a website, three authentication systems

• These are like Independent Variables, except they are not • One needs to rule out effects of these variables by

• Dependent Variable: Is the outcome of the test,

controlling them (equalizing their effect in different conditions).

the thing that is effected by the independent variable

– No of errors in finding item, time to get credit card authenticated by system

• Controlled variables in current study – Testing Conditions: light, network speed, monitor size – Instructions given: whether few/ detailed / no instructions – User Variables: age, gender, previous web & computer experience

• Controlled Variables: factors that can cause

random changes in test results. Need to be controlled

– Speed of network connection, Previous web experience of user

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

Step 5: Within or between user design
• Within user design: each user participates in all conditions. – Advantage: do not need to control for user differences – Disadvantage: Learning and Fatigue effects • Between-user design: each user participates in one condition. – Advantage: No learning and fatigue effects – Disadvantage: Need to control for between user differences • Practical decision – Depends on number of users you can afford, length of test etc. – Determines how easy it is to interpret test results, and how far one can generalize from test results
2 diff. designs

Within user study design
9 users (A-I), within-users design
Users A B C D E F G H I Order 1 MySimon Jeeves Actv.Decs MySimon Jeeves Actv.Decs MySimon Jeeves Actv.Decs Order 2 Jeeves Actv.Decs MySimon Jeeves Actv.Decs MySimon Jeeves Actv.Decs MySimon Order 3 Actv.Decs MySimon Jeeves Actv.Decs MySimon Jeeves Actv.Decs MySimon Jeeves

•Each system occurs 3 times in Order 1,2, & 3 •Study design takes care of learning, fatigue effects •Design is very efficient, but there might be learning/fatigue effects

Independent Var.: Type of Camera Advisor
Jeeves Camera MySimon Camera Advisor Advisor Users A,B,C Users A,B,C Users D,E,F Act. Dec. Camera Advisor Users: A,B,C Users: G, H, I

Within-Users design (3 users) Between-Users design (9 Users A,B,C users) www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

15

Between users study design
• 9 users (A-I), between-users design
Users A B C D E F G H I Order 1 MySimon Jeeves Actv.Decs MySimon Jeeves Actv.Decs MySimon Jeeves Actv.Decs

Mixed Study Design
• 9 users (A-I), mixed between-users design
Users A B Order 1 MySimon Jeeves Actv.Decs MySimon Jeeves Actv.Decs MySimon Jeeves Actv.Decs Order 2 Jeeves Actv.Decs MySimon Jeeves Actv.Decs MySimon Jeeves Actv.Decs MySimon

•Each system is used by 3 users, each user only uses 1 system •Study design does not pose any learning, fatigue effects •Tricky issue is to make sure users for different systems are comparable •Design is not very efficient, but easy to interpret results (if users are comparable)

•Each system is used by 6 users, each user only uses 2 system •Each system is used 3 times in Order 1 and 3 times in order 2 •Design compromises between demands of efficiency and fatigue/learning effects •Tricky issue is to make sure users for different systems are comparable

C D E F G H I

www.rashmisinha.com

BayCHI UE BOF, July 02

www.rashmisinha.com

BayCHI UE BOF, July 02

For more information go to…

www.rashmisinha.com www.uzanto.com Or email me at rashmi@rashmisinha.com

www.rashmisinha.com

BayCHI UE BOF, July 02

16

Sign up to vote on this title
UsefulNot useful