Testing the Usability of a New User Interface in JMP Custom Design

Mark Bailey, Jacob Rhyne, Joseph Morgan, Ryan Lekivetz

JMP Statistical Discovery, Cary NC 27513, USA

Motivation: User Pain

Adding factors is an important first step in designing an

experiment in the typical workflow.

Users can have difficulty with common tasks:

• Confusion about adding, deleting, and renaming factors Figure 1. Design Workflow

• Difficulty changing factor type, number of factor levels, or

the order of factors
• Confusion about which type of factor to use

Goals for prototype to be tested:

• Decrease confusion
• Reduce the number of operations Figure 2. Original User Interface for Factor Description
• Reduce time to achieve the desired result
• Usability study will compare different controls that might
achieve the goals.
 Testing the Usability of a New User Interface in JMP Custom Design
Mark Bailey, Jacob Rhyne, Joseph Morgan, Ryan Lekivetz
JMP Statistical Discovery, Cary NC 27513, USA

Methodology: Flexible Prototype

The prototype may be configured in many ways by turning

several user interface controls on and off.
• Configurability leads to 384 possible configurations.
• A practical usability study required a smaller optimal
design
Figure 3. Configurable Prototype of New User Interface for Factor Description

The options for the user interface include:

1. Trash icon
2. Delete icon
3. Plus button
4. Undo / Redo buttons
5. Add N Factors control
6. Buffet Menu button
Figure 4. Two Example Configurations
 Testing the Usability of a New User Interface in JMP Custom Design
Mark Bailey, Jacob Rhyne, Joseph Morgan, Ryan Lekivetz
JMP Statistical Discovery, Cary NC 27513, USA

Methodology: Design of Experiments

Used Custom Design in JMP to make runs for the usability study.
• Some UI controls are treated as binary factors (on/off)
• Other UI controls have more than two states
• User asked to perform several different tasks repeatedly
with different configurations.

Name Levels Description

Pin Column On, Off Row icon (in)visible
Trash Icon On, Off Trash icon (in)visible
Delete Icon On, Off Delete icon (in)visible
Buffet Icon On, Off Buffet icon (in)visible
Toolbar Type (+/–), Add N Factors, Both Sets controls to add factors
Pin Column Type Pin, Oval, Copy, Oval+Copy Set row icons
Row States On, Off Sets row highlighting
Task Delete, Change Type, Change Position Sets tasks for subjects
Figure 5. Custom Design Specification for Usability Study
 Testing the Usability of a New User Interface in JMP Custom Design
Mark Bailey, Jacob Rhyne, Joseph Morgan, Ryan Lekivetz
JMP Statistical Discovery, Cary NC 27513, USA

Methodology: Design of Experiments

The custom design was combined with special runs. Developed custom JSL application for unmoderated test.
• Special tasks complicated and not repeated • Each row in the design presents an exercise and prototype
• Last two runs contrast the current and proposed UI • Exercises include instructions for the task
• Participant and Run Order are included as covariates

Figure 6. All Runs in a Usability Study for One Subject

Figure 7. A Run Determines Presentation of Exercise and Prototype
 Testing the Usability of a New User Interface in JMP Custom Design
Mark Bailey, Jacob Rhyne, Joseph Morgan, Ryan Lekivetz
JMP Statistical Discovery, Cary NC 27513, USA

Qualitative Results: Preferences

The study survey revealed a range of user preferences.

• Plus button over Add N Factors control
• Delete icon over Trash icon
• Pin icon over Oval icon
• Prototype over existing interface

Figure 8. Preferences from Integrated Questionnaire

 Testing the Usability of a New User Interface in JMP Custom Design
Mark Bailey, Jacob Rhyne, Joseph Morgan, Ryan Lekivetz
JMP Statistical Discovery, Cary NC 27513, USA

Quantitative Results: Change Factor Order

Important factors Levels that

minimized Time
Pin Column Type (Icon Choice) Oval
Trash Icon On
Row States On
Run Order NA (learning not
important)
Figure 9. Regression Analysis and Response Profiles for Changing Factor Order

Figure 10. Subjects Preferred the Pin Icon over the Oval Icon, or
They Had No Preference.
 Testing the Usability of a New User Interface in JMP Custom Design
Mark Bailey, Jacob Rhyne, Joseph Morgan, Ryan Lekivetz
JMP Statistical Discovery, Cary NC 27513, USA

Quantitative Results: Deleting Factors

Important factors Levels that minimized Time

Remove Type Delete Icon
Run Order Time decreased over the runs (Subject
learning)

Figure 11. Regression Analysis and Response Profiles for Deleting Factors

Figure 12. Subjects Preferred the Delete Icon over the Trash Can
 Testing the Usability of a New User Interface in JMP Custom Design
Mark Bailey, Jacob Rhyne, Joseph Morgan, Ryan Lekivetz
JMP Statistical Discovery, Cary NC 27513, USA

Overall Result: Current Custom Design versus New Prototype

Figure 13. Time to Complete the Same Set of Tasks is Much Less Using the New Prototype