Transforming Complex

Laboratory Data into Holly Witteman, PhD

Meaningful Information Labquality Days 2020

for Patients and Families

Native-land.ca Eduardo Fonseco Arraes, CC BY-NC-SA 3.0

@hwitteman @wittemanlab Martin St-Amant (S23678) CC BY-SA 3.0

Declarations
I declare I have no known conflicts of interest
Funding: PI: CIHR, PCORI, FRQS, CFI, MERSTQ
Salary support: FRQS J1, J2, Canada Research Chair in Human-Centred
Digital Health
Images: CC-BY WittemanLab, CC0, exceptions noted

@hwitteman @wittemanlab
Challenge: Guess the decade
“The clinical interpretations of a laboratory finding should be given only by
a registered medical practitioner.”

@hwitteman @wittemanlab
Challenge: Guess the decade
“The clinical interpretations of a laboratory finding should be given only by
a registered medical practitioner.”

“Direct notification of abnormal test results … would result in patient

anxiety and confusion … patients lack the necessary expertise to interpret
test results.”

@hwitteman @wittemanlab
Challenge: Guess the decade
“The clinical interpretations of a laboratory finding should be given only by
a registered medical practitioner.” ← 1971
BMJ re: home pregnancy tests

“Direct notification of abnormal test results … would result in patient

anxiety and confusion … patients lack the necessary expertise to interpret
test results.”

@hwitteman @wittemanlab
Challenge: Guess the decade
“The clinical interpretations of a laboratory finding should be given only by
a registered medical practitioner.” ← 1971
BMJ re: home pregnancy tests

“Direct notification of abnormal test results … would result in patient

anxiety and confusion … patients lack the necessary expertise to interpret
test results.” ← 2015
Callen et al., J Med Internet Res, re: various laboratory tests

@hwitteman @wittemanlab
 #WeAreNotWaiting

@hwitteman @wittemanlab
People want and deserve access to
their own results.
 Challenges: access, literacy,
numeracy, technical knowledge
 Bret Victor, quoted in Pavlus J. The Utopian UI Architect – re:form – Medium [Internet]. Medium. re:form; 2014.
@hwitteman @wittemanlab medium.com/re-form/the-utopian-ui-architect-34dead42a28#.syqs5i3z0
Image credit: Holly Witteman
@hwitteman @wittemanlab Image credit: Zikmund_Fisher et al., J Med Internet Res, 2014
@hwitteman @wittemanlab Zikmund-Fisher et al., J Med Internet Res, 2018
@hwitteman @wittemanlab
Information Theory
Data: symbols
Information: symbols with meaning
Knowledge: information placed in the context of existing knowledge,
actionable

@hwitteman @wittemanlab Ackoff RL. From data to wisdom. J Appl Syst Anal. 1989;16(1):3–9.
Central Premise
Laboratory results are data; people need information & knowledge

@hwitteman @wittemanlab
1. Whenever
possible, provide a
clear takeaway
message for each
result.
Answer the patient’s question(s):
“Am I OK?”
“Do I need to do anything?”

@hwitteman @wittemanlab Zikmund-Fisher et al., JAMIA, 2017

4. Individualize the frame of
reference by allowing custom
reference ranges.
“Standard” or “reference” ranges
are not standard for every
patient, nor are they the desired
reference for every patient

@hwitteman @wittemanlab Scherer et al., under review

Key Takeaways:
Data Information Knowledge
1. Whenever possible, provide a 6. Provide conversion tools along
clear takeaway message for each with results.
result.
7. Design in collaboration with
2. Signal whether differences are users.
meaningful or not.
8. Design for both new and
3. When feasible, provide experienced users.
thresholds for concern and action.
9. Make it easy for people use the
4. Individualize the frame of data as they wish.
reference by allowing custom
reference ranges. 10. Collaborate with experts from
relevant fields.
5. Ensure the system is accessible.

@hwitteman @wittemanlab Witteman & Zikmund-Fisher, Clin Chem Lab Med, 2018
Key Takeaways:
Data Information Knowledge
1. Whenever possible, provide a 6. Provide conversion tools along
clear takeaway message for each with results.
result.
7. Design in collaboration with
2. Signal whether differences are users.
meaningful or not.
8. Design for both new and
3. When feasible, provide experienced users.
thresholds for concern and action.
9. Make it easy for people use the
4. Individualize the frame of data as they wish.
reference by allowing custom
reference ranges. 10. Collaborate with experts from
relevant fields.
5. Ensure the system is accessible.

@hwitteman @wittemanlab Witteman & Zikmund-Fisher, Clin Chem Lab Med, 2018
Human Factors Engineering
Designing for the way people are,
not the way we wish they were

Adapting systems to people, rather

than expecting people to adapt to
systems

@hwitteman @wittemanlab
 Image credit: http://www.baddesigns.com/doors.html

@hwitteman @wittemanlab
@hwitteman @wittemanlab
User
Someone who uses something (a technology/system/thing/procedure …)
­ to accomplish a task
­ to accomplish a set of tasks
­ in pursuit of a goal

@hwitteman @wittemanlab
User
Someone who uses something (a technology/system/thing/procedure …)
­ to accomplish a task
­ to accomplish a set of tasks
­ in pursuit of a goal

@hwitteman @wittemanlab
@hwitteman @wittemanlab Image: Black & Decker, Canadian Tire
Defining & Aligning Goals
What are my users’ goals?
What are my goals?
Are these the same?

@hwitteman @wittemanlab
@hwitteman @wittemanlab
Related terms
Human-centered design
­ ISO, IDEO

Design thinking
­ Roots: business, d.school @Stanford

Co-design, -production, -creation

­ Increasingly the preferred term in some
countries

Participatory design
­ Roots: participatory action research

Differences
­ Disciplinary tradition
­ Location of power, responsibility
@hwitteman @wittemanlab
Key points
1: Iterative cycles
2: User research
3: Prototype early
4: Observe, not ask

@hwitteman @wittemanlab
Example process
1. Assemble diverse, multidisciplinary team
2. Observe existing processes
3. Focus groups: users create personas (Valaitis et al., 2014, Can Fam Physician)
4. Participatory design workshop with users & other experts
5. Test candidate designs
6. Interpret test results
7. Refine design
8. Test again, repeat steps 5-7 (maybe 4) two to three times or more

@hwitteman @wittemanlab
@hwitteman @wittemanlab
User Testing
Basic concept:
See how people respond
Fix problems/adjust design accordingly

“Design like you’re right; listen like you’re

wrong.” - John Lilly, former Mozilla CEO

@hwitteman @wittemanlab
“If I had asked people what they wanted,
they would have said faster horses.”
(Henry Ford may
or may not have
actually said this.)

@hwitteman @wittemanlab Photo: Ford Motor Co.

@hwitteman @wittemanlab
 Camera for facial emotion recognition software

Electroencephalogram or EEG

Eye tracking

Galvanic skin
response

@hwitteman @wittemanlab Slide: Hina Hakim

 Hina Hakim

@hwitteman @wittemanlab
Carnet santé Québec

@hwitteman @wittemanlab
A/B testing
Version A Version B
Accueil > Historique des prélèvements Accueil > Prélèvement (date)

@hwitteman @wittemanlab
A/B testing

@hwitteman @wittemanlab
A/B testing

@hwitteman @wittemanlab
@hwitteman @wittemanlab
@hwitteman @wittemanlab
Image credit: Anonymous, University Health Network, Toronto
healthdesignchallenge.com (2013)

Best Overall Design - 1st Place: Nightingale

(Chicago, Illinois)

Amy Guterman, Stephen Menton, Defne

Civelekoglu, Kunal Bhat, Amy Seng, and
Justin Rheinfrank from gravitytank

@hwitteman @wittemanlab Image credit: healthdesignchallenge.com

Image credit: labinterpret.com
User Testing: Recommendation
Table with 5 columns:
­ 1: design element
­ 2: the effect(s) you want this element to have (a useful design exercise anyway!)
­ 3: what users understood from this element
­ 4: how this element made users feel
­ 5: other comments, key quotes

Think of it like hypothesis-testing your design

@hwitteman @wittemanlab
User Testing: Recommendation
Table with 5 columns:
­ 1: design element
­ 2: the effect(s) you want this element to have (a useful design exercise anyway!)
­ 3: what users understood from this element
­ 4: how this element made users feel
­ 5: other comments, key quotes

Think of it like hypothesis-testing your design

@hwitteman @wittemanlab
User Testing: Recommendation
Table with 5 columns:
­ 1: design element
­ 2: what you want this element to convey (a useful design exercise anyway!)
­ 3: what users understood from this element
­ 4: how this element made users feel
­ 5: other comments, key quotes

Think of it like hypothesis-testing your design

@hwitteman @wittemanlab
User Testing: Recommendation
Table with 5 columns:
­ 1: design element
­ 2: what you want this element to convey (a useful design exercise anyway!)
­ 3: what users understood from this element
­ 4: how this element made users feel
­ 5: other comments, key quotes

Think of it like hypothesis-testing your design

Straightforward reference: Rocket Surgery Made Easy by Steve Krug
Great to do it in person but can also do at a distance

@hwitteman @wittemanlab
User Testing: Recommendation
Analyze your results as a team
Look at your 5-column table: anything people aren’t getting/reacting to in
the way you want?
Plan changes for next iteration, do it again

@hwitteman @wittemanlab
Best Practices: Results from
Modified Delphi
How many cycles?
2-3 necessary
4-6 ideal
May be more depending on how you count a cycle (e.g., agile: consider 1
round of user testing per sprint)
­ See also: Wilson K, Bell C, Wilson L, Witteman HO. Agile research to complement agile
development: a proposal for an mHealth research lifecycle. npj Digital Medicine 2018; 1:
46.

@hwitteman @wittemanlab
Best Practices: Results from
Modified Delphi
How many people?
Users per round: 5 necessary, 10-20 ideal
Total number of users: 10-15 necessary, 20-50 ideal
­ See also: Nielsen 1993; Virzi et al., 1996; Faulkner 2003; Borsci et al., 2013

@hwitteman @wittemanlab
Best Practices: Results from
Modified Delphi
How many people?
Users per round: 5 necessary, 10-20 ideal
Total number of users: 10-15 necessary, 20-50 ideal
­ See also: Nielsen 1993; Virzi et al., 1996; Faulkner 2003; Borsci et al., 2013

Not strictly necessary, but ideal to recruit new users each round

@hwitteman @wittemanlab
Best Practices: Results from
Modified Delphi
How many people?
Users per round: 5 necessary, 10-20 ideal
Total number of users: 10-15 necessary, 20-50 ideal
­ See also: Nielsen 1993; Virzi et al., 1996; Faulkner 2003; Borsci et al., 2013

Not strictly necessary, but ideal to recruit new users each round
Including citizens as core team members is HIGHLY recommended
… but this does not replace user testing

@hwitteman @wittemanlab
Warning 1
User testing doesn’t test everything
Functionality: standards (various), technical testing
Accessibility: standards (WCAG 2.1), simulations
Implementability: user testing is necessary but not sufficient

@hwitteman @wittemanlab Image credit: http://www.colourblindawareness.org/wp-content/themes/outreach/images/slider/living/traffic-light_p.jpg

Warning 2
Programmers are usually
not the best at user testing,
user experience design
Such unicorns exist, but are
rare

@hwitteman @wittemanlab
 I’m looking for:
1) more examples
2) people for an
international
committee

holly.witteman
@fmed.ulaval.ca

@hwitteman @wittemanlab