Single VS.

Multiple Textbox Event Creation Experiment
Filip Mares – TIM Carleton University

Outline
     

Background Experiment Method Analysis Future Experiments Conclusion

Twegather
Create an event with 3 mouse clicks
 

 

Started in Summer 2009 Joint project for TeamCamp members Team of 5 active members Close to a Minimum Viable Product (MVP)

Difference from competition
  

Simple event creation process Registration not required for creating RSVP via Social Networks
Twitter  Facebook (soon)

Free

Experiment

Experiment
   

Natural Process Increase event creation Reduce creation process difficulty Re-design front page

Hypotheses

H1: Participants will create events quicker on average using the proposed page design opposed to the current page. H2: Participants will make fewer mistakes when creating events using the current design opposed to the proposed design. H3: Participants will prefer the proposed Twegather event creation page to the existing event creation page design.

Method

A/B test
 Design

A – Current  Design B – Proposed
  

8 participants Recorded screen with software Pre-thought event

Metrics
  

Completion Time Errors Encountered Questionnaire
 LIKERT  Designs

Preference  Designs Difficulty  Comments

Hypothesis 1
Participants will create events quicker on average using the proposed page design opposed to the current page.

Completion Time
Design A

Design B

Mean: 41.5 seconds SD: 13.6 seconds

Mean: 22.75 seconds SD: 7.28 seconds

Completion Time
 

P Value: 0.0091 Group comparison:
Design A pval: 0.779  Design B pval: 0.7648

Hypothesis Verified
 Design

A is faster to complete

Hypothesis 2
Participants will make fewer mistakes when creating events using the current design opposed to the proposed design.

Errors Encountered
Design A
 

Design B

Mean: 1.38 errors SD: 0.52 errors

Mean: 0.625 errors SD: 0.625 errors

Errors Encountered
 

P Value: 0.0199 Group Comparison:
Group A pval: 0.6376  Group B pval: 0.39

Hypothesis rejected
 Design

B had less

errors

Hypothesis 3
Participants will prefer the proposed Twegather event creation page to the existing event creation page design.

Design Preference
Design A

Design B

Mean: 3.5/5 rating SD: 0.93

Mean: 3.63/5 Rating SD: 1.19

Note: (really disliked) 1 – 5 (really liked)

Design Preference
 

P value: 0.8565 Group Comparison:
 Group

A pval: 1  Group B pval: 0.788

Hypothesis rejected
 There

were no obvious preferred design

Threats to Validity
  

Typing skill level Internet connection Test Environment

Participant Feedback

 

Single textbox hard at 1st, easier afterwards Multiple textbox requires less thinking Multiple textbox followed order of other services Good parsing of regular phrases in single textbox

Future Experiments

Event syntax auto-completion vs. preview Event creation workflow

Conclusion

Event creation completed faster with single textbox Less errors encountered in single textbox design Participants did not have a strong favorite design

Thank You
Questions?

Master your semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master your semester with Scribd & The New York Times

Cancel anytime.