Beta Testing

Jeff Hurd
PRODUCT LEADER

@jeffhurd
st
th
ns

Overview What is a beta testing?

Why use beta testing?
Planning a successful beta test
Examining test results
Beta Testing
Making a working version of a website or application
available to a group of users prior to general release
Card sorting Interactive prototypes 1-1 interviews

Tree testing Focus group Visual design

Focus group 1-1 interviews Alpha testing

1-1 interviews Wireframes v2 Design updates

Wireframes v1 Interactive prototypes v2 Beta testing

 is
th
ns

Receive feedback on your new product or

feature from ‘real’ customers
Discover bugs with a small number of users
to mitigate risk
Make power users feel special for
contributing to product’s success
Gather qualitative and quantitative data to
continue to improve the product prior to
general release
Planning a Successful Beta Test
 Target Platforms

iOS Android Mac

Windows Browsers
 Kinds of Users

“Random”
Opt-in Recruited
selection
Beta Invitation
Keep it simple!

Catchy subject line with something in it for the

user – “Our new feature was made for you”

Relevant body text based on who you are writing

to – “Our new feature X will cut your editing time
in half, we know you use our product everyday
and we estimate it will save at least 5 hours of
edit time a week!”

Add a ‘Call to Action’ at the end – “If you are

interested click the sign up button below to gain
instant access to the feature”

AB test different email copy, create different copy

based on the persona
Testing Instructions

None, let the users explore organically

Instructions for technical / specific testing
1. Create new account from the login
page
2. Select Profile and add a new image

3. Under Social choose ‘Add a new friend’

4. Add another user as your friend and
send them a link from your portfolio
Examining Beta Test Results
Examining Beta Test Results

Explicit feedback Implicit feedback

Surveys Crashes

Beta diaries Event tracking

Twitter / Social media Clicks
Forums Abandon rates
Duration spent on screen/feature
 Test Result Data

Compare beta tester Categorize feedback Meet with your team

data to the control into themes and come up with ways
to improve metrics
st
th
ns

Summary What is a beta testing?

Why use beta testing?
Planning a successful beta test
Examining test results