WHAT DOES IT MEAN TO BE

A TEST ENGINEER?
Andrii Dzynia

http://bit.ly/1GYquoR

MY BACKGROUND

8 years in software engineering

started as a test engineer in a team of 20 testers

currently working on improving product quality

and development productivity at Spotify

@adzynia

https://www.linkedin.com/in/andriidzynia

SATURDAY EVENING

https://twitter.com/mfloryan/status/655433887015948288

LET ME SHOW YOU THE PROCESS

video: https://www.dropbox.com/s/un7m49kdgeuzsbz/test-sessionandrii.mov?dl=0

FOUND BUGS

MY COLLEAGUE,
FRONT-END DEVELOPER.

WHAT DID I LEARN?

Software always has bugs. If

you have not found any, does
not disprove their existence

Developers do not spend

much time looking for edge
cases, testers do

WHAT I WANTED YOU TO

LEARN?

Testing is not a set of artefacts.

TESTING IS A PROCESS.

@adzynia

TESTING PROCESS

GUESS WHAT BUG MARCIN

WAS REFERRING TO?

MORE LESSONS LEARNED

People are different. It is hard to predict every

single combination when user can get stuck

Think slower, sometimes

WHAT MAKES YOU A TEST

ENGINEER?
Critical thinking
Fault models
Patterns recognition
@adzynia

CRITICAL THINKING

https://www.criticalthinking.org/ctmodel/logic-model1.htm

FAULT MODELS
From the model, the designer or user can predict
the consequences of a particular fault. - Wikipedia

video: https://www.youtube.com/watch?v=rEumMkAhwzA

PATTERNS RECOGNITION

https://www.nasa.gov

BASED ON EXPERIENCE BRAIN RECOGNISES

SEQUENCES WHICH LEADS TO UNDESIRED
OUTCOME.
NOTE: BE CAREFUL WITH FAULTY ASSUMPTIONS.
FACE ON THE MOON SURFACE IS PAREIDOLIA.

Pareidolia is a psychological phenomenon when the mind perceives a

familiar pattern of something where none actually exists. - Wikipedia

LOOK FOR PATTERNS WITH

HIGH FAILURE RISK.

video: https://www.youtube.com/watch?v=qoQssWPRNR0

BRINGING VIDEO TO SPOTIFY

USERS

http://techcrunch.com/2015/05/20/spotify-introduces-video-clipspodcasts-and-activity-based-playlists/

TEAM SETUP

COMMUNICATION CAN BE A
PROBLEM

http://dilbert.com/strip/1989-05-31

SOLUTION WAS
Create a #slack channel
Start asking video integration questions
Rise bugs early and facilitate conversations

DID WE PREDICTED ALL DEFECTS BEFORE TESTING?

SURELY, NOT.

Data collected by Stefan Rungardt

BUT HOW DID THAT HELP?

Integration ended up to be much easier
Testing started earlier
# of production issues was low

WAS THAT EASY?

Not really. You have

to ask questions
people do not want
to hear

http://media2.giphy.com/media/wrTHBbla7suPu/giphy.gif

Tester: Shall video be playing when I lock a screen?

Programmer: I do not know.
Tester: who can know?
Programmer: Product manager?
Product manager: I do not know.
Tester: who can know?
Product manager: lets take a look how Youtube does it.

After a while we figured why video has to be paused

when you lock a screen. Licensing constraints.

DEFECT PREVENTION
VS
DEFECT DETECTION
Bugs are so 90s

LEAN MANUFACTURING

https://en.wikipedia.org/wiki/Lean_manufacturing

KEEP YOU WORK ENVIRONMENT

IN A GOOD SHAPE

engineering quality

product quality

strive for improvements

ENGINEERING QUALITY

Start testing as early as possible

Build safety nets

Fix bugs as fast as they occur

Conduct root-cause analysis and take actions

http://www.slideshare.net/AndrewDzynia/quality-built-in

PRODUCT QUALITY

Product success metrics

Time to first video frame
Number of users streaming video content
Average viewing time, etc.

Number of production defects

SPOTIFY TEST CERTIFIED MODEL

Inspired by Google
https://mike-bland.com/2011/10/18/test-certified.html

TAKE AWAYS

Testing is a process, not a set of artefacts

Cost of defect can be lowered. Think about that

Find ways to influence quality on organization

level. It is hard, but possible, even for a test
engineer

THANK YOU!
@adzynia
andrii@adzynia.com