Neils Quest for Quality

Colophon

Editing and typesetting LINE UP boek en media, Groningen

Book design Jan Faber
Cover photo Getty Images
ISBN 978

90 75414 83 7 (book)
978 90 75414 84 4 (ePub)

No part of this publication may be

reproduced or replicated and/or made
public (regardless of the intention) through
print, photocopying, microfilm, sound
track, electronic resources or through any
automatic retrieval system or in any other
way without prior written permission from
Sogeti Nederland B.V.

Neils Quest for Quality

A TMapHD Story

Aldert Boersma
Erik Vooijs
Thomas Veltman [Project Leader]

Contents
Foreword6

A TMapHD story 9

1 Square one 10
Building Block 1 Test manager 18
Building Block 2 Test manager in traditional
environments19
2 Getting acquainted 22
Building Block 3 Assignment 32
3 Meeting the team 34
Building Block 4 Test organization 41
4 A running start 44
Building Block 5 Test plan 50
Mr.MikkelsMusings 1 on building blocks 52
5 Rookie mistake 54
Building Block 6 Product risk analysis 60
6 A choice 64
Building Block 7 Test strategy 71
Building Block 8 Performance testing 75
7 Go/no go 80
Building Block 9 Test approaches 86
Building Block 10 Crowd-testing 88
8 Unhappy end 92
Building Block 11 Test varieties 96
9 The start of something new 100
Mr.MikkelsMusings 2 on the elements 108

Neils Quest for Quality

10 What Rupert wants 114

Building Block 12 Test manager in agile
environments123
11 Working on the plan 126
Building Block 13 Permanent test organization 134
12 Learning from others 136
Building Block 14 Model-based testing 142
13 Coordinating quality 146
Building Block 15 Quality policy 152
14 Guarantees 156
Building Block 16 Using test tools 162
15 A plan for Rupert 166
Building Block 17 Quality-driven characteristics 174
16 On track 178
Building Block 18 Integrated test organization 184
17 Inspiration 188
Building Block 19 Implementing test tools 194
18 Old habits die hard 198
Building Block 20 Reviewing requirements 202
19 Looking back and moving forward 206
20 The final curtain 212
Mr.MikkelsMusings 3 on built-in quality 213
21 Other musings 219
Acknowledgements232
References236

Contents

Foreword
Since the mid-1980s, Sogeti has been dedicated to quality and
testing in it. Starting off with a small team, in 30 years we
have grown to become a world leader employing 8000 skilled
professionals, geared to testing and quality. On a day-to-day
basis, they support our clients in increasing their success by
applying proven methodologies and the latest insights and
tools in the fast-moving world of it.
And of course noblesse oblige. As a leader we feel the
obligation to contribute to innovation. Hence the appeal to
our people to bring together their experiences and insights
to provide you with skills, approaches, methods and tools
that you can apply to enable your organization to perform
even better. Weve been doing this since 1995, when we
introduced TMap, our world leading quality method. Right
now were taking the most important step with our testing
method since the introduction of TMap. This book is a big
part of that.
The story you are about to read is about building great
software. It shows how quality assurance and testing play
a central part in a new world where organizations are confronted with a short time-to-market, new ways of working,
such as Agile development methods, and new technology
that is developing at the speed of light.

Neils Quest for Quality

In this new world, quality is of huge importance to everyone:

first of all, to our clients, the people who use software in
their everyday lives, to transfer money, to check in to public
transport, at their place of employment. We need to provide
them with confidence in the software we build.
Second, everyone involved in software development has to
have quality high on the agenda. This applies to developers
and testers, managers as well as consultants. Full confidence
is only achievable if everyone in an organization is aware of,
and works towards, quality. All this requires a new way of
software testing.
In this new style of software testing, we do everything as
simply as possible, but not simpler than that. We integrate
testing with all other competencies, in order to gain efficiency and make sure that everyone is aware of his or her
role in the quality process. We industrialize as much of our
work as possible, using tooling wherever we can. And all
of this is possible because of the skills and experience of the
people with whom we work.
We hope this book will inspire you to start improving the
quality of your organizations software. It is our firm belief
that delivering quality software is the best way to keep costs
low and the time-to-market short, and ultimately to drive the
success of organizations.
This book has been produced by our organization. I particularly want to thank Marco van den Brink, director of Testing
and Quality Control of Sogeti Netherlands, for making this
possible. Of course, I must express my gratitude to everyone

Foreword

else who participated, including our clients and our employees.

If, after reading the book, you feel we can assist you in your
quest for quality, please do not hesitate to contact us. We will
be glad to offer you our capabilities to serve you.

Piet Wybe Wagter

ceo Sogeti Netherlands

Neils Quest for Quality

A TMapHD story
This is a new book in the TMap suite. Together with the
website TMap.net it is the innovative next step, building
on the existing TMap knowledge. This book contains the
TMap Human Driven (TMap HD) story, consisting of a
business novel, building blocks, Mr. Mikkels musings and
contributions from the innovation board in testing.
Between the chapters of the novel, building blocks are
included with in-depth text, for anyone who wants to know
more on these specific topics. Mr. Mikkel, our mysterious
mentor, muses a few times on the subjects in the book and
introduces the use of building blocks in his first musings.
TMap.net contains all further techniques, methods and
theory, for anyone looking for still more depth and detail.
Enjoy.

Try not to become a man of success.

Rather become a man of value.
Albert Einstein

A TMap HD story

Square one

It took me a while to realize that the faint buzz emanating

from somewhere on the left side of our table was coming
from my phone. Looking left I could see the illuminated
screen slowly vibrating its way out of the hideaway of menus
and napkins I had created for it. After a moments hesitation I
threw Danielle a guilty look and reached for my phone. Right
then, the thing went both quiet and dark.
Theyll call again if its imp The buzzing had started
again. Get it, Danielle said, youll be distracted until you
find out whos calling anyway. I smiled at her, took my
phone and glanced at the screen while I made my way to the
hall.
Francine. That was odd. Until about six months ago I had
worked for her at bnu, one of the largest banks to be institutionalized by our government in the wake of the banking
crisis. While parts were either being sold, deferred to a bad
bank or retained, I was part of the team of project managers
that had hacked and slashed through the ensuing it frenzy.
Francine had been our program director. She proved to be
quite capable of shoveling a tremendous amount of work our
way. However, if it hadnt been for her and her gatekeeping
capabilities, tremendous would surely have turned into horrendous. After a bunch of delays and shifts of scope we did
manage to get the job done, which was just short of a miracle.

10

Neils Quest for Quality

Hi Francine, two phone calls in a row. This should be good,

but can you keep it short? Im at dinner with Danielle, for the
first time in forever.
Ill be quick. So, you know I accepted an offer from zbo
last May right? ok. I knew they had it challenges; thats why
they needed a new it Manager. Its a big part of what led me
to accept their offer, hoping to be able to make a difference
fast. But this Seriously, flying to the moon in a paper plane
would be easier. In short, it challenges are surprising me at
every step I take, and I want you to help out. Can you meet
me tomorrow?

Hold on now, what do you need me for? And when? I cant

just drop everything and run.
Well get to that. Meet me for coffee first thing tomorrow.
You remember Ginas, the place on the corner of 11th and
Hastings? Say 7.30?
Ill be there. But I cant promise you a thing.
Danielle never liked Francine much, although they had never
met. The last project I did with Francine had me pretty much
living in the office for the better part of a year. And now,
barely 6 months free from the claws of that project, she had
called to enlist my help. On the one hand, it was flattering. I
had apparently made a lasting impression. On the other, it was
also daunting. Pocketing my phone, fragments of all-too-frequent all-nighters and endlessly micromanaged PowerPoint
slides started to flood my mind.

1Square one

11

As I approached our table, I noticed that my risotto had

already arrived and Danielle was nipping her Chardonnay,
gesturing me to hurry up. Who was that? she asked.
Francine. She wants me to come and work for her, at zbo.
I could see a certain chill slipping into Danielles demeanor.
She took another sip of wine, and started fidgeting with her
salmon fettuccine, her eyes fixed on the center of her plate.
Oh, I see. Are you going to?
I dont know. I dont even know if I can. I still have stuff to
do at bnu. Were meeting for coffee tomorrow morning.
Right.
Its just coffee, Dan, just coffee. Ill see what she has to say
and well talk about it, okay?
I know you. Youll plunge in and, well, well be having date
nights at the breakfast table, if any.
We wont. I wont let it happen. Itll be different this time.
Back from my meeting with Francine the following morning
I was hardly through the door before Danielle was firing a
barrage of questions into the hallway: And? How did it go?
Are you going to join zbo?
Its a challenge, I said, a good one though. I think.

12

Neils Quest for Quality

That morning coffee with a former boss turned into a twohour business bagel breakfast during which the entirety of
zbo and its problems were explained in painstaking detail.
In one dazzling hour, Francine painted a vivid picture of the
stand-off between innovation, bureaucracy and the extremely
motivated competition in which zbo was caught up. She was
as fast-paced as ever: if not hastily jotting down explanatory
diagrams, then excitedly gesturing as if trying to conjure
those diagrams out of thin air.
zbo was one of the countrys oldest and most prestigious
insurance companies. It had started as Zachary Insurance in
1889, being among the first to offer car insurance. Having
barely survived the first half of the century, it had grown to
be the market leader in residential insurance by the 1970s,
and retained that position until the turn of the millennium.
Around that point, Zachary Insurance began to notice a
decline in market share and revenue an early indicator of
the financial crisis that was to break out in 2008. Adopting a
new strategy to win back some of the market share lost, they
turned to takeovers and acquired Barbon and Oliver, both of
which were fairly young and up-and-coming online-based
insurance companies. Zachary Insurance thus became zbo.
Barbon focused on life insurance and Oliver on health,
which were deemed complementary to Zacharys portfolio. As both companies had made promising starts and
had caused some disquieted looks among Zachary and its
longstanding competitors, these takeovers were perceived
as talent acquisition aimed at speeding up the transition
from a paper and intermediary-based company to a multi-

1Square one

13

channel-based company. But, with the actual merging of the

companies well, all hell broke loose.
Francines monologue was fast and interspersed with all
kinds of, I presumed, zbo jargon. However fast it was, she
was clearly excited to work there, sometimes enthusiastically elaborating on their portfolio, then on a colleague and
even on the view from the top floor. Taking a sip from her
espresso, she continued her short history of zbo.
Barbon and Oliver were considered good buys, as they
excelled where Zachary had failed multiple times. They
were lean-and-mean it and result-driven companies that
focused on it as an enabling factor for the company as
a whole, certainly in comparison to Zacharys taller and
slower bureaucratic structure, which regarded it almost as
a necessary evil. The inevitable culture clash resulted in the
resignation of almost all talent acquired in the takeovers.
Now zbo was stuck in the middle, determined to wrestle
through its it adolescence, but slowed down by hierarchical
impediments and lack of specific employee skills.
To make matters worse, the former ceos of Barbon and
Oliver had eagerly awaited the expiration of their non-
compete clause. Within a year of expiration they had joined
forces and pulled off what zbo was still struggling to do.
They built a fresh insurance company from scratch, from of
the ashes of the failing merger of Z and bo, helped along by
funds acquired from the sale of their previous companies.

14

Neils Quest for Quality

As I was typing in her latest remark I said: ok, youve got

trouble on your hands, but why am I here? What do you
need?
A test manager.
I stopped typing and gave her a puzzled look from behind
my laptop. I could almost see the combination of anticipation
and impatience flowing through her veins.
Im not a test manager. Thats not
I know. Lets just say the previous test manager left for
greener pastures after I politely encouraged him to do so. The
project is a shambles, and I need someone I can rely upon. Ive
interviewed a couple of test managers but I didnt feel a connection with them. You know it, you know software delivery
lifecycles and, most importantly, I know what youre capable
of. And, most importantly, I want someone who can look at it
with a fresh perspective.
Im sorry Francine, youre asking me to fulfill a role I havent
fulfilled before, and the first thing you mention regarding the
work you want me to do is that its a shambles. I mean, really,
what do you want me to say to that?
First of all, its a stellar project. zbo has fought long and
hard to get to its current position. The intermediary network
supporting our sales force is by far the countrys largest, even
accounting for the fact that we had to let go of almost 10% of
our staff. Still, zbos offices and representatives are present in
malls, inner cities, suburbs, you name it. It has served us well
and will continue to do so, but the world changed while we
werent paying attention.

1Square one

15

As a result, our online presence is thin at best, although

we have been throwing money at it for ages. Now, this is
where Seabiscuit comes in, we are re-inventing zbos service
model by creating an online portal accessible from any
connected device, where just about anyone can get, add,
change, enhance, transfer and even cancel their insurance
policies. Im convinced itll give us a definitive edge over
our competition, its that good.
Second, it has a stellar deadline as well, which is why I need
good and reliable resources. Thats why youre here. Third,
I have secured help regarding the testing and quality aspect
of your assignment. Have you ever heard of Mr. Mikkel?
He gave a keynote speech on how to improve software
quality by using something he calls building blocks at
that project management conference at Financial Plaza I
attended last year
After listening to Francine for the last 15 minutes while
barely getting a word in, I had just taken a bite from my
bagel. Needing a minute, Francine took the opportunity
to call over a waitress and order another espresso. When
she was done I said: Ive heard of him, hes some kind of
testing guru, right?
Thats the one. During his speech he wanted us to test the
chair we were sitting on and then spent five minutes berating us for not asking about risks and requirements before
giving it the thumbs up. Anyway, he lectures now primarily, but Ive arranged for him to coach us for the duration of
Seabiscuit. Hell be on call if you need him.

16

Neils Quest for Quality

Seabiscuit? The project is called Seabiscuit? Seriously?

Like the horse?
Exactly. Wasnt my idea, but you know, racing to the front
of the pack from a losing position. Our ceo liked it, and
that was that.
All right. Tell me about this horse of yours then.
Over the next hour, while enjoying four espressos, Francine
kept drawing air diagrams, only stopping now and again
to check if I was still on the same track as she was. At the
end of her monologue she slid a white envelope across
the table. Her hand still on the envelope she said: Its a
good offer Neil. Think about it, but think fast. I tried to
hide the fact that she had already sold me on the project
an hour ago. She was right, it was a stellar project, which
could catapult zbo to the front of the pack. Being able to
consult Mr.Mikkel for test-related issues had somewhat
smoothened my reservations as far as testing tasks were
concerned. Getting up from my chair, I suppressed an
all-too eager smile and told Francine I would need a couple
of days to think about it.
It was a good offer, so good that it would significantly
speed along the relocation plans Danielle and I had. After
weighing up the pros and cons, and me promising D
anielle
around five times that I would not become a live-in
stranger who only showed up to sleep, change clothes, and
consume world-record-breaking breakfasts, I let Francine
know the following day that I would give bnu two weeks
notice and join her at zbo.

1Square one

17

Building Block 1

Test manager
What do test managers do? In traditional organizations, they
assign people to projects, oversee the testers progress, provide
feedback, and maybe offer some coaching to people who want
it. Test managers build trusting relationships with their staff
and build up the capacity of the testing group. How does that
change with a transition to Agile? Is there still a need for test
managers? The answers to these questions are given in the
Test manager in traditional environments and Test manager
in agile environments building blocks. The first will be given
directly below this building block, the Test manager in Agile
environments building block will follow after chapter 10.
In this book we use test manager as a generic term. In
practice, you can find many different terms that refer to this
role, such as test coordinator, test leader, project leader
testing, test director, and many more. Sometimes these terms
refer to different levels in the organization, when several test
coordinators are subordinate to a test manager, for example.
We advise you always to make a clear definition of the role and
the responsibilities in your specific situation.

18

Neils Quest for Quality

Building Block 2

Test manager
in traditional environments
In traditional organizations, the test manager leads a team of
test coordinators and/or testers. Since the test manager oversees
the entire testing process, he ought to be able to prevent a
fragmented approach. Todays test manager also tries to shift
the focus of testing at the end of a project toward other quality
measures that can be implemented at the start of a project,
such as reviews, inspections, proofs of concept. He or she is
the linking pin in drafting the test strategy, bringing all the
necessary parties and information together. The test manager
is responsible for the planning, management and execution
of testing, ensuring that it is on time and on budget and at
the right quality, for multiple test varieties. The test manager
reports in line with the overall test plan on the progress of the
test process and the quality of the test object.
Examples of the test manager tasks:
Creating the instructions for the test products delivered by
the various test varieties
Checking adherence to the instructions (internal reviews)
Coordinating the various test activities that apply to the test
varieties, such as setting up and managing the technical
infrastructure
Creating guidelines for communication and reporting
between the test varieties, and the test process and the
suppliers
Setting up overall test-method-related, technical and functional support
Keeping the various test plans consistent

Building Block 2

19

Building Block 2

Reporting on the overall test progress, budget and quality

of the test object, preferably automated with a test management tool
Managing expectations of different stakeholders with respect
to test progress and quality
Deploying/hiring (extra) test personnel.
The relationship between the specified roles, the test varieties
and the relationships with the other stakeholders in the system
development process must be determined and documented.
The testing organization is clearly part of the bigger (project)
picture. Refer to figure 1 for some examples. In these examples,
reporting lines and supporting departments, such as a test
expertise centre for example, have been omitted.
Explanation of the examples:
Example a
The test manager is completely independent of both the project
manager and the subproject realization lead, and works at
the same level as the project manager.
Example b
The test manager is dependent on the project manager, but
independent of the subproject realization lead, and works at
the same level as the subproject realization lead.
Example c
The test manager is dependent on the subproject realization
lead.

20

Neils Quest for Quality

Test
Manager

Project
Manager

Requirements
& Architecture
Lead

Realization
Lead

Implementation
Lead

Project
Manager

Requirements
& Architecture
Lead

Realization
Lead

Implementation
Lead

Test
Manager

Project
Manager

Requirements
& Architecture
Lead

Realization
Lead

Implementation
Lead

Test
Manager

Figure 1. Examples of positions of the test manager in projects

Building Block 2

21

Mr.MikkelsMusings on building blocks

52

MUSINGS
A journey of a thousand miles
begins with a single step Lao Tze

Typically, when Im coaching someone on quality and testing,

I find that they can be overwhelmed if they are presented with
a whole method for quality and test all at once. It is much
easier to present each part of the method independently and
acquaint them with one part before introducing the next. Often
it is best to start with the parts that are most important to their
situation, have them learn and implement these, and then start
on the next one.
The same thing works on a larger scale as well. When organizations
want to change to a better quality and testing method, it is much
easier for them to learn one part well, implement that part to
solve a particular problem, and then look for a next part. When
an organization is confronted with a whole new process all at
once, this often leads to poor understanding of this process. In
those cases, many steps and tools are not very well understood.
This leads to situations where following the method becomes
the goal rather than solving the problem at hand. This leads to
the in-name-only variants of standard methods.
Furthermore, every organization is different and has different
needs for its testing method. Are you Agile, or more traditional?
Do you have to meet very formal quality standards or not? Do
you have very experienced people in your organization or are
you a young and eager company that has to learn a lot?

Neils Quest for Quality

All those things and more have an influence on how you model
the testing and quality method for your organization. This
means that every organization has its own optimum method. A
method that can be optimum for an organization at one point
in time, can become less optimum when something changes in
the situation. For instance, the introduction of a new tool that
makes it easier to test certain things may demand a change
in the method.
What people and organizations find very helpful is to build
up the method gradually themselves, with the aid of building
blocks. A building block is a process step or a tool or a role
that can solve a particular testing and quality problem in your
organization. A building block can also be fitted into the existing
method, or moved around within the method. For instance, a
specific test may be shifted to a point earlier in the lifecycle to
detect certain errors earlier in the process.
You can also make your own building blocks. If your organization
has to conform to specific standards, for example, you can
create a special building block to check whether or not these
standards are being met.
A great starting collection of building blocks of TMap HD can
be found on tmap.net. Feel free to use them as you please and
adapt them to your specific situation!
I can imagine that you would like more inspiration on the topic
of how to link building blocks together. This is all part of the
TMap Suite and can be found on tmap.net as well.

Musings 1

53

TMap HD - Human Driven

CLOUD

The goal of TMap HD is providing confidence in IT solutions through a quality

driven approach. Confidence is the fifth element over and above all others. The
four other elements are helping you to get there. These guidelines enable you to
make the right choices in applying the building blocks to suit your organization.

MODEL
BASED

RISK
ANALYSIS

AGILE

MOBILE

TEST
PLAN
CROWDTESTING

The TMap HD elements leading to confidence

218

Simplify

Integrate

Make things as simple as possible but not

more simple than that. Start small and work
from there. A complicated process will lose
focus on the result.

Integration with respect to testing denotes

to a shared way of working, with a shared
responsibility for quality. Testing is not a standalone process and should integrate seamlessly
in its environment.

Industrialize

People

Industrialization is very important in improving

testing and optimizing quality. Test tools can
be used to test more, more often, and faster.

Only people realize moving from testing according

to TMap to testing with TMap. People with a
broad knowledge of quality and testing. With
the right mindset to apply building blocks that
suit the company. A real human driven approach.

Neils Quest for Quality

21

Other musings

The testing innovation board is a think tank for testing.

Several organizations from different industries are represented in this think tank. In conjunction, they form a vision
and devise solutions to common testing problems in order
to support one another and other organizations in the Dutch
testing market.
The views of the innovation board helped us in the process
of shaping our view on testing and quality as described in
this book.
To show that zbo is not alone in its quest for quality and that
there is a multitude of testing situations and opinions, weve
asked the members of the innovation board to give an insight
in their vision on testing and quality.
These stories show that although there are many organizations with all kinds of mutual differences, they all strive
towards building better quality software and have need of a
testing and quality methodology to support them.

21Other musings

219

The tester of the future

The test competency has developed into a real profession with
its own methodologies, techniques, tools and specialists. Being a
complete overview of testing, TMap has played an important role
in this evolution. During the last decade, our profession has been
the subject of continuous change because of new development
techniques and because the business needs it solutions to be
faster, better and cheaper. Until now, the test methodologies and
techniques seemed to be sufficient to deal with this, but the biggest
current challenges are tooling and people.
Tooling is crucial in modern testing. If you dont use testing tools
you will not be able to keep up with the speed of development.
However, the proven tools are considered too complex, expensive
and old for Agile development, and lag behind modern technologies.
This problem is visible in many projects where test management
tools have been banned by development teams. A consequence
of this change is that we have to revise our way of thinking about
structured testing. Traceability is still considered important by our
principals but will be very difficult if it becomes impossible to integrate
test management tools and development tools. For traceability,
we need to invest in positioning the Product Risk Analysis and its
related (regression) testware as business assets. By integrating the
Product Risk in the Product Backlog, you can ensure the quality
of it products, obtain flexibility to focus on high-value areas and
know upfront what needs to be automated. Test automation is not
a choice but a fact. However, if you want to be successful, testing
and development should unite their approaches and tools.
People are the most important asset of the test community. Testers
are considered to be the bridge builders between development,

220

Neils Quest for Quality

operations and business because of their ability to look at complexity

from different perspectives, to question irregularities, and to explain
defects to multiple stakeholders. These skills make the tester a
strong team member in Agile product teams, but the focus will
come to lie less on test execution and much more on refining user
stories and being responsible for (automated) regression testing.
Knowledge about product risks, test automation and test design
techniques will be the key to success. The change in the toolset
requires a change in the skillset of the test consultant. As a test
professional you must be able to validate and use all kinds of tools,
often covering just a small part of the required test solution, and
advise the development team about possible usage.
One thing is certain with regard to the future: changes will come
at increasing speed and smart testing will become more and more
important. Smart testing is the ability to translate a good knowledge
of methods, techniques and tools into practical implementations.
To have maximum added value, the test consultant needs fast access
to knowledge and best practices, but should also be prepared to
share gathered knowledge with the test community. I expect that
the new TMap can facilitate these needs. If we, as individuals, can
comply with the Agile principles (support people, add business
value, integrate business and it and be adaptive for change), we
will have a bright future ahead of us.

Henk van Merode

Test Architect at Air France/klm

21Other musings

221

Highlighting exploratory testing

Exploratory testing is, in my view, an underrated topic. This structured
approach goes well with an Agile development method. And, just
as Agile, it is an answer to common problems such as bad project
documentation and coping with a myriad of changes after the
project has started. Exploratory Testing also addresses a popular
budget-driven Lean or pragmatic approach on the one hand, whilst
it allows for a structured approach towards qualitatively sound
products on the other.
Currently, much attention is being paid to testing in Agile. Exploratory Testing, however, much to my regret, receives considerably
less. Is this due to a misinterpretation of this approach? Many
people take the term too literally and associate it with error
guessing, free format testing, or monkey testing. I want to remove
these misconceptions surrounding Exploratory Testing. It is a
professional, structured and quick test approach, which, if in the
hands of experts, will deliver excellent results. The approach can
be combined with existing TMap test techniques. It makes optimal
use of Product Risk Analysis (pra) assessments by focusing on the
primary processes and functionalities. Verbal communication is
stimulated, as most knowledge is stored in peoples minds, not
on paper. So why would you settle for using paper, a possibly
erroneous and diluted version of reality?

Many times I have been sent on an errand to collect information:

Documentation? Here are some memos and emails. For the rest
we have your expertise and skill to collect the rest. Sometimes
it was a bit of a shock, but I consulted several departments and
retrieved the test charters. Next, I organized a pra meeting. With
the information I had collected I quickly created the testset. The

222

Neils Quest for Quality

developers and business representatives were pleased with my

testset, as it helped them to improve as well. Because the developers
also lacked a flexible and focused approach, and well-written and
complete documentation in order to do their jobs properly. And
thus a Test Driven Development methodology was born within this
project. Life can be simple.
It remains a pity that Exploratory Testing is such an underrated
subject, plagued by misconceptions. It is neither a means to save
money nor for shortening the time spent on testing or documenting.
In the hands of experts, Exploratory Testing is a structured way
to measure the quality of the information system in a relatively
short time and to reach an objective view on the risks involved.

Niki van Dreumel

a.s.r. Insurances

21Other musings

223

What type of human will still be tester in 2025?

In 2025 I expect that every human being will contribute to the
designing, building and testing of software. End users will report
any issues they encounter in their everyday use of information
systems by means of a support robot that constantly monitors the
interaction between human and computer. Whether products meet
the customers expectations will be measured by clients themselves.
Developers will have been replaced by coding robots that have a
constant connection to support robots, thus establishing a continuous
process of improving the software.
When this becomes a reality, I believe that we will let go the fundamentals
of structured and integrated testing. The structured TMap approach
of the 1990s has been challenged regularly in recent years. This has
resulted in new ways of testing such as a risk-based approach or a
business driven approach, and at present we are adopting an Agile
approach to testing. In my opinion, little attention is being devoted to
humans themselves in all forms of testing. This is not unique to the
testing process, but actually applies to the entire software business.
Nevertheless, to me, humans themselves are the most important
link in this constantly changing environment. With this insight with
humans being the pivot I see many changes within the industry
that will lead to new insights, auxiliary tools, and faster processes.
It is also my observation that the tester constantly needs to adjust
his task and assignment in the overall Quality Assurance process.
It is my conviction that humans are the most important link in
an adequate testing process, testers must continue to develop
themselves. The test professionals should continue to follow the
latest developments and innovations, and the testers must want to
actually use these in their daily activities too.

224

Neils Quest for Quality

In a contemporary way, this book contributes to knowledge development and innovation and thereby supports the testing process.
In 1995, the four pillars of a structured test approach and the use of
testing techniques were important innovations. In recent years, the
trend has focused more on the adaptiveness of the testing organization and the use of new test approaches and tools. In the coming
years, the focus will be on the development of the test professionals
themselves. This will result in a further focus on the soft skills and
competences of the tester. Development of personal and emotional
characteristics will make the test professional an even better team
player who knows exactly where business benefits and value lie. The
test professional cares (as he/she is an empathetic human being) about
good collaboration within the team. As a consequence of empathy,
the test professional can understand and translate the requirements
of the client, the specifications from an engineering point of view, and
the wishes of end-users. Thus the requirements are consolidated and
mutual understanding and respect are gained. Understanding the
vision of the architect and listening to the dilemmas of the developer
are important parts of bridge-building, an activity in which the test
professional engages every day. The test professional pays attention to
his cognitive abilities, which will cause growth in perception. This being
the case, there will always be a difference in comparison to a robot.
In short, the future test professional is a professional pur sang, based
on knowledge, skills and experience. As a human, the professional
builds bridges between all parties and remains very sharp on
imperfections. He/she will not fail to pursue adequate quality. In
conclusion, the test professional is also a very involved and valuable
team player who excels in collaboration and collegiality.

Jan Mellema
Dutch Police

21Other musings

225

The 5 principles of continuous testing

With regard to the most important sectors, technological developments currently play a major role in all recent organizational
business models. Such developments are considered crucial to
survival or to the reinvention of oneself. Innovations of new products
and services are released one after the other at a high rate. The
required high-quality innovations have an unstoppable character
which makes a proper development aiming at achieving reliable
end products indispensible. This compels organizations to set
up Continuous Delivery models for which the development of
technology (such as information technology for example) amounts
to a continuous process of rapid successive activities. To many
organizations, the ultimate objective for the future is to develop into
a DevOps organization in which the development and management
activities are brought together within integrated teams.
Within the test domain, this leads to new insights which will lead
to Continuous Testing. It is my expectation that the dynamics
and the mindset within organizations, related to testing (in the
coming years), will receive an unprecedented impulse. In order
to maintain high-level test competence, it will be necessary for
existing test organizations to adapt themselves on the basis of
these developments.
In my view, the professional test organization will have to develop
in terms of the following five principles:
1. Time
It is necessary, within a short time period, to produce working
technological products as well as to automatically detect errors
at an early stage. This is necessary because there is not much
time available for error repairs and automated re-testing.
2. Quality
Quality does not just apply to the eventual realization of a

226

Neils Quest for Quality

working end product but also to a measure of predictability

that ensures that quality is always the same, regardless of the
speed and frequency with which technological developments
are expected to appear. Possessing high-quality test environments and test data certainly applies here as well.
3. Test automation
Test automation of all the different test types across multiple
application chains is of top priority with respect to achieving
success in an integrated organization, as well as supporting the
principles of time and quality.
4. Measurability
It is necessary for the test results to be transparent and always
available, in order to provide proper direction to the development process. Possessing good dashboards is a precondition
of correct choice.
5. Group dynamics
Testing will develop toward becoming a group process within
an integrated organization rather than simply being an activity
for specialists. This will lead to results becoming available at an
earlier stage through a different experience.
Of course, these principles are also applicable within a traditional
test organization and are actually not new. The essence and the
difference lie in the fact that more dynamics and mindset are
required nowadays.

Hugo Mutter
Test manager Eneco

21Other musings

227

Dear test manager, it is not too late yet!

The world is increasingly becoming a collection of modules or,
in other words, building blocks, such as telephone packages
( internet or not?), sms and how many mbs for example? Do
you receive an off-peak discount, a hotel room with or without
breakfast, the house with or without a dormer window? And this
dormer window will it stay attached to your house during a bad
storm? Could the house collapse from the weight of this new and
additional structure?
When you look at the profession of a test manager you will see
that he/she is becoming more and more involved in the testing of
disconnected modules but also in the testing of the impact these
disconnected modules will have upon a larger whole.
Iterative development, continuous delivery and Scrum have been
meaningful slogans for a while now. These have become regular
daily practices and what does that mean for the contemporary
test manager (in 2014)?
Slogans such as testing is dead, the test manager no longer has a
future because of agile and Scrum are becoming more frequent,
but are these justified?
In my opinion, there is still definitely a future as long as you, as
a test manager, realize in time that certain things will have to
change. So, you have work to do!!
If one would just purely Scrum (which is often not the case), then
the realization of products would come in first place within that
iterative world. Ready shippable products: these would have to be
tested from various different angles during the interactive Scrum
process. In this context, we can consider test varieties such as the
unit test (ut), the integration test (it) and the system test (st). But, of

228

Neils Quest for Quality

course, these products need to fit within a greater whole. And it is

self-evident that the Scrum team must hand these products back to
the maintenance department team. This is where the contemporary
test manager (2014) reappears in the picture the test manager
who acts as a bridge between realization and maintenance: the
tester active within maintenance, the tester active within a Scrum
team, the test manager being the bridge between both. Based on
the production analysis (pra), the test manager will prepare a test
strategy to make sure that the correct things are performed at the
right locations with the adequate coverage.
The test manager is a master in chain testing, regression testing
and the testing of non-functionals such as performance, installability
and manageability. But, above all, he secures continuity.
In order to be able to achieve all this, the skills of the test manager
will have to change. Whereas it used to be possible to earn your pay
as an acceptance test manager possessing just a bit of business
knowledge, nowadays you really do need some considerable
technical knowledge, such as tooling knowledge (for regression
testing and the testing of non-functionals for example). Before we
choose to optimize our test sets, we do have to make sure that
the test cases offer sufficient coverage. For that purpose the test
manager will need the TMap test specification techniques that
the team will use to achieve an optimal number of test cases with
the highest possible coverage. Once these test sets are have been
primed to go, you can start thinking about how to go ahead and
automate these. Automating test scripts without knowing the
coverage of these scripts is a completely useless activity. Claims
such as I have 500 test cases which are automatically run on a
daily basis dont mean a thing. Why not 5000 or 10000 test cases?
Its much better to be sure that the important functionalities are
covered, and not just the good paths of course.

21Other musings

229

There are quite a few test managers in the Netherlands, but

are they actually good test managers? Arent many of these
test mangers actually project leaders in disguise, with just a bit
more test knowledge? I believe that a test manager should be
capable of performing tests on his own, as well as being able to
demonstrate vast experience in testing techniques. And he must
be able to transfer that knowledge and experience to the other
people involved, not just within his test team but also, especially,
to the rest of the project and organization. After all, how can a
test manager select a test technique for a tester without knowing
how this technique actually works, the depth he can achieve with
it, and the test base that it requires? Is he (or will he be) able to
answer and explain the question: Will I truly be able to cover the
risks identified in the pra with the test techniques selected? This
knowledge supplemented by the correct tool knowledge will ensure
that the contemporary test manager will be prepared to handle
the situation as it is now and as it will be in the future.
Actually, our profession will become even more fun than it already
was, with the test manager as the spider in the web, with more
technical skills and forming the bridge between project and
maintenance.
Lets get started, before the train leaves the station!!

Peter Betting
Dutch Rail

230

Neils Quest for Quality

231

Acknowledgements
A book like this can only be written with the help of many
people, whose support, thoughts and friendly criticism helped
shape the book into what it has become.
The writers were selected by means of a writing competition. We,
as the writing team, would like to thank the other competitors
who provided great inspiration with their entries. These are, in
no particular order: Shyamalee Bhand, Wannes Laurens, Edwin
Markink, Niek Fraanje, Marcel van Donge, Bertrand Cornanguer, Tim Selous-Hodges, Maurice de Mare, Ron Pleunis and
Bob Legrand. A special thanks goes out to Ralph Klomp who
received an honorary mention and was kind enough to lend the
term Building Blocks to TMap HD.
Throughout the process of writing the book, the steering committee for the new TMap program was of great support. Not
only in word and in providing funds, but also in action. They
were all actively involved in making the book, assuming the roles
of project leader, reviewer, coach, writer, designer, and much
more. We thank Hans Kapteijns, Michiel Rigterink, Rik Marselis
and Marco van den Brink very much for this.
We have profited much from other peoples knowledge. And
although there are only two names on the cover, many more
people contributed to the book. These are, in no particular order:
Leo van der Aalst, Sven Fanslau, Bert Linker, Marco Jansen van
Doorn, Rik Marselis, Ben Visser, Richard Ammerlaan.

232

Neils Quest for Quality

And these arent the only group of writers who contributed.

We are very grateful that people from other organizations were
willing to inspire us with their vision, and that we could add their
musings to the book. So we also offer sincere thanks to Hugo
Mutter (Eneco), Peter Betting (Dutch Rail), Niki van Dreumel
(a.s.r. insurances), Henk van Merode (klm) and Jan Mellema
(Dutch Police).
One other group that provided great inspiration was what we
called the international team. They helped us with many ideas.
For one thing, Francine would not be in the book if it werent
for them. They are Gitte Ottosen, Sven Fanslau, Geert Vanhove,
Danil Maslyn, Ole Hansen and Tapani Aaltio.
Then there are a lot of other people who also contributed, participating in reviews, generating ideas or sharing in some other
way. These are people from all over the world and all kinds over
organizations. This list below is an attempt to name them all. Our
sincerest apologies to the people we didnt mention there are
probably more who deserve thanks.
That being said, and in no particular order, we would like
to thank: Randy Semeleer, Wessel Laurens, Eric Belt, Eva
Gransson, Gert Stad, Jos Punter, Mark Rumpf, Cecile Davis,
Mark Roozeboom, Arno Balemans, Sonja Boelhouwer, Mark
Schut, Wouter van den Heuvel, Caleb Repkes, Michel Henrietta,
Stephen Hyland, Jolanda Verhagen, Harold Brcheler, Bert
Noorman, Edwin Markink, Frank Goorhuis, Andr V
erschelling,
Eveline Moolenaars, Perry Heymans, Dr Robben, Rick
Meandor, Trude Rosendal, Jan Boersma, Thailitha van Gortel,
Steven Goudart, Tom van der Ven, Marc Rabel, Gerrit de Vries,
Wim van Uden, Shalendra Rakhan, Michel van den Brande,
Rens Webbers, Darko Andrieski, Patrick Schilder, Robin Klein,

Acknowledgements

233

Filip Joele, Gro Rognstad, Rob Baarda, Amir Zalbetian,

Alexander van Ewijk, Rochus Gorkink, Chantal Regout,
Bart Vrenegoor, Daniel Hein, Marlies van Steenbergen,
Pepijn Paap and Raymond Frusch.
Within Sogeti, we are lucky that even high management is willing
to invest their time and capacities to help create a book like this.
For this we thank Hans van Waayenburg, ceo of Sogeti Group,
and Piet Wybe Wagter, ceo of Sogeti in the Netherlands.
In the end, no project can succeed without great management
backing and the leadership to take the right decisions at the right
time. We would like to thank Marco van den Brink for making
the book possible.

Aldert Boersma
Erik Vooijs
Thomas Veltman (Project leader)

234

Neils Quest for Quality

235

The world of IT is changing rapidly, with innovations following one

another in breathtaking succession. This applies not only to the
field of technology, but also to our way of working. Time-to-market
and cost limiting are becoming more important by the day. These
new ways of working are occasionally perceived as a threat to the
quality of our software products. This book shows the opposite:
TMapHD, a human-driven and quality-driven approach offers opportunities to improve software quality in these newly emerging
circumstances.
I am immensely proud of the way this book has been created. The
main writers, Aldert and Erik, were selected by means of a competition
in which the entire international Sogeti test community participated.
They were supported by a writing team, over a hundred reviewers
and the whole Sogeti organization. All were jointly responsible for
this innovative book. I trust that this work will inspire you to adopt a
more quality-driven approach to your way of working.
Marco van den Brink
Director of Testing and Quality Control, Sogeti Netherlands

Aldert Boersma is a senior consultant a researcher, a connector, an

innovator who brings solutions. He possesses diverse talents: senior
quality manager, architect, R&D manager are just a few of the positions
he has held in the past.
Erik Vooijs is an experienced international senior test manager. He
founded the Talentmanagement program for new and driven younger
testers for Sogeti Netherlands.
Thomas Veltman s the project leader for the new TMap project. He is
a senior test consultant with a special interest in mobility and usability.

TMap HD is part of the TMap Suite

More content on tmap.net