TestRail Customer Survey 2021

Business Cases

25.
Which metrics or KPIs does your team track
and report on? (Select all that apply.)

Number of defects reported in production

Development lead time (time to market)

Development velocity

Test pass / fail rate

Test cycle time

Requirements coverage

Defect traceability

Total number of tests executed

Total number of tests created

Total number of automated tests created

Total number of automated tests executed

 Percentage of automated vs. manual tests (%)

Test effectiveness (% bugs found during testing

vs. production)
 Testing progress (passing/failing/blocked etc)

Number of defects reported per functional area

(code integrity)

Workload & resource allocation

Test activity / effort (number of tests run,

number of bugs found per test)

Change failure rate

Number of customer reported defects

(production incidents)

None - we don’t report on metrics

Don’t know

Other (please specify)

26.
Which metrics would you like to be able to
report on?

27.
How is ROI calculated for testing in your
organization?

We don’t currently measure, but are having

discussions about how to do that

We don’t currently measure, but will at some

point
 We don’t currently measure and it’s doubtful if
we will at any point

We currently measure ROI: (please specify)

28.
What are the primary objectives for your
team’s software development and QA teams
around quality right now? Rank from highest to
lowest priority.

´ Increase test coverage

´ Increase the frequency of deployments

´ Shorten the time new releases spend

in QA

´ Automate more tests

´ Reduce bugs in production

´ Implement a continuous integration /

continuous deployment model of
development

´ Reduce the amount of time it takes to

fix a bug once identified

´ Shift more testing to earlier in the

development cycle

´ Make testing more efficient

´ Implement more robust security

testing

´ Integrate security testing in your

development pipeline
 ´ Include more stakeholders in the
testing process

29.
On a scale of 1 to 5, rate how important each of
the following activities is to your success. (1 means
not at all important and 5 means your highest
priority.)

  1 2 3 4 5

Automating
more of your
team’s test
cases

Improving
collaboration
with the
development
team

Completing
testing in
less time

Being more
efficient

Involving
more non-QA
team
members in
testing (eg.
developers,
business
users)

Making sure
the correct
tests are
being run,
tracking test
coverage
  1 2 3 4 5

Transitioning
to agile,
DevOps
testing

Easily
reporting on
the progress
of your
testing

Working
effectively
with external
/ offshore /
outsource
testers

Tracking the
quality of
your product
over time

Determining
when a new
feature has
been tested
enough to
release
Maintaining
compliance
with
regulatory
requirements

Accelerating
time to
market for
new features
or bug fixes
  1 2 3 4 5

Maintaining a
historical
record of all
of your
testing
activities
Traceability

30.
What are your / your team's top 3-5 biggest
challenges?

Earlier involvement in the development of new

requirements/features

Managing data and testing environments

End-to-end testing across integrated systems

Developing automated tests

Having enough time to complete QA tasks

Not having enough team members dedicated to

QA

Helping the organization understand the value

and role of QA

Burnout

Working with business to understand

requirements before coding begins

Training

Getting everyone in the organization to take a

more active role in quality

Political & cultural issues

 Getting access to the tools you want to use

Keeping up with changes in the industry (new

tools and methodologies)

Adapting QA to new development workflows in

your organization (agile, CI/CD)

Not having enough budget

Collaborating with the development team

Working with external / outsource QA teams

Other (please specify)

31.
What key initiatives or challenges is your team
planning to focus on over the next 1-3 years?

Prev
Next

Powered by

See how easy it is to create a survey.