Professional Documents
Culture Documents
DEVELOPMENT
QA IN AGILE PROCESS
The traditional models of software development have TESTING as a separate activity at the end
of the development process. In Agile, there is no such distinct process. Rather, testing is
ingrained into the development process.
QA is NOT the gatekeeper of the quality of the product in agile. Role of QA is around defending
the quality for the team by developing proper measures. On Agile Teams, testers serve as
quality coaches within the team, sharing testing knowledge and supporting quality assurance
work within the team. Quality is responsibility of the ENTIRE Scrum Team.
CHALLENGES
QA face several challenges when working in an agile development:
Changing requirements.
Pace of each iteration or cycle squeeze the test team's ability to develop and maintain tests.
Code that worked in previous sprints gets churned by new features in each
subsequent sprint, increasing the risk of regression
In the section below I have listed the various phases of agile development and the role of QA:
1. Release planning
2. Sprint Planning meeting
3. During the Sprint
4. Daily Stand up meeting
5. Bug triage meeting
6. Sprint review
7. Sprint retrospect
1. Release planning stage
QA perform the following tasks during release planning:
Scope of testing
Types/ level of testing based on the complexity of the features being tested
Infrastructure consideration (Test environment/ both OS and devices which testing has to be performed).
Resourcing
Test estimations
During planning, QA will have a great deal of input and raise helpful questions developers
may not have thought about.
QA is involved early on user stories with product owners and business analysts.
QA provides feedback on user stories while they are being analyzed.
QA determines the testability of the user stories.
QA needs to work with business owners to define Acceptance criteria
Acceptance criteria are the requirements that have to be met for a story to be
assessed as complete. Acceptance criteria constitute our Definition of Done. QA
help to determine if stories and their acceptance criteria are well defined and if
they satisfy customer requirements. User stories are subjective.
Each user story needs to have associated user story acceptance criteria. It helps
to remove ambiguity and improve understanding. Product owner will verify the
deliverable with the acceptance criteria. User story is said to be done only when
all the acceptance criteria for a story is met. A user story can be either done or
not done.
QA Estimate
The QA time effort estimate is a part of overall story point assignment. It is a common mistake
to discount QA time as part of estimate.
o In addition to known defects, we must also allocate time for fixing defects which are
not yet known -- that is, the defects which will be found during the upcoming sprint.
These defects may be related to the stories under development in the current sprint.
Or they may be found during system integration or other testing which could only
take place after an earlier sprint had been completed. Neglecting to allocate time to
fix unknown defects makes the sprint plan inaccurate.
More often overlooked is the test-fix-retest cycle thats necessary when a critical defect is
found that prevents the story from being accepted. For our sprints to be successful, its critical
to include these in our sizing.
Design
QA should be proactive and test if the designs have any issues:
Code
QA work side by side with the developers in a single team, taking a proactive approach and try
to avoid problems rather than fixing it at the end.
Test Ideas
In traditional work, there are often lots of test cases. They also occur in Agile work,
sometimes largely, and sometimes to a lesser extent. The disadvantage of the
traditional test case in an Agile project is that it takes quite a lot of time to plan and
write them. In many cases test ideas work better.
Make a list per area to be tested and in each case describe in 1-2 lines what should be
tested. In the end, your list will look like a bunch of one-liners. It is also wise to create a
supplementary checklist with general tests. If the need arises in the future they can be
converted to traditional test cases.
Agile stresses on working software over comprehensive documentation. Of course,
this does not mean that all documentation is unnecessary. Documentation is a tool to
achieve the goal, and the goal is working software.
Preparation of test data, Configuring testing tools etc
Setup QA test environment
Create traceability between the requirements and test ideas and try to improve the test
coverage.
Agile projects have short iterations enabling the project team to receive early and
continuous feedback on product quality throughout the development lifecycle. One way
to provide rapid feedback is by continuous integration.
o Continuous Integration (CI) is the practice, in software engineering, of merging
all developer working copies with a shared mainline multiple times a day. The
main aim is to overcome integration issues. Continuous Integration
automatically:
Static code analysis
Compile
Unit test (functional/ non-functional)
Deploy
Integration test (functional/ non-functional)
Pair Testing
When a developer develops a feature, he runs the unit tests and also a quick round of
testing to ensure that the feature is working. He then invites the tester to test the
functionality in dev environment. If there are any issues then developer will fix it
immediately. Once the tests pass the developer will commit the code to the master.
This helps to identify and fix issues early. In this way bugs are detected and resolved
early.
The tester performs further tests in the testing environment. If the tests fail then the
developer continue to fix the issues. Pair testing can be performed when developer
has completed a feature or fixed an issue.
Test Executions
3. After completion of testing the testing team will send out an end of testing report. The
report will explain the outcome of testing in detail. The report will consist of:
1. Tasks accomplished in the sprint
2. User stories tested
3. Tests performed
4. Summary
5. Matrices
6. Plan for next sprint
Once the testing is complete the team determines, along with the client, which defects are to
be fixed in the current sprint. The estimates for each item should be planned keeping the bug
fixes in mind.
4. Bug triage
The process of deciding which bugs to fix and which to leave in the product is called as Triage.
A bug triage meeting should be held regularly at during the testing cycle of a project. At the bug
triage meeting, each defect should be discussed, even those that are rated at a lower priority.
The developer should present the level of complexity and the risk associated with fixing each
defect. The Change control board (CCB) makes the final decision whether to fix, defer or reject
the bugs. CCB usually consists of representatives of key stakeholder groups. Scrum master
updates the defect tracer as and when the priorities are decided.
Triaging a bug involves:
Making sure the bug has enough information for the developers and makes sense
Making sure the bug is filed in the correct place
Making sure the bug has sensible "Severity" and "Priority" fields
If a bug is rejected or deferred or rejected and if QA feels that bug is really important to fix
then they can appeal the no-fix decision. QA can say fix it and can argue passionately for
the fix. They can investigate that bug some more and try to add more details to the bug and
ask the scrum master and product owners to reconsider the issue.
Build deployment
1. Dev should freeze the code after the allocated time is completed and must deploy an
integrated build. It is the responsibility of the developers to perform unit testing, integration
testing before deploying build to QA.
2. Developers should send out a mail when they deploy a build. It is mandatory to update JIRA
and make the items Ready for QA.QA team will only take up the defects in Ready for QA
status. The deployment mail should clearly mention the below:
3. Testers will reject the build if the app fails the smoke tests or if the mail is not testable.
4. Version control is mandatory. A build should be versioned appropriately (Eg: Inblox 1.0). QA
will mention the build in the bug reports so that bugs can be tracked efficiently. The new
features, bugs fixed in every version should be tracked in a shared folder with the version
number.
QA cycles
In the beginning of the first sprint the team prioritizes and decides which user stories should be
present in the sprint. While the development team starts the implementation simultaneously
the QA team begins work on the test ideas.
The QA starts the testing process once the build is deployed. The defects found are raised.
When QA are testing the sprint 1 the Dev team should move forward and start developing the
new features for sprint 2.
In some sprints we might encounter lots of issues and a hardening sprint can be planned.
A hardening sprint is an additional sprint that some Scrum teams run at the completion of a
regular sprint. In a hardening sprint, the team stops focusing on delivering new features or
architecture, and instead spends their time on stabilizing the system and getting it ready to be
released. A hardening sprint can be used for bug fixes in previous sprints. Bugs that are
prioritized will be considered here. Hardening sprint may not be a good indication and suggests
that the lots of issues are occurring.
5. Daily Standup meeting
It's important for every QA to attend and contribute to the daily stand-up meetings. QA need
to communicate the following in a stand up meeting:
In Scrum, each sprint is required to deliver a potentially shippable product increment. This
means that at the end of each sprint, the team has produced a coded, tested and usable piece
of software.
So at the end of each sprint, a sprint review meeting is held. During this meeting, the Scrum
team shows what they accomplished during the sprint. Typically this takes the form of a demo
of the new features.
The sprint review meeting is intentionally kept very informal. A sprint review meeting should
not become a distraction or significant detour for the team; rather, it should be a natural result
of the sprint.
7. Retrospective
In Agile development, a retrospective is a meeting held at the end of each iteration to discuss
what was successful, what could be improved, and how to incorporate the improvements and
retain the successes in future iterations.
Retrospectives cover topics such as the process, people, organizations, relationships, and
tools.
Testers should play an important role in the retrospectives. Testers are part of the team and
bring their unique perspective. Testing occurs in each sprint and vitally contributes to
success.
Testers can provide input on both testing and non-testing activities.
Retrospectives can result in test-related improvement decisions focused on test
effectiveness, test productivity, test case quality, and team satisfaction. They may also
address the testability of the applications, user stories, features, or system interfaces. Root
cause analysis of defects can drive testing and development improvements.
BEST PRACTICES
MIND MAPPING
INCREMENTAL TEST DESIGN
Mind mapping is a useful tool when
testing. Test cases are gradually built from user
stories and other test bases, starting
For example, testers can use mind with simple tests and moving toward
mapping for tracebility,reporting,test more complex ones.
cases, improving coverage etc
TEST EARLY
PAIR TESTING
Testing should involve in all phases
A developer and a tester sit for an of SDLC. Test planning should be
exploratory session sharing a single started from beginning of the
screen. They test a feature by project.
mutually sharing ideas
Start testing early in the software
Pair testing has proven to be development would solve the
successful when the deadlines are problem, as the earlier you find a
close and we need a quality product bug, the cheaper it is to fix it.
in a short span. Pair testing can help
the team to get together, share
ideas and be benefited by each
others strengths.