Test Automation Strategies

Test automation strategies
Youre the QA director or CIO at a growing software

Contents
The benefits and drawbacks of test automation Devising a test automation strategy: Getting started
organization. Youre sold on the idea that test automation is necessary in order to keep your team on track, provide quick feedback and manage technical debt. But like many software organizations, you have none at all. Where do you get started? In this E-Guide, readers will learn the benefits and drawbacks of test automation, as well as key strategies for test automation. The benefits and drawbacks of test automation
By: Peter Walen What are the benefits of automation in software testing? What are the drawbacks? This is a question that, if you ask two knowledgeable people, you are likely to get three answers. Part of this is because automation covers a great deal of ground, all of which is dependent on the context they work in. From my experience, the question revolves around what people hope to achieve from automation. Most are looking to reduce costs or speed the testing process or make testing more efficient or better in some way. This is fine as far as it goes. The issue is that most people are looking to automation to fix all their testing ills. They are hoping that automation tools will address problems that manual testing practices have not been able to address. They are looking for a silver bullet to address their problems. The simple fact is, if you have problems with manual testing, automation will not fix them it is likely to compound them. In reality, automation can help you by executing mundane tasks, allowing good testers to examine the results carefully for anomalies that may inform their testing effort. Depending on your situation, it can also help you create scenarios that are not capable of being tested manually.
Page 2 of 8
Sponsored by
For example, you may send an invalid transaction to a system you are testing to make sure it is handled correctly. You may send three or four other invalid transactions. To send 300 or 400 transactions, invalid in a combination of ways, may be more than a manual test effort can absorb. One area where I believe automation is an absolute, hands-down success is in a continuous integration environment. Allowing a series of basic processes to run, exercising sample functions of the application, can quickly find problems that would delay or prevent other, more directed tests. This can free up testers to focus on less mundane tests that can reveal interesting behavior about the system. In this way, some basic automation exercising happy-path functions can allow more detail-oriented work to be done by the testers to investigate aspects of the application they may not be able to otherwise. I believe this is the greatest benefit of test automation not reducing testing, but allowing more productive, result-oriented work.
Contents
Devising a test automation strategy: Getting started

By: Lisa Crispin Youre the QA director or CIO at a growing software organization. Youre sold on the idea that test automation is necessary in order to keep your team on track, provide quick feedback and manage technical debt. But like many software organizations, you have none at all. Where do you get started? The whole-team approach The history of test automation is littered with failed test automation projects. Test automation sounded like a good idea, so the company bought a vendor GUI test automation tool, gave it to the test team, and waited for the magic to happen. Heres the news: automating tests is software development. It must be done with the same care and thought that goes into writing production code. And it needs a diversity of people and skills: testers to know the right things to test, customers to provide examples of desired behavior, programmers to design maintainable tests and more.
Page 3 of 8
Sponsored by
Test automation succeeds when the whole development team treats it as an
Contents
integral part of software development. Its not any harder or easier than writing production code. Learning to do it well takes time, effort and lots of small experiments. Change is hard, and people are motivated to change when they feel pain. Start by sharing the pain of manual regression testing. Your product cant be released to production without regression testing, right? Ask the testers to work with customers to identify the most critical areas of the application. They can then write manual regression testing scripts to ensure those areas work. The last day or two of each iteration, get everyone on the team together to divide up these manual test scripts and execute the manual regression testing. Ive been on three teams that took this approach. Believe me, having to slog through tedious manual regression test scripts provides major motivation for automating tests! Overcome barriers to automation Your cross-functional team has all the skills needed to overcome barriers to test automation. Get together for some time-boxed brainstorming meetings to identify the impediments standing in the way of automating tests. In my experience, the best approach is to identify the biggest obstacle, then try a small experiment to get around it. For example, when my team wanted to try test-driven development (TDD), which involves automating unit-level tests, we couldnt figure out how to do that with our nasty legacy code. First we tried brown bag sessions at lunch where the whole team tried writing unit tests. Then we brought in an expert to do training on it. That helped, but finally we just had to budget time to get traction on it. Should you tackle automating unit tests with test-driven development? Would GUI tests be the easy win? Its helpful to outline a strategy for automating tests.
Page 4 of 8
Sponsored by
Plan your automation strategy
Contents
There are so many tests to automate and so little time; it pays to focus on the automation that will provide you with the best return on investment (ROI). Mike Cohns Test Automation Pyramid provides a good guideline.
Unit and component tests verify our code design and architecture, and provide the best return on investment. They are quick to write, quick to run and provide a good safety net as part of a continuous integration process. They form the solid base of just about any test automation strategy. If your team has no test automation, learning to automate unit tests is the obvious place to start.
Page 5 of 8
Sponsored by
The next-best return on investment is the middle layer of the triangle, at the API or service level. These tests generally take the place of the user interface. They pass test inputs to production code, obtain results, and compare actual and expected results. Writing and maintaining them takes more time than unit tests, and they generally run more slowly, but they provide crucial feedback. Todays test libraries and frameworks allow us to automate and maintain GUI tests more cheaply than the old-school record/playback tools. Even so, they run more slowly than unit or API-level tests, and may require more frequent updating. In my experience, a two-pronged approach to start automating tests on a legacy system works well. While the team masters TDD and grows its unit test library, you can get some quick wins with simple GUI tests covering the most critical functionality. The test automation pyramid represents our long-term goal, but you wont get there overnight. Most teams start with an upside-down pyramid, more GUI tests than anything else. You can flip the pyramid over time. Choosing tools We dont have a wide array of tools at the unit test level; you just use the flavor of xUnit that goes with your production programming language, such as JUnit for Java or NUnit for .Net. For API and GUI level tests, start by experimenting with how youd like to specify your tests. Does a given/when/then format work well for your business experts? Perhaps your business domain lends itself to a tabular test format. Or, you might have a domain where specifying inputs and expected outputs in a spreadsheet works better. Some businesses prefer working with a time-based scenario. Once youve found a format to try, look for a test library or framework that supports that. If your team has enough bandwidth, you might be able to grow your own. Use retrospectives to evaluate whether your experiment is
Contents
Page 6 of 8
Sponsored by
having good results. If not, start a different experiment. This is a big investment. The right choices now will mean big returns in the long term.
Contents
Collaborating Automating tests is coding. Automated test code deserves the same respect, care and feeding as production code. It makes sense for the people writing production code to also write the test automation code. Testers are expert at knowing the right tests to automate. Other team members contribute expertise that helps us get timely feedback, with data and scenarios that represent production. It just makes sense to have everyone on the team work together to automate tests. Get started Pick one problem to solve, and do a small experiment to overcome it. If automating unit tests seems too daunting, try defect-driven development, an idea from Jared Richardson. For each defect that your team fixes, first write a unit test to reproduce the problem, then correct the code. Once the test passes, check in the test and the code. In December 2003, my team had zero automated tests. It took us eight months just to get traction on automating unit tests via TDD, and to create a suite of GUI regression tests for the most critical parts of the legacy code. Eight years later, we have over 6,000 JUnits, over 500 FitNesse test pages and over 122 GUI test scripts. Each of these contains multiple assertions. We know within 45 minutes of any code check-in whether it broke anything. As our application changes, we must continually look for the most effective ways to automate our tests. This is a journey, not a destination. Just take that first step.
Page 7 of 8
Sponsored by
Contents
Free resources for technology professionals

TechTarget publishes targeted technology media that address your need for information and resources for researching products, developing strategy and making cost-effective purchase decisions. Our network of technology-specific Web sites gives you access to industry experts, independent content and analysis and the Webs largest library of vendor-provided white papers, webcasts, podcasts, videos, virtual trade shows, research reports and more drawing on the rich R&D resources of technology providers to address market trends, challenges and solutions. Our live events and virtual seminars give you access to vendor neutral, expert commentary and advice on the issues and challenges you face daily. Our social community IT Knowledge Exchange allows you to share real world information in real time with peers and experts.
What makes TechTarget unique?

TechTarget is squarely focused on the enterprise IT space. Our team of editors and network of industry experts provide the richest, most relevant content to IT professionals and management. We leverage the immediacy of the Web, the networking and face-to-face opportunities of events and virtual events, and the ability to interact with peersall to create compelling and actionable information for enterprise IT professionals across all industries and markets.
Related TechTarget Websites
Page 8 of 8
Sponsored by

Test Automation Strategies

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Test Automation Strategies

Uploaded by

Copyright:

Available Formats

Test automation strategies