Chapter 1

General Approach to Security
Testing
In this chapter:
Different Types of Security Testers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
An Approach to Security Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Security testing is one of the most technical, time-consuming, yet rewarding areas of software
testing. When people think about software testers, they often visualize individuals who use a
software program the way the software company anticipates customers will use it. If you
are a software tester, you know that testers are often responsible for many kinds of testing,
including the following types:
■ Accessibility testing

Testing that the software is able to be used by people with disabil­

ities
■ International testing

Testing that software versions work correctly in other locales,
including functionality that might be customized in a locale or how the user interface is
displayed in that locale’s language

■ Performance testing

Testing how fast the software operates

■ Upgrade testing

Testing how the new software operates when a previous version is
already installed on the customer’s machine

■ Security testing

Testing the software’s ability to withstand attacks

This seems like a lot of testing, but, excluding security, all of these types of testing involve
scenarios that legitimate users of the software are likely to encounter. Sometimes the product
group making the software will come up with several different types of customers and use
scenarios for the product and will test to verify that the product behaves according to the
design specification for each customer and scenario combination. Accordingly, it’s crucial to
understand the issue in each category. For example, for accessibility testing it is helpful for
testers to consider that the user might be unable to use the mouse to click buttons. With this
knowledge, a tester can verify that all functionality on the menus can be accessed using the
keyboard.
Testing the product’s functionality in these legitimate use scenarios is important, but it doesn’t
test whether the product is secure. Security testing is different from all other types of testing.
1

The security tester’s job is to uncover these types of bugs.com/en-us/assistance/HP010440221033. If functionality testers or pen-testers aren’t testing software security.microsoft. Good testers can distinguish all of an application’s individual components and sub­ components from one another. how they all work together to provide an end-to-end solution. more important. Different Types of Security Testers For many software products. most good security testers are also excellent functionality testers. pen-testers work as security consultants and are hired by the software development company to perform security tests (also known as pen-tests). For example. it is a sensible way to conduct testing because the feature tester knows the specific functionality extremely well and is able to focus on exactly how the feature works.aspx. other times. Good testers can then identify problems that might occur based on the tester’s knowledge of how the appli­ cation works internally. third-party pen-testers. including accessibility. don’t worry! The product’s security will still be tested—by criminals. or pen-testers for short—work on the same test team as functionality testers. but testing this functionality doesn’t normally help the security tester identify whether a malicious user might employ the feature—for example. (For more information about Web beacons and tracking a user by using an HTML image. per­ formance. They understand how each works individually and. Sometimes security experts—also known as penetration testers. A legitimate customer (nonattacker) would not use the product as it is used in many of the scenarios tested during security testing. As it happens. Another common approach to security testing is to assign it to security experts. It doesn’t matter whether someone is abusing a coding error or using a feature that was designed to perform an insecure behavior. Although testing all these areas involves a great amount of work and is a great responsibility. spies. and security. see http://office.) When it comes to security. Some people call security testing “negative testing” because the tester verifies that bad things don’t happen. the person responsible for testing a specific feature is also responsible for testing all aspects of that feature. to track recipients of the e-mail message by using the image as a Web beacon. Tip Good security testers also have many of the characteristics of good functionality testers. and security hobbyists. In the subset of test scenarios that a legitimate user would experience. it doesn’t matter who uses the product and how the product is used if a feature or flaw can enable an attacker to compromise users or data. .2 Hunting Security Bugs Security testing attempts to find vulnerabilities in the software and to verify whether it’s pos­ sible for an attacker to misuse the software program for malicious purposes. a legitimate user might include images in e-mail messages. globalization. the security impact of a software flaw might not be realized.

Understand deeply what you are testing. such as password protec­ tion. They can spend extreme amounts of time examining a small piece of a program. often developers are unclear on exactly how all test cases will be handled by their code. The hobbyist attempts to figure out how all the pieces of the software work and how they can be used together to cause some insecure behavior. it is more important to understand the general approach to security testing. Some criminals and spies are prepared to use vast resources to attempt to find security vulnerabilities in target software if compromising the user or data is rewarding enough. Criminals might test a product’s security so that they can find a way to perpetrate a crime. fails. Pen-testers typically notify the software developer and sometimes the general public when a vulnerability is discovered. Even a seemingly unimportant home computer might be of interest to an attacker if it can be used as a tool to launch additional attacks and promote anonymity. Spies look for software security vulnerabilities for other reasons. Although it’s important to understand these attacks. Testing security functionality is not the equivalent of security testing. Security hobbyists test software for fun and challenge. Some security consulting companies are hired to test a company’s security by attempting to break into the company’s premises or networks. Security hobbyists are in a unique position because they can decide which software is interest­ ing to them and they can test it as long as they like. Security testing is often like a compli­ cated puzzle. If they find an issue. and experienced. Testing the functionality of security features is important—if a security feature. See Chapter 20. which is as follows: 1. the product might be unusable or insecure. An Approach to Security Testing This book describes various attack techniques. Many legitimate reasons exist for external pen-testers to test software whether or not they have been hired directly by the creator of the software. Spying isn’t limited to the corporate world. If the target company uses particular software. many are related to specific technologies. Perhaps an underhanded company will try to obtain confi­ dential information about its competitors by exploiting software vulnerabilities. Security hobbyists should not be thought of as novices—many are extremely knowledge­ able. On the other hand. For example. . clever. “Reporting Security Bugs” for more information on that notification process. security hobbyists usually notify the software creator and perhaps the general public the same way a third-party pen-tester does. a criminal might test the security of a banking Web site to find a software vulner­ ability that will enable access to the bank customers’ money. penetration-style security testing includes the deliberate testing of all the product’s features to be sure they can withstand attack.Chapter 1: General Approach to Security Testing 3 Important Do not confuse the testing of security features with security testing. Because software is very com­ plex. that software could be targeted for penetration. government agencies also spy.

Security weaknesses creep into software for various reasons. but you’re even more interested in how that feature was implemented. must be aware of hundreds of attack tricks and must defend against all of them. Whereas external attackers might be suc­ cessful in their malicious endeavors if they find a few security flaws a year. make the application accessible for people with disabilities. Although they use many methods similar to those of malicious attackers. The developer. security testers working on a product test team are expected to find numerous security bugs in specific prod­ ucts within tight production schedules. Attack the target by applying your malicious ideas. ultimately their job is to help protect the application by finding as many vulnerabilities as possible. several of which are discussed in this book. “Finding Entry Points. which makes it difficult to fend off all possible attacks successfully. You are interested in how someone will use an application feature. a bug finder must have a thorough and varied under­ standing of how the target application works. As an attacker—bug hunters need to see themselves as attackers—your job is easier than that of the developer attempting to block attacks: attack­ ers must find only one way to break into an application to be successful. on the other hand. For example. 4. For example. plus as-of-yet unforeseen issues. You can inspect the target application to verify your assumptions of how it was implemented.”) Next you can conduct various tests to see whether the developer was aware of the security issues or whether you’ve uncovered a security flaw. we find the approach described in this chapter to be the most effective and notice that many experi­ enced security testers use the same or a very similar approach. developers have many responsibilities in addition to blocking attacks.4 Hunting Security Bugs 2. Understanding Deeply What You Are Testing To be effective at finding security bugs. Stay informed about new attacks that might affect the target you are testing. Then you can consider the implementation and design of each software option and how the developer might have included protection of security weaknesses. . There are alternative approaches. A good understanding of how software is developed enables you to surmise several methods a developer might have used to implement the functionality you are security testing. and ensure the application is eas­ ily localizable. whereas attackers with deep security knowledge can spend large amounts of time trying to break an application by focusing on very small parts at a time. they must design and code the application’s functionality. Each bug that is found and fixed makes it more difficult for external bug hunters to abuse the software and helps protect the customers who use the product. Think maliciously about the target. some people conduct security testing strictly by searching the source code for commonly misused function calls. but it is difficult to be aware of all the many and varied attacks. Software developers are usually very clever people. In our experience. Security testers employed by a software development company are in a challenging position. Also. 3. to be successful. (Software inspection is discussed in Chapter 3. Developers must contend with aggressive ship schedules and so find their time limited.

the bar moves away from the notch in the shackle. as it turns out. Someone named “Jas­ onLynn” posted a video on a message board detailing how to cut an aluminum can so that it can be used as a shim to open a combination lock. . some locks are difficult to pick. anyone can pick the lock. Most people assume only someone with the threenumber combination or a bolt cutter can open the lock and access the protected property. how you can break it. In fact. you could simply use bolt cutters to cut it open. in turn. but many are not. allowing the shackle to move upward and the lock to open. Figure 1-1 Standard combination padlock used to secure possessions It might seem difficult or impossible to move the bar without physically damaging the lock. If you wanted to bypass the lock. The gap is about 0. a metal bar keeps the shackle from moving upward when the lock is in the locked position. Locksmiths already knew this technique and used a professional tool known as a shim for the same purpose.) But a close examination of the lock reveals a very narrow gap between the shackle and the body of the lock. as shown in Figure 1-1.Chapter 1: General Approach to Security Testing 5 Taking It Apart The advantages of thinking carefully about how the application was designed or implemented are clear. By using knowledge of how a com­ bination lock works internally and the right tool. However. An attacker needs only to move the metal bar away from the notch in the shackle to open the lock. Inside the combination lock. (Picking a lock is about opening the lock without damaging it. In addition to brainstorming ideas about how the application might have been put together. many people use a combination lock. a combination lock can be opened in less than 15 seconds.2 millimeters wide—so narrow that not even a fingernail could fit in it. picking a lock sounds difficult. This same approach is used in the physical world: to some­ one who does not understand how locks work. To protect their prop­ erty. When the correct com­ bination has been entered. With a piece of metal cut to the proper size. the aluminum of a soda can is just the right thickness and strength. The numbers on combination locks range from 0 to 39 and the locks have a shackle with a notch in its side. you can take apart the target application to get a better understanding of how it works and. if you know how they operate.

6 Hunting Security Bugs Tip Bypassing software security is very similar to picking locks. and so on. Other times. Many people set up BBSs on their per­ sonal computers so that callers could create new limited accounts. when Caller ID became available. people used bulletin board systems (BBSs) to exchange messages (often only with other users of the same BBS). A few examples can show you how to use understanding of how an application works and a malicious mind-set to identify security issues. Much of this book focuses on how to take software apart and obtain information about how it is implemented. Just as an accessibility tester must understand how a user with disabilities might think about and interact with the software. It is extremely important to understand the target software and whether it has any dependencies. Thinking maliciously isn’t something most people are accustomed to doing but is key to being a successful security tester. it is sometimes obvious how the application can be compromised. Dependencies can include the protocols used. You can take applications apart to gain this knowledge. Later. you must think maliciously about how specific features could be used. it turned out not to be good for identity verification anyway because it can be spoofed by the caller. Attackers think maliciously about software functionality. Callback Verification Before the Internet was widely available. to find security bugs before attackers do. A BBS is a computer set up to receive calls. new users were instructed to type ATA in their terminal window . deep understanding of the inner workings provides more information about which attack method might be successful and what type of tools you might need to build to attack the target appli­ cation. You want to develop a deep understanding of how the internals work. the underlying operating system. but usually required some valid con­ tact information about the users because they needed to track malicious users. This knowledge makes it much easier to find security vulnerabilities. “Spoofing. Once you have this knowl­ edge. Users of the BBS used their modems to connect over the telephone lines. and play very limited games (often text based). the compiler. the code libraries called. And so. wanted to allow people to use their BBSs. We also discuss common problems with different implementations and designs so that you can pick the soft­ ware apart and find security vulnerabilities. you need to think about how a malicious user could employ various features of the software being tested.”) Some sysops manually validated new users by calling each one and talking with the user at the phone number used to register the account. The owners of the systems. called sysops. transfer files. Thinking Maliciously About Your Target After you have a deep understanding of the software. you must also think maliciously about the software. During registration. You can easily see why you want to have a deep understanding of how the software works internally. Other BBSs automatically verified new users by using a feature called callback verification (CBV). (Phone com­ panies didn’t offer the Caller ID service yet. see Chapter 6.

This probably sounds like a pretty typical merchandise return experience. noted the return in the point-of-sale software.Chapter 1: General Approach to Security Testing 7 when their phone rang for verification. many BBS sysops could have been spared much confusion. the new user would be granted access to the BBS. the customer service agent made a mis­ take that someone can exploit to steal merchandise from the store. If these were correct. The ATA command enabled the user’s modem to answer the phone call. Malicious users quickly realized the full potential of the CBV feature. right? Maybe. However. People who are good at finding security flaws in software usually are able to think maliciously about everyday encounters. the designers/developers of the CBV feature never anticipated users would use the feature to force the BBS system to call 911. The CBV feature then disconnected the new user’s session and immediately dialed the phone number the user registered with. and then gave us our money back. A malicious mind-set and technical knowledge (deep understanding) are required to find new security attacks against both new and existing technologies. The agent examined the receipt. too. it actually dialed the 911 emergency center and police were dispatched to the location of the 911 call. If someone had anticipated this design flaw before the CBV feature was released. Once the BBS called the new user back and a connection was reestablished. Recently. We got the box home. and noticed the table was the wrong color. When we reached the front of the customer service line. if you think carefully and maliciously about the situation. Merchandise Returns at a Retail Store This book outlines various understood attacks against common technologies. From a functionality standpoint. CBV programs were modified to include a list of numbers the system was prohibited from calling. A security tester’s job is to be malicious in thinking about a product’s design. which was where the BBS was running! Clearly. Can you think of ways to use this feature maliciously or to bypass it? The CBV feature dials any number the user types in. but finding security flaws often involves malicious thinking independent of any technology. the CBV feature would prompt the new user for a user name and password. it could be abused in such a way as to have the police arrive at the sysop’s home any time the malicious user chose: in the United States. a malicious caller could call the BBS and enter 911 as the first three digits of the callback phone number. the agent asked for the box that contained the pieces of the table and the sales receipt. the CBV feature worked. Understanding already-known attacks can help you catch them in your testing as well as help you establish a malicious mind-set toward software. because they have developed a malicious mind-set. However. For example. The agent checked the receipt and accepted the original merchandise back from us. we purchased a kitchen table from a local store that sells unassembled furniture. once the potential for abuse was under­ stood. opened it. The agent only . but now you need to think mali­ ciously about it. We went back to the store to return the table. The CBV example is a computer-related example. When the CBV feature called the modem number supplied.

the store security tapes show them making the return. which makes it even easier for some people to forget or ignore the legalities and ethical consider­ ations of their actions. common behaviors can be exploited to perpetrate an attack. Attacking the Product Once you’ve come up with some malicious ideas about how the software might be vulnerable to attack. etc. while keeping the new table at home. most software stores do not let you return software if the package has been opened.) to perform the attack. Software Returns To help prevent piracy. Unlike this example. We did not try to do this! It would be illegal and unethical to shoplift a table. The belief is that if the shrink wrap is still intact. the validation process is faulty. Often the way the store knows whether the package was opened is if the shrink wrap has been broken. money. remove the software. given a target that is valuable enough. This is an example of faulty validation. A customer can break the shrink wrap. Chapter 2. The agent did not look inside the box. these could be faked). Attack­ ing the product can be time-consuming. it is worth the time and money spent obtaining such equipment. and most returns require an ID or a phone number plus a signature (certainly. and then re-shrink-wrap the box. but you can see that with a little thought. we demonstrated how it wasn’t necessary to discover the lock’s security problem because someone else had already discovered the weakness and shared the information. Other times. One of the most important concepts for a security tester to remember is that software often acts like the customer service agent in the example and blindly trusts the data it is given with­ out properly validating it. an attacker will spend the resources (time. malicious people can use the information about bypassing .” discusses how to prioritize testing. We easily could have filled the original box with garbage weighing about the same as the kitchen table and returned that for a refund of the purchase price. and if the target software is valuable enough.8 Hunting Security Bugs checked the receipt and looked at the box containing the original merchandise. software can often be attacked with a great deal of anonymity. Because most of this book discusses specifically how to perform various attacks against software. some people do. you need to test and determine whether the product actually is vulnerable. “Using Threat Models for Security Test­ ing. Although the average person doesn’t own shrink-wrapping equipment. Certainly. Stay Informed About New Attacks In the earlier combination lock example. we briefly mention this step here and then move on to the last step in our general approach to security testing. When people return merchandise to a store. This shrink-wrap example points out something else important: if an attack is possible but difficult for the aver­ age user to accomplish. the software hasn’t been opened.

malicious users who hunt for security vulnerabilities so that they can commit crimes or spy. It is important for security testers to spend time understanding the issues posted. and Defcon (http://www. presenters have included employees of European and U. Black Hat (http://www. security consultants who are hired to break into a target. CanSecWest (http://www. it is important for you to stay up-to-date on the latest vulnerabilities and exploits by reading security mailing lists and/or attending security conferences because software security testing is a rapidly changing area. and gives you some insight into how attackers think when they attack software. gives you new ideas for test cases you might want to try. and the information is published. a company that designs combination locks could use this information to ensure its new design does not fall prey to the shim attack. In the past. conferences provide a great environment in which to discuss and learn about attackers’ attitudes toward finding vulnerabilities and exploiting them and to network with other security-minded people. . Anyone can post to these mailing lists. Some of the best resources providing information about new computer security issues are security mailing lists. but others can use this information for positive purposes.securityfocus. Once you have a good understanding of how the tested functionality works. In addition to the presentations.blackhat .grok. Every day. which means the quality of the content varies. you need to think maliciously about how the function­ ality could be abused.uk/pipermail/full-disclosure). Security conferences are also good resources for security testers.com).cansecwest. Summary Functionality testing does not test a product’s security.org.org) are popular security conferences in North America. Two of the more popular com­ puter security mailing lists are Bugtraq (http://www. Much like the combination lock example. Various types of people test software secu­ rity: security testers who work for software development companies. some of the most respected people in the security community.defcon. Attendees and presenters at these conferences are diverse. often unveil new attacks and/ or new tools to help attack software.com/archive/1) and Full-Dis­ closure (http://lists. someone securing property using a combi­ nation lock might consider using a higher-security lock because of the shim attack.com). Throughout the process. Even if a vulnerability is difficult to dis­ cover. new computer security issues are discovered con­ tinuously.Chapter 1: General Approach to Security Testing 9 a combination lock to steal. Understanding the flaws in detail helps you gain a better understanding of why a particular attack works. Then you test your malicious ideas against the target. govern­ ment agencies as well as people previously convicted of computer crimes. someone will discover it given enough time. The more information you have about how an application works.S. and hobbyists who do it for fun and profit. For example. Thorough security testing requires a deep understanding of how the tested functionality is implemented. information about newly discovered security flaws is posted and discussed. the more insight you will have in finding security vulnerabilities. The presenters.

.

. . . . . . . . we focus on 11 . . . . . . . . . . . . . . . . . . . In the lock example. . . 22 As demonstrated in the combination lock example in Chapter 1. . . . . . . . . . . . . . . . . . . . . . . . . . describes how to create threat models quickly. . . . . . . . The second edition of Michael Howard and David Leblanc’s Writing Secure Code (Microsoft Press) and Michael Howard and Steve Lipner’s The Security Development Lifecycle (Microsoft Press) also contain information about threat model­ ing. . .msdn. . . . . . . . . . . . . In this chapter. This data could potentially be used to control the application in ways unwanted by the software creators. . . . . . . . . . . . Threat Modeling Software threat modeling is a process that has evolved quite a bit over the last few years. . . and how data enters and leaves the software or part of the software and to enumerate potential security threats. . . . . . . . . . . . . . You can use these valuable resources to expand your understanding of threat modeling beyond what we discuss here. . in an excellent article titled “Guerrilla Threat Modelling” at http://blogs. . . . How many people would quickly think of inserting a small piece of metal between the lock and the shackle? Not many. . . . . . . . . . 11 How Testers Can Leverage a Threat Model . . . we discuss how to better understand a piece of software. Threat modeling is a process that can be used to outline how a piece of software works. . . . . . . . . . . . . . . . . . . . . . . . . . . Microsoft Press published an entire book written by Frank Swiderski and Window Snyder titled Threat Modeling on the subject. . . . . . . . . . . . In addition to these books. . . what the software interacts with. . .aspx. . . .” it is important to really understand how something works to identify poten­ tial security issues. . 14 Enumeration of Threats . . . . . . In this chapter and the next chapter. . . . . . . . . . . . . . . . . . . 18 Implementation Rarely Matches the Specification or Threat Model . . . . . . . . . . . . . we discussed how it might not be easy to spot the potential threats. . 12 Data Flow Diagrams . . . . . . . including how attackers can send data to it and where that data is used. . . . . . . . . . . 13 Enumeration of Entry Points and Exit Points. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . “General Approach to Security Testing. .Chapter 2 Using Threat Models for Security Testing In this chapter: Threat Modeling . . . . . 15 How Testers Should Use a Completed Threat Model. . . . . .com/ptorr/archive/2005/02/22/378510. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Summary . . . . . Peter Torr. . . . . . . . . . . . . . . . . . . . .