You are on page 1of 18

What's the difference between load and stress testing ?

One of the most common, but unfortunate misuse of terminology
is treating "load testing" and "stress testing" as synonymous. The
consequence of this ignorant semantic abuse is usually that the system
is neither properly "load tested" nor subjected to a meaningful stress
test.

1. Stress testing is subjecting a system to an unreasonable load
while denying it the resources (e.g., RAM, disc, mips, interrupts,
etc.) needed to process that load. The idea is to stress a system to
the breaking point in order to find bugs that will make that break
potentially harmful. The system is not expected to process the
overload without adequate resources, but to behave (e.g., fail) in a
decent manner (e.g., not corrupting or losing data). Bugs and failure
modes discovered under stress testing may or may not be repaired
depending on the application, the failure mode, consequences, etc.
The load (incoming transaction stream) in stress testing is often
deliberately distorted so as to force the system into resource
depletion.

2. Load testing is subjecting a system to a statistically
representative (usually) load. The two main reasons for using such
loads is in support of software reliability testing and in
performance testing. The term "load testing" by itself is too vague
and imprecise to warrant use. For example, do you mean representative
load," "overload," "high load," etc. In performance testing, load is
varied from a minimum (zero) to the maximum level the system can
sustain without running out of resources or having, transactions
suffer (application-specific) excessive delay.

3. A third use of the term is as a test whose objective is to
determine the maximum sustainable load the system can handle.
In this usage, "load testing" is merely testing at the highest
transaction arrival rate in performance testing.

What's the difference between QA and testing?
Most testing jobs I see are nevertheless advertised as "QA". Some people
suggest that QC is a better set of initials to describe testing.

In my courses and my practice, I stick to the ANSI/IEEE
definitions, which agree with what is generally held *outside* the
software arena. The definitions boil down to:
* TESTING means "quality control"
* QUALITY CONTROL measures the quality of a product
* QUALITY ASSURANCE measures the quality of processes used
to create a quality product.

What is black box/white box testing?

Black-box and white-box are test design methods. Black-box test design
treats the system as a "black-box", so it doesn't explicitly use
knowledge of the internal structure. Black-box test design is usually
described as focusing on testing functional requirements. Synonyms for

But I was involved/participated actively with my Team Leader while creatiing Test Plans. meets the requirements and expectations and is maintainable. In practice. plans. etc. which . Unit testing is usually associated with structural test design. What do you like and dislike in this job? Like: Finding faults with the code and driving an application to perfection. White-box test design allows one to peek inside the "box". code. Behavioral test design is slightly different from black-box test design because the use of internal knowledge isn't strictly forbidden. Verification evaluates documents. and not vice versa. carrying out testing with limited resources. system testing. requirements and specifications. opaque-box. walkthroughs and inspection meetings. While black-box and white-box are terms that are still in popular use. How do you determine what to test? The duties of the software Tester is to go through the requirements documents and functional specification and based on those documents one should focus on writing test cases.black-box include: behavioral. One has to use a mixture of different methods so that they aren't hindered by the limitations of a particular one.) can use any test design methods. Have you ever created a test plan? I have never created any test plan.limited time and limited effort and uncovering bugs is a very challenging & creative task. but others wish we'd stop talking about boxes altogether. specifications and requirements. I developed and executed testcases. but this is because testers usually don't have well-defined requirements at the unit level to validate. and it focuses specifically on using internal knowledge of the software to guide the selection of test data. Dislikes: There is nothing that I can state as my dislike for my role as a tester.challenging by thinking about the application in the end user's perception. The input of verification are checklists. it hasn't proven useful to use a single test design method. The output of verification is a nearly perfect set of documents. it is interesting that we are doing destructive work but in the long term it is a consructive work. Some call this "gray-box" or "translucent-box" test design. Synonyms for white-box include: structural. glass-box and clear-box. Note that any level of testing (unit testing. Validation evaluates the product itself. Define quality for me as you understand it It is a software that is reasonably bug-free and delivered on time and within the budget. and closed-box. plans. it's very interesting. It is important to understand that these methods are used during the test design phase. functional. many people prefer the terms "behavioral" and "structural". reviews and meetings. Describe the difference between validation and verification Verification takes place before validation. and their influence is hard to see in the tests once they're implemented. issue lists. The input of validation is the actual testing of an actual product. but it's still discouraged.

Software testing is a trade-off between budget. the main course of testing is to check for the existance of defects or errors in a program or project or product.use cases. The integration of one or more components is a component. Unit Testing: in unit testing called components (or communicating components) are replaced with stubs. Correctness testing and reliability testing are two major areas of testing. this type of testing conducting who knows the domain knowledge.B). larger. it does not include any called sub-components (for procedural languages) or communicating components in general. or reliability estimation.Also we can say that the Purpose of Testing is to Analyse that the Product or project is according to Requirnments. verification and validation. As defined. Testing can be used as a generic metric as well. What are unit. components A and B are integrated to create a new. Thus.sanity testing in this situation i will try to test the application with the perception of end user. Note: The reason for "one or more" as contrasted to "Two or more" is to allow for components that call themselves recursively. Once the build is ready for testing. or trusted components. A unit typically is the work of one programmer (At least in principle). What is the purpose of the testing? The purpose of testing can be quality assurance. based up on some predefined instructions or conditions. The tester should carry out all these procedures at the time of application under development. so that all other testers can be ready for testing their modules/functionality. They have been compiled. b.it just means that A is a big component . The unit is tested in isolation.frame work. Make sure. Calling components are replaced with drivers or trusted super-components. and i use my(or someones) previous experiences. component and integration testing? Unit. The smallest compilable component. here we are not prepare any formal test plans and testcase documents. They have successfully passed the integration tests at the interface between them. component: a unit is a component. we know what to test and how to proceed for the testing. this type of testing conduting ad-hoc testing.covers all the functionality to be tested. time and quality. component (A. and loaded together.this is mainly called ad- hoc testing. How do you test if you have minimal or no documentation about the product? based on previous experiance on that particular product we start testing . most of the time we perform ad-hoc on this type of applications. main functionality is tested first. simulators. linked. Two components (actually one or more) are said to be integrated when: a. component testing: the same as unit testing except that all stubs and simulators are replaced with the real thing. Note that this does not conflict with the idea of incremental integration -.

of course. passage of data through global data structures and/or the use of pointers.not based on any knowledge of internal design or code. There have been variations on these definitions. Let A and B be two components in which A calls B. branches. and B. the component in question is the entire system). and regression testing. is a small one. Examples of system testing issues: resource loss bugs. Not always easily done unless the application has a well- designed architecture with tight code. as it requires detailed knowledge of the internal program design and code. Note: Sensitize is a technical term. .based on knowledge of the internal logic of an application's code. Tbsa + Tab == the integration test suite (+ = union). the component added. requires that various aspects of an application's functionality be independent enough to work separately before all parts of the program are completed. recovery. Tests are based on requirements and functionality.the inputs are to A. incremental integration testing . transaction synchronization bugs (often misnamed "timing bugs"). especially as concerns integration testing. based on them. System testing specifically goes after behaviors and bugs that are properties of the entire system as distinct from properties attributable to components (unless. done by programmers or by testers. Integration testing: This is easily generalized for OO languages by using the equivalent constructs for message passing. What kinds of testing should be considered? Black box testing . but the key point is that it is pretty darn formal and there's a goodly hunk of testing theory.continuous testing of an application as new functionality is added. throughput bugs. not B. the word "call" is to be understood in the most general sense of a data flow and is not restricted to just formal subroutine calls and returns -.for example. may require developing test driver modules or test harnesses. The inputs are to A. paths.the most 'micro' scale of testing. to test particular functions or code modules. OO testing. The outcome of the test of B may or may not be affected. Tbsa The tests in B's suite for which it is possible to sensitize A -. Tbsa is the set of tests which do cause A to follow a path in which B is called. performance. White box testing . As to the difference between integration testing and system testing. conditions. In the following. Tests are based on coverage of code statements. or that test drivers be developed as needed. security. It means inputs that will cause a routine to go down a specified path. Let Ta be the component level tests of A Let Tb be the component level tests of B Tab The tests in A's suite that cause A to call B. unit testing . Not every input to A will cause A to traverse a path in which B is called. Typically done by the programmer and not by testers.

if the new software is crashing systems every 5 minutes. Also used to describe such tests as system functional testing while under unusually heavy loads. context-driven testing .re-testing after fixes or modifications of the software or its environment. informal software test that is not based on formal test plans or test cases. failover testing . applications. Programmers and testers are usually not appropriate as usability testers. It can be difficult to determine how much re-testing is needed. minor design changes may still be made as a result of such testing.black-box type testing geared to functional requirements of an application.determining if software is satisfactory to an end-user or customer. functional testing . or upgrade install/uninstall processes. or interacting with other hardware.term often used interchangeably with 'stress' and 'load' testing. or systems if appropriate. the software may not be in a 'sane' enough condition to warrant further testing in its current state. covers all combined parts of a system. surveys. ad-hoc testing . end-to-end testing . or corrupting databases. acceptance testing . user acceptance testing . For example. etc.testing of combined parts of an application to determine if they function together correctly. or other catastrophic problems.comparing software weaknesses and strengths to competing products. partial. involves testing of a complete application environment in a situation that mimics real-world use. Clearly this is subjective.similar to exploratory testing. recovery testing .term often used interchangeably with 'load' and 'performance' testing. or based on use by end-users/customers over some limited period of time.final testing based on specifications of the end-user or customer. compatability testing . hardware failures. Ideally 'performance' testing (and any other 'type' of testing) is defined in requirements documentation or QA or Test Plans. such as testing of a web site under a range of loads to determine at what point the system's response time degrades or fails. especially near the end of the development cycle. alpha testing . and other techniques can be used. regression testing . using network communications. The 'parts' can be code modules. bogging down systems to a crawl.testing of an application when development is nearing completion. not by programmers or testers. exploratory testing . stress testing .testing of full.) system testing .typically an initial testing effort to determine if a new software version is performing well enough to accept it for a major testing effort.integration testing .similar to system testing.testing driven by an understanding of the environment.black-box type testing that is based on overall requirements specifications. input of large numerical values. install/uninstall testing . performance testing . environment. etc.testing an application under heavy loads. This doesn't mean that the programmers shouldn't check that their code works before releasing it (which of course applies to any stage of testing. User interviews. usability testing . Automated testing tools can be especially useful for this type of testing.typically used interchangeably with 'recovery testing' security testing . and intended use of software. testers may be learning the software as they test it. culture. client and server applications on a network. load testing . For example. comparison testing .often taken to mean a creative.testing for 'user-friendliness'. willful damage. . but often taken to mean that the testers have significant understanding of the software before testing it.testing how well a system recovers from crashes. video recording of user sessions. heavy repetition of certain actions or inputs. Typically done by end-users or others. etc. this type of testing should be done by testers. sanity testing or smoke testing . This type of testing is especially relevant to client/server and distributed systems. may require sophisticated testing techniques.testing how well the system protects against unauthorized internal or external access. large complex queries to a database system. and will depend on the targeted end-user or customer.testing how well software performs in a particular hardware/software/operating system/network/etc. the testing approach for life-critical medical equipment software would be completely different than that for a low-cost computer game. the 'macro' end of the test scale. individual applications. such as interacting with a database.

by deliberately introducing various code changes ('bugs') and retesting with the original test data/cases to determine if the 'bugs' are detected. easy to chage What are basic. walkthroughs usin check lists.when project deadlines comes 3. core.a method for determining if a set of test data or test cases is useful.less complexity 5.Bug rate falls down to certain level 5. 3.its a continuous process. CVS and many others. tools.all the test cases are executed with the certain percentage of pass.well documented 6. Proper implementation requires large computational resources. his work is preventive work. he does his work by conducting reviews. libraries.verifying the completeness and correctness of ITEMS. compilers. problems.Alpha and beta tesing period expires 6. patches and changes made to them. practises for a QA specialist? QA always monitors the project from beginning to end.meetings. What is Configuration management? Tools used? Software configuration management (SCM) is the control. designs. software configuration management is the decipline for systematically controlling the changes that take place during development.The Exit criteria are met 2. Typically done by end-users or others.independent 4. SCM covers the tools and processes used to control. PVCS. if there is process violation takes places then he takes a CAPA(corrective action and preventive action). SCM is the process of identifying and defining the ITEMS in the system. here configuration means arrangement of the parts of the software. and the recording that are made to the software and documentation throughout the software development cycle (SDLC). requirements.bug free 2. 1. mutation testing .beta testing . coordinate and track code. 2. change requests.testing when development and testing are essentially completed and final bugs and problems need to be found before final release. 8. and to keep track of who males the changes. not by programmers or testers.But there can be some factors influencing the span of testing process: 1. Tools include Rational ClearCase.recording and reporting the status of the ITEMS and change requests 4. Managemet which configures the software is called SCM. documentation. Doors. issue lists. .when all the core functionality is tested 4.reusable 3.All high seviority bugs are closed What is good code? a code which is 1.when all the test cases are passed with certain level 7. controlling the change of these ITEMS throughout their life cycle.. When should testing be stopped? There can be no end to Testing.

require walkthroughs and inspections when appropriate. design. miscommunication . make extensive use of group communication tools . as an automation engineer. If changes are necessary. and documentation. we can test data base retrieval time by using some complex and simple querries. intranet capabilities. budget etc. and how do you manage them? Main issue is the frequent change request. we need to take care of the changing objects and functionalities to update the scripts. Use prototypes to help nail down requirements. What are 5 common problems in the software development process? poor requirements . networked bug-tracking tools and change management tools.allow adequate time for planning. etc. 'Early' testing ideally includes unit testing by developers and built-in testing and diagnostic capabilities. attainable. insure that information/documentation is available . What issues come up in test automation.clear. complete. they should be adequately reflected in related schedule changes. In 'agile'-type environments. problems are inevitable. inadequate testing .requests to pile on new features after development is underway. personnel availability.be prepared to defend against excessive changes and additions once development has begun. groupware. detailed. What are 5 common solutions to software development problems? solid requirements .if too much work is crammed in too little time.There are several points to be considered here. incomplete. featuritis .e-mail. and be prepared to explain consequences. If possible.Exhaustive testing is not possible and also a time consuming process and hence not recommended. bug fixing.no one will know whether or not the program is any good until the customer complains or systems crash. work closely with customers/end-users to manage expectations.Should we test every possible combination/scenario for a program? Its not mandatory. changes. problems are guaranteed. re-testing. testing.Using boundary value analysis techinque and equivalence class we can reduce the no of possible combination Is an "A fast database retrieval rate" a testable requirement? it depends on the end user requirement if the end user required that data from the database must retrieved fast then it is a testable requirement.if developers don't know what's needed or customer's have erroneous expectations. But we should conver to the maximum extent satisfying the primary functionalities which are important and too much visible from the customer point of view. re-test after fixes or changes. and not testable. there will be problems. extremely common. adequate testing . too general.start testing early on. communication . personnel should be able to complete the project without burning out. continuous coordination with customers/end-users is necessary. realistic schedules . plan for adequate time for testing and bug-fixing. availble resources. Delivery time available.if requirements are unclear.. stick to initial requirements as much as possible . complexity of the application under testing. If there are frequent changes in the system. cohesive. This will provide them a higher comfort level with their requirements decisions and minimize excessive changes later on. testable requirements that are agreed to by all players. unrealistic schedule .

Level 3 . and 0.S. Defense Department to help improve software development processes.and up-to-date . meets requirements and/or expectations. It will depend on who the 'customer' is and their overall influence in the scheme of things. delivered on time and within budget. processes. 27% were rated at Level 1. successes may not be repeatable.S. successful practices can be repeated. developed by the SEI. quality is obviously a subjective term. magazine columnists.) The median size of organizations was 100 software . However. Project performance is predictable. Defense Department contractors. many of the QA processes involved are appropriate to any organization. now called the CMMI ('Capability Maturity Model Integration').preferably electronic. etc. requirements management. Level 1 . Level 2 .the focus is on continouous process improvement.characterized by chaos. realistic planning. 23% at 2.4% at 5. a Software Engineering Process Group is is in place to oversee software processes. 39% at 2.standard software development and maintenance processes are integrated throughout an organization. What is SEI? CMM? CMMI? ISO? IEEE? ANSI? Will it help? SEI = 'Software Engineering Institute' at Carnegie-Mellon University. promote teamwork and cooperation. and heroic efforts required by individuals to successfully complete projects. not paper. and 5% at 5. and if reasonably applied can be helpful. CMM = 'Capability Maturity Model'. The impact of new processes and technologies can be predicted and effectively implemented when required. and products. It is geared to large organizations such as large U. and training programs are used to ensure understanding and compliance. use protoypes if possible to clarify customers' expectations. 6% at 4. Organizations can receive CMMI ratings by undergoing assessments by qualified auditors.metrics are used to track productivity. What is software 'quality'? Quality software is reasonably bug-free. and is maintainable. 2% at 4. initiated by the U. Of those. and configuration management processes are in place. and quality is consistently high. Few if any processes in place. customer acceptance testers. A wide-angle view of the 'customers' of a software development project might include end-users.software project tracking. Each type of 'customer' will have their own slant on 'quality' . 13% at 3. 23% at 3. (For ratings during the period 1992-96. Perspective on CMM ratings: During 1997-2001. 62% were at Level 1. 1018 organizations were assessed. Level 5 . Level 4 . customer management. stockholders. periodic panics. future software maintenance engineers. customer contract officers. However. the development organization's management/accountants/testers/salespeople.the accounting department might define quality in terms of profits while an end- user might define quality as user-friendly and bug-free. It's a model of 5 levels of process 'maturity' that determine effectiveness in delivering quality software.

The full set of standards consists of: (a)Q9001-2000 . Bootstrap.Quality Management Systems: Guidelines for Performance Improvements. ITIL.S. Also see http://www.iso.S. By using a test plan that requires you to test each unit and ensure the viability of each before combining units. creates standards such as 'IEEE Standard for Software Test Documentation' (IEEE/ANSI Standard 829). TickIT. This allows high-level logic and data flow to be tested early in the process and it tends to minimize the need for drivers. 'IEEE Standard for Software Quality Assurance Plans' (IEEE/ANSI Standard 730).The ISO 9001:2000 standard (which replaces the previous standard of 1994) concerns quality systems that are assessed by outside auditors. publishes some software-related standards in conjunction with the IEEE and ASQ (American Society for Quality). You can do integration testing in a variety of ways but the following are three common strategies: The top-down approach to integration testing requires the highest-level modules be test and integrated first. MOF.it indicates only that documented processes are followed. Another disadvantage of top-down integration testing is its poor support for early release of limited functionality. and others. two units that have already been tested are combined into a component and the interface between them is tested.org/ IEEE = 'Institute of Electrical and Electronics Engineers' . The idea is to test combinations of pieces and eventually expand the process to test your modules with those of other groups. production. development. and certification is typically good for about 3 years. In the U. However. and it applies to many kinds of production and manufacturing organizations. A component. refers to an integrated aggregate of more than one unit.Quality Management Systems: Requirements. 'IEEE Standard of Software Unit Testing (IEEE/ANSI Standard 1008). testing.. they should be tested in pairs rather than all at once. (c)Q9004-2000 . Beyond that. Eventually all the modules making up a process are tested together. utility modules are tested early in the development process and the need for stubs is minimized. Trillium. Integration Testing :- Integration testing is a logical extension of unit testing. which are in turn aggregated into even larger parts of the program. 32% of organizations were U. These units are frequently referred to as utility modules. Other software development/IT management process assessment methods besides CMMI and ISO 9000 include SPICE. not just software. It covers documentation.S. By using this approach. In a realistic scenario. and CobiT. This method reduces the number of possibilities to a far simpler level of analysis. the need for stubs complicates test management and low-level utilities are tested relatively late in the development cycle. ANSI = 'American National Standards Institute'. The downside. federal contractors or agencies. and other processes. if the program is composed of more than one process.among other things. (b)Q9000-2000 . however. ISO = 'International Organisation for Standardization' . the primary industrial standards body in the U. installation. engineering/maintenance personnel. the most problematical key process area was in Software Quality Assurance. many units are combined into components. For those rated at Level 1. Note that ISO certification does not necessarily indicate quality products . design. In its simplest form.Quality Management Systems: Fundamentals and Vocabulary.asq.ch/ for the latest information. Integration testing identifies problems that occur when units are combined. To be ISO 9001 certified. after which a complete reassessment is required. servicing. in this sense. the standards can be purchased via the ASQ web site at http://e-standards. The bottom-up approach requires the lowest-level units be tested and integrated first. is . a third-party auditor assesses an organization. you know that any errors discovered when combining units are likely related to the interface between units.

Often it is noticed that during testing of the application. This paper presents an approach to handle different factors that affect application security testing. the bottom-up approach also provides poor support for early release of limited functionality. Infrastructure security etc are not under developers’ control. The developers should only define the highest priority features. the inputs for functions are integrated in the bottom- up pattern discussed above.Trust is a key component to customer adoption and retention. Subject: What's the 'spiral model'? Basically. Like the top-down approach. Why Web Application Security Should be Part of Your Web Risk Management Program There are many reasons your organization should identify and address web application security vulnerabilities as part of your web risk management program: Reduce Cost of Recovery and Fixes -. and other features of a description of functionality. With this knowledge. First. operating system hardening procedures. in that it can be less systematic than the other two approaches. intrusion detection systems.The basic idea involves counting inputs. etc. Web based Application Security Testing 1. Most organizations already have some degree of online security infrastructure like firewalls. Subject: What's a 'function point'? Function points and feature points are methods of estimating the "amount of functionality" required for a program. The outputs for each function are then integrated in the top-down manner. security doesn't get due focus and whatever security testing is done it is mainly limited to security functionality testing as captured in the requirements document. and are thus used to estimate project completion time.that the need for drivers complicates test management and high-level logic and data flow are tested late. they should then go back to define and implement more features in smaller chunks. Application Security is under developers’ control but other security aspects like IT security. requires testing along functional data and control-flow paths. • Encourage Website Adoption -. This white paper talks about application security for web based application. 2. The Tower Group cited that 26% of customers don’t use online banking for security fears and another 6% do not due . The potential weaknesses of this approach are significant.outputs. then get feedback from users/customers (such feedback distinguishes "evolutionary" from "incremental" development). Network Security. Don't define in detail the entire system at first. It also helps minimize the need for stubs and drivers. Introduction Security is one of the prime concerns for all the Internet applications.Consumers are still not adopting websites as a preferred channel for doing business.Computer security attacks cost as much as $10 billion a year. The primary advantage of this approach is the degree of support for early release of limited functionality. however. the idea is evolutionary development. The third approach.3 Ensure Customer Trust -. it's intended to help manage risks. Define and implement those. Any mistake at requirement gathering stage can leave the application vulnerable to potential attacks. The problem is that they often overlook the need to secure and verify the integrity of internally developed applications and code pages against external attacks. leading to the need for more regression testing. using the waterfall model for each step. sometimes referred to as the umbrella approach.

damage to credibility.to privacy issues. are probed daily. Prominent sites from a number of regulated industries including financial services. healthcare. 3. They include forms that collect personal. most companies deploy several patches a week.000 lines of code. government. therefore. legal liability.5 full workweeks or 700 hours.000 servers can spend $300. No one on the Internet is immune from security threats.200 vulnerabilities published by CERT last year for just 10 minutes would have required 1 staffer to research for 17. a business gets done. and confidential information such as medical history. many Web-based applications have inherent vulnerabilities and security- oriented design flaws. . Web applications are used to perform most major tasks or website functions. and loss of customer trust. and retail. Other organizations spend millions on outsourced security assessment and ethical hacking resources. has noted that almost 75 percent of attacks are tunneling through web applications.Many organizations are using trust as a key competitive advantage and are leveraging customer fears to proactively implement security and privacy programs to ease the uncertainty.000 lines of code. Web-based applications are enabling interaction among customers. and partners. classified. Consider these statistics: On average. Why Web Application Security is Important The Internet has forever changed the way. web applications have been developed and deployed with minimal attention given to security risks. or roughly $30. there are anywhere from 5 to 15 defects per 1. Internet- based attacks exploit these weaknesses to compromise sites and gain access to critical systems. In the race to develop online services. resulting in a surprising number of corporate sites that are vulnerable to hackers. Fixing one of these defects takes 2 to 9 hours each. prospects. Unfortunately.4 Maintain Competitive Advantage –. • Gartner Group estimates that a company with 1.000. The consequences of a security breach are great: loss of revenues. Security awareness for Web-based applications is. Gartner Inc. credit and bank account information and user satisfaction feedback. Web application security is a significant privacy and risk compliance concern that remains largely unaddressed. test security using costly manual processes that cannot address all potential website risks. Reduce Cost of Manual and Outsourced Security Testing -– Many organizations today. • A 5-year Pentagon study concluded that it takes an average of 75 minutes to track down one defect. essential to an organization’s overall security posture. That translates to 150 hours. to clean every 1.000 to test and deploy a patch. • Researching each of the 4. especially ones in regulated industries.

• Attacker can easily change any part of the HTTP request before submitting o URL o Cookies o Form fields o Hidden fields o Headers • Encoding is not encrypting • Inputs must be validated on the server (not just the client) Countermeasures • Tainting (Perl) • Code reviews (check variable against list of allowed values. The baseline is "not trust user input" and "not trust client side process results". with a user falling into a member of groups or roles with different abilities or privileges.1. to some users and not others. and eGraffiting 9 Stealth Commanding Concealing Weapons 10 3r d Party Mis-configuration Debilitating a Site 11 XML and Web services New layers of attack vectors & Vulnerabilities malicious use 12 SQL Injection Manipulation of DB information 3. Identity Theft. Threat Two: Access Control Access control is how an application grants access to content and functions.Examples of vulnerabilities S No Hacking attack What hackers use it for 1 Cookie Poisoning Identify theft/ Session Hijack 2 Hidden field Manipulation eShoplifting 3 Parameter tampering Fraud 4 Buffer Overflow Denial of Service/ C losure of Business 5 Cross Site Scripting Identity Theft 6 Backdoor and Debug options Trespassing 7 Forceful Browsing Breaking and Entering HTTP Response Splitting Phishing. access control flaws are very likely to happen.2. If access control rules are implemented in various locations all over the code. Possible Security Threats 3. A web application's access control model is closely tied to the content and functions that the site provides. • Usually inconsistently defined/applied • Examples o Insecure session IDs or keys o Forced browsing past access control checks o Path traversal o File permissions – may allow access to config/password files o Client-side caching . Threat One: Input Validation files. not vice-versa) • Application firewalls 3.

3. XSS occurs when an attacker uses a web application to send malicious code to a different end user. While this does not attack the website directly. and trust relationship. session ID protection. in effect sending new instructions to the attacked computer that could. are all taken care for you. and the second defense is to HTML-encode all input when it is used as output. but in some cases even web servers are also directly affected. The first defense is input validation.4. collect userids and passwords. perhaps by sending it in an official looking e- mail or using some other approach. the script will then execute in the user's browser. corrupting or overwriting the valid data held in them. password strength. Threat Four: Cross-Site Scripting (XSS) Flaws Cross-Site Scripting (XSS) attacks require the execution of Client-Side Languages (JavaScript. etc. Attacker uses a trust application/ company to send malicious code to end-user Attacker can “hide” the malicious code o Unicode encoding 2 types of attacks o Stored o Reflected Wide-spread problem! Countermeasure: input validation o Positive o Negative: “< > ( ) # &” o Don’t forget these: “&lt &gt &#40 &#41 &#35 &#38” 3. password storage. ActiveX. the extra data may contain codes designed to trigger specific actions. etc. Java. Since buffers are created to contain a finite amount of data. • Weak authentication o Password-only o Easily estimable usernames (Admin. It involves disguising a script as a URL variable within a URL from a legitimate site. Threat Three: Broken Account and Session Management If the application use the specification approved "user authentication infrastructures". If the site does not check URL variables. In buffer overflow attacks. Threat Five: Buffer Overflow A buffer overflow occurs when a program or process tries to store more data in a buffer (temporary data storage area) than it was in tended to hold. etc. VBScript.can overflow into adjacent buffers. Flash.). Buffer overflow is an increasingly common type of security attack on data integrity. the extra information . protecting credentials in transit.) o Unencrypted secrets are also insecure • How to break in o Guess password o Reset password o New password emailed by application o Sniff password • Backend authentication o How database passwords are stored o Trust relationships between hosts (IP address can be spoofed.) 3. for .which has to go somewhere . This will reduce dangerous HTML tags to more secure escape characters.3. Cross-site scripting is a relatively new approach that is being used to attack websites. and tricking a user into clicking on that URL. or anything else that a script can be used to do.5. it can be used to run arbitrary code in a user 's browser. Common sources of these kinds of attacks are web mail applications and web-based bulletin boards. then password change control..

com 25 o Replace this with something like this… o char shellcode[] = “\xeb\xlf\x5e\x89\x76\x08…” Countermeasures o Keep up with bug reports o Code reviews o Use Java 3.7. To test this vulnerability. Allows attacker to relay malicious code in form variables or URL o System commands o SQL o Interpreted code (Perl. Python.. Giving a generic error messages would take the application away from vulnerability scanning radar screens. etc. When user input is not adequately checked./” o Add more commands: “. DB dumps Helps attacker know how to target the application Inconsistencies can be revealed too “File not found” vs. etc. Mostly affects web/application servers Can affect apps/libraries too Goal: crash the target application and get a shell Buffer overflow example o echo “vrfy `perl –e ‘print “a” x 1000’`” |nc www. test the input with all the OS commands that the server box is set up to support and a defect can be filled for developers' investigation if HTTP 200 ok with an OS command input comes. it is possible to exec uting operating system commands. Threat Six: Command Injection flaws Majority of programming languages have syst em commands to aide their functionality. rm –r *” o SQL injection Countermeasures o Taint all input o Avoid system calls (use libraries instead) Run application with limited privileges 3.targetsystem.6. 401. This can greatly reduce script kids’ attacks.) . “Access denied” File-open errors Need to give enough information to user w/o giving too much information to attacker Countermeasures o Code review Modify default error pages (404. Examples: stack traces. change data.example. or disclos e confidential information.) Many applications use calls to external programs o Send mail Examples o Path traversal: “. damage the user's f iles. Buffer overflow attacks are said to have arisen because the C programming language supplied the framework. Threat Seven: Error Handling Problems All current exit vulnerability-scanning tools identify known vulnerabilities through known web site response or error messages. and poor programming practices supplied the vulnerability.

If successful. SQL Injection is the ability to inject SQL commands into the database engine through an existing application. Threat Ten: Database Testing The root cause of SQL injection is that an application uses dynamic SQL calls. SQL injection mainly exists in the context of Web applications where: (1) Web application has no. which are generated from users' input. These functions. passwords. or poorly implemented.10. Threat Nine: Web and Application server Misconfigurations Most of networks have deployed some sort of security patch update servers. SUS or SMS.3. Tension between “work out of the box” and “use only what you need” Developers web masters Examples o Un-patched security flaws (BID example) o Misconfigurations that allow directory traversal o Administrative services accessible o Default accounts/ passwords Countermeasures o Create and use hardening guides o Turn off all unused services o Set up and audit roles. Threat Eleven: Privilege Elevation Privilege elevation is a class of attacks where a hacker is able to increase his/her system Privileges to a higher level than they should be. which could be a patch missing on the patch server. etc.9. permissions. have proven difficult to code properly. this type of attack can . Threat Eight: Insecure use of Cryptography Web applications often use cryptographi c functions to protect information and credentials. Insecure storage of credit cards. along with the code to integrate them. a test en gineer needs to walk through all the boxes to ensure that security patch update client software is installed and started. input validation (2) The web application communicates with a database server and (3) An attacker has access to the application through the Internet.8.11. For example. 3. Poor choice of algorithm (or invent your own) Poor randomness o Session IDs o Tokens o Cookies Improper storage in memory Countermeasures o Store only what is must o Store a hash instead of the full value (SHA-1) o Use only vetted. As part of deployme nt testing. and accounts o Set up logging and alerts 3. public cryptography 3. It's also a good idea to run a pa tch update scan to ensure no security patches are missing. often resulting in weak protection.

b) id s and passwords being transmitted over a network or stored on a server without encr yption. 3. Recent new vulnerabilities and attack methods discovered or reported show an al arming trend toward attacks with multi- faceted damages and even anti-forensics capabilities. As web applications become more pervasive and more complex.ciac. General consensus has pegged SQL Injection as the method used behind the massive compromise of credit card numbers in February of last year.2 could gain the ability to run arbitrary code with super user privileges. allowing an attacker to 'poison' the cookie. How do these Vulnerabilities Affect Your Customers Your customers can be affected in a variet y of ways: from identity theft to session hijacking to the compromise of confidenti al and private customer data. so do the techniques and attacks hackers are using against them. Threat Twelve: Identify Spoofing Identity spoofing is a technique where a hacker uses the cred entials of a legitimate user to gain access to an application or system . In this example. An example of such an attack can be found at http://www. 4. Many of the phishing email-based schemes use cross-si te scripting and other application layer attacks to trick users into giving up their credentials. This means hackers are using more powerful attacks to cause significantly more damage. We still see many cases where cookies aren't properly secu red. This can be a result of: a) users being careless with their ids and passwords. it attacks the user via a flaw in the website that enables the attacker to gain access to login and account data from the user.org/ciac/bulletins/m-026. or by the use of hardware tokens. he/she can login to the application with all of the privileges normally assigned to that user. 5. or c) users setting easy to guess passwords.12. Once a hacker has possession of a user's credentials.shtml.result in a hacker gaining privileges as high as root on a UNIX system. Cross-site Scripting is one of the leading methods used in identity theft (and an obvious concern to financial and healthcare institutions). The approach to be taken depends on the value of the data protected by the id and password. This threat can be reduced or eliminated by re quiring the use of strong pa sswords and forcing frequent password changes. hijack active sessions or manipulate hidden fields to defraud e-commerce sites. Security Testing: Approach Identify security threats and map with specific application Create Security Test Plan Write Security Test Cases Execute Security Test Cases after functional testing is completed Analyze test results Report security flaws to the developers Track to Closure 6. the entire system is effectively compromised. while at the same time covering their tracks is becoming easier. authorized users of versions of OpenSSH earlie r than 3.0. Some common mistakes in application security System Admin password is hard-coded in the application . SQL injection is one of the main atta cks used when backend databases are compromised. Once a hacker is able to run code with this level of privilege. by not storing or transmitti ng clear-text passwords.

login error messages. APIs. client function manipulation. communication protocols used between .Password is sent over the wire in unencrypted form Database roles are not used Permissions are given in an adhoc manner Application roles are not used Multiple applications run in a single machine Stored procedure level security is almost missing Stored procedures are not encrypted Errors triggering sensitive information leak SQL injections Back doors and debug options: Cross-site scripting Weak session management Insecure use of cryptography 7. Recommended areas of security testing S Test Area Description No 1 Authentication User ID complexity. 7 Server Side Web application execution environment. hidden form fields. caching. client Vulnerabilities code reverse engineering susceptibility. no customized error pages revealing useful information 10 Security of 3rd Party Single sign on. susceptibility to session hijacking/ session replay attacks 3 Information Leakage Review HTML page source code for: revision history. password policy 2 Authorization Cookie use and complexity. database Application interaction. lockout after x number of incorrect login attempts. internal proxying. internal host information. username harvesting. reusing older credentials to gain access 6 Cache Control No sensitive information stored locally. susceptibility to brute forcing. Control system calls. saving of passwords locally. cross site scripting. unauthorized Logic command execution 8 Client Side Software Code obfuscation. SQL injection. URL re-writing 5 Session Time-out and No back button. error messages 4 Field variable Buffer Overflow. no database or middleware errors that reveal software used. multiple logins Log-out allowed for single user. no tracking material stored that could replay session. client privilege escalation 9 Error Handling No 404 errors. middleware or content control Applications applications with published security flaws. comments. email addresses. cookie validation. tracking logic. client connection DOS.

3. 2. key management/ revocation. remote Administration access security 12 Use of Encryption Obfuscation. Finally. there needs to be criteria by which to measure the successes or failures of the procedures implemented. ports. scalability 8. it is imperative to get the right information to the right stakeholder. Risk. Web Application Security Action Plan A Web Application Security Process can be implemented using 3 key guidelines: 1. Similarly. secured integration of software into the Web environment. password storing. . Development needs to understand what these vulnerabilities and compliance exposures are in development terms. Using their existing methods and metrics they. There are several good sources both online and in security testing tools for developers. Performing security testing during the application development lifecycle at key points during the various stages from develo pment to QA to Staging will reduce costs and significantly reduce your online risk. secure configuration of software. Ultimately. can properly prioritize and monitor the defect remediation process as well as accurately assess release candidacy. Understand: Perform security audits and defect testing throughout the Application lifecycle. one must not forget that the application development lifecycle is the breeding ground for the defects that cause the risks. i. trend and regression analysis on the security defects just like they do for performance and functionality flaws. Organizations use trending and defect remediation analysis metrics to identify areas and issues to focus on. Production applications are an obvious first place to implement regular audits and analysis to determine security and compliance risk to an organization. QA must be able to perform delta. with the ever-increasing number and scope of Government and internal regulations and policies. Communicate: After risks and security defects have been identified. applications. Compliance and R&D need to communicate and validate application risks against these very real business drivers. This means providing details around how the attack works and guidance on remediation.e. there may be a certain security defect type that keeps cropping up which can then be identified and dealt with through targeted education and training to recognize repeated risks with a particular infrastructure product or vendor. along with Product Management. login credentials. At the same time. 11 Application Policy procedures. teams from Security. Measure: For any process to be successful. measuring and analyzing scan results will contribute to a reduction in liability and risk brought about by implementing a web application security plan.