1.

Quality Principles
A. Before an organization can begin to assess the quality of its products and services and identify opportunities for improvement, it first must have a working knowledge of quality principles. This category will test the CSQA candidate’s understanding and ability to understand and apply these principles.

B. Definitions of Quality: 1. Quality
i. Totality of characteristics of an entity that bear on its ability to satisfy stated and implied needs. The term "quality" should not be used as a single term to express a degree of excellence in a comparative sense, nor should it be used in a quantitative sense for technical evaluations. To express these meanings, a qualifying adjective should be used. ii. QUALITY - The degree to which a system, component, or process meets specified requirements, or customer or user needs or expectations. iii. QUALITY FACTORS - The characteristics used to formulate measures of assessing information system quality. iv. The New 2000 ISO 9000 Standards - The four primary standards are as follows:  ISO 9000: Quality management systems - Fundamentals and vocabulary  ISO 9001: Quality management systems - Requirements  ISO 9004: Quality management systems - Guidance for Performance Improvement  ISO 19011: Guidelines on Quality and Environmental Auditing v. Quality software is reasonably bug-free, delivered on time and within budget, meets requirements and/or expectations, and is maintainable. However, quality is obviously a subjective term. It will depend on who the 'customer' is and their overall influence in the scheme of things. A wide-angle view of the 'customers' of a software development project might include end-users, customer acceptance testers, customer contract officers, customer management, the development organization's management/accountants/testers/salespeople, future software maintenance engineers, stockholders, magazine columnists, etc. Each type of 'customer' will have their own slant on 'quality' - the accounting department might define quality in terms of profits while an end-user might define quality as user-friendly and bug-free. vi. In Quality Is Free, Phil Crosby describes quality as "conformance to requirements.” vii. J.M. Juran’s definition of quality. He spends a goodly portion of an early chapter in Juran on Planning for Quality discussing the meaning of quality, but he also offers a pithy definition: fitness for use. In other words, quality exists in a product—a coffee maker, a car, or a software system—when that product is fit for the uses for which the customers buy it and to which the users set it. A product will be fit for use when it exhibits the predominant presence of customersatisfying behaviors and a relative absence of customer-dissatisfying behaviors.

2. Producer’s View of Quality

i. A more objective view ii. Conformance requirements iii. Costs of quality (prevention, appraisal, scrap & rework, warranty costs)  Prevention costs: training, writing quality procedures  Appraisal costs: inspecting and measuring product characteristics  Scrap and Rework costs: internal costs of defective products  Warranty costs: external costs for product failures in the field
3. Customer’s View of Quality
i. A more subjective view ii. Quality of the design (look, feel, function) iii. Consider both feature and performance measures to asses value
____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 1

4.

 Value = quality / price (determined by individual customers) International Standards Organization document ISO 9126. i. This standard proposes that the quality of a software system can be measured along six major characteristics:  Functionality: Does the system provide the required capabilities?  Reliability: Does the system work as needed when needed?  Usability: Is the system intuitive, comprehensible, and handy to the users?  Efficiency: Is the system sparing in its use of resources?  Maintainability: Can operators, programmers, and customers upgrade the system as needed?  Performance: Does the system fulfill the users’ requests speedily?

C. Quality Concepts: 1. Cost of Quality
i. ii. iii. iv. Prevention costs – maintaining a quality system Appraisal costs – maintaining a quality assurance system Internal failures – manufacturing losses, scrap, rework External failures – warranty, repair, customer, product service Jim Campenella illustrates a technique for analyzing the costs of quality in Principles of Quality Costs. Campenella breaks down those costs as follows:

v.

 Cquality=Cconformance+Cnonconformance vi. Conformance costs include prevention costs and appraisal costs. Prevention costs include money
spent on quality assurance—tasks like training, requirements and code reviews, and other activities that promote good software. Appraisal costs include money spent on planning test activities, developing test cases and data, and executing those test cases once. Nonconformance costs come in two flavors: internal failures and external failures. The costs of internal failure include all expenses that arise when test cases fail the first time they’re run, as they often do. A programmer incurs a cost of internal failure while debugging problems found during her own unit and component testing The costs of external failure are those incurred when, rather than a tester finding a bug, the customer does. These costs will be even higher than those associated with either kind of internal failure, programmer-found or tester-found. In these cases, not only does the same process described for tester-found bugs occur, but you also incur the technical support overhead and the more expensive process of releasing a fix to the field rather than to the test lab. In addition, consider the intangible costs: angry customers, damage to the company image, lost business, and maybe even lawsuits. The flip side of the quality approach is Philip Crosby's “quality if free" idea. Basically, Crosby's thesis is that bad quality is very expensive. If you add up the costs of scrap, rework, delays in scheduling, the need for extra inventories to compensate for schedule changes, field service costs, product warranty expenses, and most of all customer dissatisfaction with your product, that can cost one heck of a lot of money. Companies need to get their arms around these costs of quality and quantify their impact. Most companies have estimated that their cost of quality is 25% to 35% of product cost. With that as the incentive, a firm can start to go to work and attack the root causes that result in those bad quality costs, reduce them, and end up at the same place as the Deming approach which is to produce high quality goods and eliminate screw-ups in the manufacturing process. They both have the same objective, to get the quality up throughout the whole process instead of waiting and inspecting the product at the end.

vii.

viii.

2. Plan-Do-Check-Act

i. A Problem Solving Process ii. The well known Deming cycle instructs us to Plan, Do, Study and then Act upon our findings, in order to obtain continuous improvement. For a software organisation this might be rephrased as "Plan the project, Develop the system, Scrutinise its implementation and Amend the process".
____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 2

iii. The continuous improvement uses a process that follows the plan-do-check-act cycle. The situation is analyzed and the improvement is planned (Plan). The improvement is tried (Do). Then data is gathered to see how the new approach works (Check or study) and then the improvement is either implemented or a decision is made to try something else (Act). This process of continuous improvement makes it possible to reduce variations and lower defects to near zero.

Plan – The Change

1) Step 1: Identify the Problem
i. ii. iii. iv. 2) Step 2: i. ii. iii. iv. v. vi. vii. viii. ix. Select the problem to be analyzed Clearly define the problem and establish a precise problem statement Set a measurable goal for the problem solving effort Establish a process for coordinating with and gaining approval of leadership Analyze the Problem Identify the processes that impact the problem and select one List the steps in the process as it currently exists Map the Process Validate the map of the process Identify potential cause of the problem Collect and analyze data related to the problem Verify or revise the original problem statement Identify root causes of the problem Collect additional data if needed to verify root causes

Do – Implement The Change
1) Step 3: Develop Solutions i. Establish criteria for selecting a solution ii. Generate potential solutions that will address the root causes of the problem iii. Select a solution iv. Gain approval and supporters of the chosen solution v. Plan the solution 2) Step 4: Implement a Solution i. Implement the chosen solution on a trial or pilot basis ii. If the Problem Solving Process is being used in conjunction with the Continuous Improvement Process, return to Step 6 of the Continuous Improvement Process iii. If the Problem Solving Process is being used as a standalone, continue to Step 5

Check – Monitor and Review The Change
1) Step 5: Evaluate The Results i. Gather data on the solution ii. Analyze the data on the solution

Act - Revise and plan how to use the learning’s
1) Step 6: i. ii. iii. iv. v. Standardize The Solution (and Capitalize on New Opportunities) Identify systemic changes and training needs for full implementation Adopt the solution Plan ongoing monitoring of the solution Continue to look for incremental improvements to refine the solution Look for another improvement opportunity

____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 3

The central idea behind Six Sigma is that if you can measure how many "defects" you have in a process.’ Decrease your process variation (remember variance is the square of your process standard deviation) in order to increase your process sigma. A Six Sigma defect is defined as anything outside of customer specifications. data-driven approach and methodology for eliminating defects (driving towards six standard deviations between the mean and the nearest specification limit) in any process -. v. iv. Six Sigma is a disciplined. to be produced.4 defect Parts Per Million (PPM) opportunities. The end result is greater customer satisfaction and lower costs. the variation of the process around the mean value decreases. As the process sigma value increases from zero to six. A Six Sigma opportunity is then the total quantity of chances for a defect. The objective of Six Sigma Quality is to reduce process output variation so that ±six standard deviations lie between the mean and the nearest specification limit. also known as Defects Per Million Opportunities (DPMO). Six Sigma is a highly disciplined process that helps us focus on developing and delivering nearperfect products and services. Six Sigma i. To achieve Six Sigma. With a high enough value of process sigma. What’s Involved In a Six Sigma Initiative? ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 4 . Process sigma can easily be calculated using a Six Sigma calculator. iii. a process must not produce more than 3. the process approaches zero variation and is known as 'zero defects. This will allow no more than 3.Plan-Do-Check-Act Cycle 3.from manufacturing to transactional and from product to service. Why "Sigma"? The word is a statistical term that measures how far a given process deviates from perfection.4 defects per million opportunities. you can systematically figure out how to eliminate them and get as close to "zero defects" as possible. ii. The statistical representation of Six Sigma describes quantitatively how a process is performing.

This process is to be repeated until we reach a defect level of 3. there are six distinctive roles in the implementation process. The challenge to leadership is to harness the ideas and energy of many people across functions. marketing. quality. The essence of this method centers on identifying problems. the philosophy is called the Six Sigma philosophy. They typically also manage the individuals who are selected for Black Belt training. executive managers provide overall leadership and must own and drive Six Sigma. 1) Define. This quantum leap in quality needs people to make it happen. 2) Measure. their requirements. sales. These tools apply a refined methodology of measurement and discovery to gain a comprehensive understanding of performance and key variables affecting the quality of a company’s products and services. Master Black Belts: These are the full-time trainers for a company’s Six Sigma efforts. the sigma level goes up. a senior management leader or leaders is assigned to provide day-to-day top management leadership during implementation. A three Sigma indicates a level of 66. Identify the key measures. 5) Control. Training these individuals to become Black Belts and “change agents” is critical to successful implementation of Six Sigma Problem Solving Technology.4 or lower for every million opportunities. 3) Analyze. Supervisory-Level Management: These managers play a pivotal role because they own the processes of the business and must ensure that improvements to the process are captured and sustained. Define the customers. However. As sigma increases. Six Sigma Champions: As a group. It can be applied to all disciplines: production. Achieving Six Sigma performance across an organization is an enormous challenge. determining their root causes. Develop. the team charter. Generate and determine potential solutions and plot them on a small scale to determine if they positively improve process performance. testing those improvements. and the key process that affects that customer. Implementing Six Sigma requires mobilizing people resources and arming them with the tools they need to accomplish the goal of quality improvement and impressive financial results.          Six Sigma is a philosophy to eliminate variation in process.4 only per one million opportunities. one must note that we cannot achieve a Six Sigma level in one go. 4) Improve. Listed below is a high level overview of the Six Sigma improvement methodology that various companies have used to practice its process improvement initiative. formulating ideas around what would result in improvement. Every time we improve the process. and maintaining improvement. These individuals are referred to as Champions. Pg 34. To become a Six Sigma company. service. From within this group. and must understand the challenges facing them as well as be willing and empowered to remove any roadblocks to progress. The Six Sigma Revolution. whereas 6 sigma brings defect levels to 3. Considering the enormous reduction in defect level and variations. They will grow them from the ranks ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 5 . They act as coaches and mentors for Black Belts. customer satisfaction goes up while at the same time cycle time goes down and costs plummet. and execute the plan for data collection.807 defects per one million opportunities. Going from Four to Six Sigma is almost a 2. George Eckes. While all employees need to understand the vision of Six Sigma and use some of its tools to improve their work. and even business groups. Analyze the data collected as well as the process to determine the root causes for why the process is not performing as desired. A level of Six Sigma represents the peak of quality — the virtual elimination of defects from every product and process within an organization. it takes more than technology. Six Sigma is the disciplined application of statistical problem-solving tools that show you where wasteful costs are and points you toward precise steps to take for improvement. sites. the Data Collection Plan for the process in question. document. and implement a plan to ensure that performance improvement remains at the desired level.000 percent improvement! No one person and no one area can accomplish this alone. knowledge and organization. Sigma is a metric that indicates how well the process is performing.

BENCHMARK . (Source ISO 9004: 1987. performance. 3) A measurement system.) i. 2) A cultural value or philosophy toward your works. they help Black Belts complete projects and extend the reach of Black Belts. ii. Quality Objectives: 1. 4.    of the Black Belts with the help of the Six Sigma partner’s team of experts. focuses on improving customer satisfaction through continuous and incremental improvements to processes.3. including by removing unnecessary activities and variations. Continuous improvement. (SWCMM (IEEE-STD-610) 5. Reduce development costs/improve time-to-market capability iii. that provide process and cross-functional knowledge. Quality Attributes: 1. Finding. In summary. ii. 6. For the corporate quality policy. Green Belts: As part-time resources. Project Team Members: These are the project. such as fitness for use. Improve Customer Satisfaction ii. When a Black Belt has access to the time and expertise of Green Belts. Benchmarking i. Continuous Improvement i. operational. and financial advantage. leading and facilitating the completion of Six Sigma projects. management should define objectives pertaining to key elements of quality. it allows the Black Belt to work on overlapping projects. Six Sigma is: 1) A measure of variation that achieves 3.4 defects per million opportunities. The process of identifying. the best Black Belts become the Master Black Belts. sharing.specific. The revisions of ISO 9001 and 9004 are based on eight quality management principles that reflect best management practices. thus completing more projects in a given period of time. It focuses on how to improve any given business process by exploiting top-notch approaches rather than merely measuring the best performance. ii. Best Practices i. studying and implementing best practices provides the greatest opportunity for gaining a strategic. “the teams”. These eight principles are:  Customer focused organization  Leadership  Involvement of people  Process approach  System approach to management  Continual improvement  Factual approach to decision making  Mutually beneficial supplier relationship D. safety and reliability. They help sustain the gains achieved by Six Sigma projects. To sustain a program. part-time people resources.A standard against which measurements or comparisons can be made. 4) A goal 4. Green Belts also work on smaller projects inside their functional areas. Improve Processes E.1. Reliability ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 6 . in regard to organizational quality and performance. and using knowledge and best practices. and eventually take 100 percent ownership of a Black Belt project. Black Belts: Full-time employees who are 100 percent focused on identifying. (ISO 9001) A complete cycle to improve the effectiveness of the quality management system.

Effort required to learn. product evaluation. maintain. and the system can produce the desired result correctly. Busy people prepare input and make mistakes in input preparation. Proper documentation of standards and procedures is necessary since the SQA activities of process monitoring. Outputs may be lost. 6. ii. Procedures are the established criteria to which the development and control processes are compared. and auditing rely upon unequivocal definitions to measure project compliance. They define legal language structures. control. Effort required to implement zero-defect functionality. Maintainability i. misdirected. On the other hand. All of these affect the correctness of the application results. 2. style conventions. ii. Flexibility i. if an input transaction is entered perfectly. that same system which processed using imperfect input may fail to produce correct results. and application support personnel perspectives. etc. Interoperability i. Effort required to implement new enhancements or fix operational errors. The characteristics of software that allow or enable adjustments or other changes to the business process. For example. the SQA role is to ensure their existence and adequacy.  Code Standards specify the language in which the code is to be written and define any restrictions on use of language features. production control. They provide rules and methods for translating the software requirements into the software design and for representing it in the design documentation. 4. operate. instructions are unknown. and internal code documentation ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 7 . The NASA Data Item Descriptions (DIDs) are documentation standard. Thus. such as using wrong program versions. The ability of software and hardware on different machines from different vendors to share data.i. not including all of the input or adding input which should not be included.Automated applications are not run in a sterile environment. Correctness i. and users of systems experiment with input transactions. and misinterpreted by the people that receive them. and product documentation and provide consistency throughout a project. Extent to which a system or release can be expected to perform its intended function with required precision and without interruption to execution and delivered functionality. Establishing standards and procedures for software development is critical.  System versatility (or system robustness) is the capability of the system to allow flexible procedures to deal with exceptions in processes and procedures. then the correctness quality factor would be rated perfect. The reliability factor measures the consistency with which the system can produce correct results. ii. 5. Effort required to couple or interface one system with another. 3.  Documentation Standards specify form and content for planning. People operate the system and make mistakes. since these provide the framework from which the software evolves. Standards are the established criteria to which the software products are compared.  System adaptability is the capability to modify the system to cope with major changes in business processes with little or no interruption to business operations. Standards and procedures establish the prescribed methods for developing software. reliability may score low. RELIABILITY . Forms are misinterpreted. Standardization i. while correctness would score high. Effort and response time required to enhance an operational system or program.  Design Standards specify the form and content of the design product. rules for data structures and interfaces. mislaid. ii. and test the system or project enhancement from user. Conformance to accepted software standards to include additional enhancements to the standards.

performance. Testability i. Usability i. 11. 10. Scalability i. like the other quality factors. ii. Those attributes of the software that provide for useful inputs and outputs which are readily assimilated. comprehension. pull-down menus. or any user-operated device.  CLARITY: The screen layout needs to be clear and uncluttered. The resources that need to be utilized to test the system to ensure the specified quality has been achieved. Usability: To create good user interfaces. The wording should be considered carefully. An application is portable across a class of environments to the degree that the effort required to transport and adapt it to a new environment in the class is less than the effort of redevelopment. 12. That attribute of a computer system that characterizes the timeliness of the service delivered by the system. function keys. Usability is the measure of the quality of a user's experience when interacting with a product or system — whether a Web site. Quality Metric Criteria: Criteria Accuracy Clarity Quality Measurement Those attributes of the software products that provide the required precision in calculations and output products and fully meet the functional. Hyper documents. and so on.7.  COMPREHENSION: Developing on-line documentation. Pictures may also be extremely helpful. an extension of hypertext. and consistency. a software application. the amount of resources allocated to testing can vary based on the degree of reliability that the user demands from the project. They should be used where appropriate to make applications easy to learn and use. Usability is a combination of factors that affect the user's experience with the product or system. should be discussed and negotiated with the user responsible for the application. Effort required to prepare for and test a system or program to assure it performs its intended functionality without degraded operational performance. Security 14. ii. The choice of words and colors should be consistent. there should be consistent use of screen layouts. but to actually take full advantage of it. 9. Performance i. Portability i. mobile technology. HELP. Availability 13.  (1) The degree to which a system or component facilitates the establishment of test criteria and the performance of tests to determine whether those criteria have been met. It is the ability not only to function well in the rescaled situation. 8.  CONSISTENCY: A new application should look and feel as familiar as possible to users. Testing. However. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 8 . multiple-choice mechanisms. are also an appropriate form of on-line documentation. It is the ability of a computer application or product (hardware or software) to continue to function well as it (or its context) is changed in size or volume in order to meet a user need.  (2) The degree to which a requirement is stated in terms that permit establishment of test criteria and performance of tests to determine whether those criteria have been met (IEEE-STD-610). and operational requirements. and tutorial explanations requires the ability to write clear English and avoid jargon. attention must focus on clarity. ii.

ease of maintenance. Pioneer of Modern Quality Control. Quality Pioneers: 1. ISO 9000 Definitions: i. Quality Assurance  All those planned and systematic activities implemented to provide adequate confidence that an software package will fulfill requirements for quality 1) Data Gathering  Problem Trend Analysis  Process Identification  Process Analysis  Process Improvement 2) Focus on Process. Walter A. lending itself to simplicity and modularity.A planned and systematic pattern of all actions necessary to provide adequate confidence that a software work product conforms to established technical requirements G.Communications Those attributes of the software that provide the use of commonality standard protocols and interface routines. Prevent Defects iii. Proactive. Expandability Modularity Those attributes of the software products that provide for increasing. and future expansion. Conciseness Consistency Data commonality Encapsulation Error Tolerance Execution efficiency Those attributes of the software that provide for minimum execution processing time without decrease in functionality. Software objects that protect themselves and their associated data. Simplicity Timeliness Those attributes of the software products that provide maintenance and implementation of the functions in the most understandable manner. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 9 . Quality Control  The operational techniques and activities that are used to fulfill requirements for quality 1) Problem Identification  Problem Analysis  Problem Correction & Feedback to QA 2) Focus on Product. Quality Assurance vs. Those attributes of the software products that provide for simplicity and completeness in presentation and for implementation of a function with a minimum amount of code. For Year 2000 projects. eliminating random effects on other objects in the same system when encountering error conditions. Self-descriptiveness Those attributes of the software that provide explanation of the maintenance and implementation of a function. programming techniques. JCL coding. F. Line Function. Those attributes of the software products that are delivered on time or run on schedule. Those attributes of the software that provide the use of standard data representations and structures. Consistency is measured for report and screen formats. etc. lending itself to simplicity. Those attributes of the software products that provide a structure of highly independent modules with each serving a particular function and accordingly. Those attributes of the software products that provide continuity of operation under non-nominal conditions. Shewart i. ii. changing. Recognized the need to separate variation into assignable and unassignable causes. SOFTWARE QUALITY ASSURANCE . Those attributes of the software products that provide uniform design and implementation techniques. Communicativeness Those attributes of the software products that provide useful inputs and outputs that can be assimilated. Find Defects ii. also those attributes of the software products that can be delivered before the event horizon. Quality Control: 1. Staff Function. and customizing functionality. Reactive.

Founder of the control chart. engineering. 8) Eliminate numerical goals.  Main contribution is his Fourteen Points to Quality. warranty. 14) Put everybody in the company to work to accomplish the transformation. 2) Adopt the new philosophy. 2) System for causing quality is prevention not appraisal. 2. W. 4. Perhaps the first to successfully integrate statistics. 4) Measurement of quality is the cost of nonconformance. Kaoru Ishikawa i. Joseph Juran i. Philip Crosby i. Quality management  Four absolutes of quality including: 1) Quality is defined by conformance to requirements. Costs of quality may be separated into costs for prevention. 11) Institute modern methods of training. Defined quality in terms of objective and subjective quality. and economics. etc. 3) Cease dependence on mass inspection to achieve quality. v. We are in a new economic age. Arman Feigenbaum i. 7) Break down barriers between departments.  Enlightened the world on the concept of the “vital few. vi. appraisal. not close enough.  objective quality: quality of a thing independent and subjective quality.  subjective quality: quality relative to how people perceive it. 10) Institute modern methods of supervision. 5.  Directed most of his work at executives and the field of quality management. Stressed a systems approach to quality (all organizations must be focused on quality) ii. Developed concept of true and substitute quality characteristics  True characteristics are the customer’s view  Substitute characteristics are the producer’s view  Degree of match between true and substitute ultimately determines customer satisfaction ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 10 . Originator of the plan-do-check-act cycle. Contributions:  Also well known for helping improve Japanese quality. trivial many” which is the foundation of Pareto charts. Edwards Deming i. 6) Drive out fear. and failures (scrap.) 6. 5) Remove barriers. Contributions:  Well known for helping Japanese companies apply Shewart’s statistical process control. 13) End the practice of awarding business on price tag. Studied under Shewart at Bell Laboratories ii. 9) Eliminate work standards (quotas). quality control. and quality improvement.iii. 3) Performance standards of zero defects.  Developed the “Juran Trilogy” for managing quality: 1) Quality planning. The 14 points are: 1) Create constancy of purpose toward improvement of product and service. iv. 12) Institute a program of education and retraining. 3. 4) Constantly and forever improve the system.

 Respect for humanity as a management philosophy – full participation 7. based on the participation of all of its members. objectives. It maintains that organizations must strive to continuously improve these processes by incorporating the knowledge and experiences of workers.  Next process is your customer. TQM (Total Quality Management) i. engineering. iii. and aiming at long-term success through customer satisfaction and benefits to all members of the organization and to society. improvement. 1960s – 1980s ii. Quality planning covers product planning. quality assurance. All activities of the overall management function that determine the quality policy. Quality Management i. Improvements can be effected in two ways:  By improving the process itself  By improving the outcomes of the process. Quality Improvement Cycle i. ii. Advanced the use of quality circles (worker quality teams) iv. and implement them by means such as quality planning. centered on quality. A quality improvement cycle is a planned sequence of systematic and documented activities aimed at improving a process. Quality Planning i. Genichi Taguchi i. quality control.ii. continuous improvement. The activities that establish the objectives and requirements for quality and for the application of quality system elements. It views organizations as a collection of processes. ii. Total quality management is the management approach of an organization. managerial and operational planning. Quality loss function (deviation from target is a loss to society) iii. A management philosophy which seeks to integrate all organizational functions (marketing. and the preparation of quality plans. Quality Vocabulary: 1. Total Quality Management is a structured system for satisfying internal and external customers and suppliers by integrating the business environment.  focus is on “engineering design”  robust design/parameter design H. customer service …) to focus on meeting customer needs and organizational objectives. aimed at continually improving performance over the long term by focusing on customers while addressing the needs of all stakeholders. for leading and operating an organization. 2. Promoted the use of parameter design (application of Design of experiments) or robust engineering  Goal: develop products and processes that perform on target with smallest variation that are insensitive to environmental conditions. Advocate of the use of the 7 tools iii. ii. production. and responsibilities. and quality improvement within the quality system. ii. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 11 .  Use facts and data to make presentations. 4. finance. A comprehensive and fundamental rule or belief. design. Quality planning embodies the concepts of defect prevention and continuous improvement as contrasted with defect detection. 3. and breakthroughs with development. Developed the concept of Japanese Total Quality Control  Quality first – not short term profits. and maintenance cycles while changing organizational culture.

5. It involves techniques that monitor a process and eliminate causes of unsatisfactory performance at all stages of the quality loop. Operational techniques and activities that are used to fulfill requirements for quality. 2. if this particular stuff is broken then whatever you're testing fails. in effect. Quality control is a formal (as in structured) use of testing. and the company agrees to correct any problems within a specific time.. iii. quality assurance is any systematic process of checking to see whether a product or service being developed is meeting specified requirements. parts. AQP includes the methods and controls (i. although it often used synonymously with testing. ii. 6. The organizational structure. tests) that will be used in the design and production of a specific product or family of products (i. reduction in the total product development cycle. making sure that any agreed-upon standards and procedures are followed. procedures. It is oriented to 'prevention'. A detailed report describes the parts of the standard the company missed. Today's quality assurance systems emphasize catching defects before they get into the final product. A quality assurance system is said to increase customer confidence and a company's credibility. processes. Many companies have a separate department devoted to quality assurance. Quality Assurance i. Roughly. the company is certified as in conformance with the standard. Conformance to ISO 9000 is said to guarantee that a company delivers quality products and services. Quality Function Deployment (QFD) . measurements. and ensuring that problems are found and dealt with.a planning tool for incorporating customer quality requirements through all phases of the product development cycle.e. The planned and systematic activities implemented within the quality system and demonstrated as needed to provide adequate confidence that an entity will fulfill requirements for quality. Software Development.e. Software QA involves the entire software development PROCESS . In developing products and services. Key benefits to this approach are product improvement. Next. Quality System i. iii. and resources needed to implement quality management. To follow ISO 9000. Acquisition and Operation Processes ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 12 . to improve work processes and efficiency. Quality control describes the directed use of testing to measure the achievement of a specified standard. ii. and increased market share. Quality assurance was initially introduced in World War II when munitions were inspected and tested for defects after they were made. Quality control is a superset of testing. Quality Control i. iii. The concept of quality control in manufacturing was first advanced by Walter Shewhart. a company's management team decides quality assurance policies and objectives.. A system of management which assures that planning is carried out such that ALL staff know what is expected and how to achieve the specified results. the company or an external consultant formally writes down the company's policies and requirements and how the staff can implement the quality assurance system. you test to see if something is broken. Once this guideline is in place and the quality assurance procedures are implemented. an outside assessor examines the company's quality assurance system to make sure it complies with ISO 9000.monitoring and improving the process.iii. increased customer satisfaction. Once the problems are corrected. and with quality control you set limits that say.  ISO 9000 is an international standard that many companies use to ensure that their quality assurance system is in place and effective. ii. materials). Advanced (Product) Quality Planning (AQP / APQP) is a structured process for defining key characteristics important for compliance with regulatory requirements and achieving customer satisfaction. 7. and to enable a company to better compete with others.

Software Development. the development process is invoked to effect and complete the modifications properly. The operation of the software is integrated into the operation of the total system. or other. A supporting process supports any other process as an integral part with a distinct purpose and contributes to the success and quality of the project. Software architectural design. Operation and Maintenance Processes i. and Software retirement.1. Software qualification testing. supply. by the acquisition. as needed. Process Knowledge a. The objective is to modify an existing system while preserving its integrity. `This life cycle process contains the activities and tasks of the developer of software. System design. These activities and tasks may be used to construct one or more developmental models (such as. ISO 12207 – Software Life-Cycle standard iii. or the need for an improvement or adaptation. These activities may be iterated and overlapped. evolutionary. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 13 . Software integration. v. The maintenance process contains the activities and tasks of the maintainer. operate and maintain software systems. The development process consists of the following activities along with their specific tasks: Process implementation. vi. This process is activated when a system undergoes modifications to code and associated documentation due to an error. the Spiral. Whenever a software product needs modifications. migration. incremental. This process consists of the following activities along with their specific tasks: Process implementation. A supporting process is invoked. iv. Modification implementation. The process covers the operation of the software and operational support to users. Operational testing. System operation. All the tasks in an activity need not be completed in the first or any given iteration. Development Process (ISO 12207) 1. The term development denotes both development of new software and modification to an existing software. This process consists of the following activities along with their specific tasks: Process implementation. Understanding the processes used in the organization to develop. a deficiency. a problem. and Software acceptance support. The process ends with the retirement of the system. but these tasks should have been completed as the final iteration comes to an end. or a combination of these) for a project or an organization. Operation Process (ISO 12207) 1. Problem and modification analysis. This life cycle process contains the activities and tasks of the operator of a software system. Supporting Processes (ISO 12207) 1. System integration. This process provides for developing software as a stand-alone entity or as an integral part of a larger. and User support. The positional sequence of these activities does not necessarily imply a time order. System requirements analysis. Software coding and testing. ii. or an activity may be recursed to offset any implied or default Waterfall sequence. Software installation. The CQA candidate must understand how quality software is built to be effective in assuring and controlling quality throughout the software life cycle 2. System qualification testing. Maintenance Process (ISO 12207) 1. total system. Maintenance review/acceptance. Software detailed design. the Waterfall. The development process is intended to be employed in at least two ways: (1) As a methodology for developing prototypes or for studying the requirements and design of a product or (2) As a process to produce products. Software requirements analysis.

Configuration control. it supplements them. Process assurance.This is a process for recording information produced by a life cycle process. design. as-built system fulfills its specific intended use. Configuration evaluation.  Validation Process . This process consists of: Process implementation.  Configuration Management Process . or another supporting process. design. requirements. The reviews are at both management and technical levels.  Verification Process . integration. The process covers verification of process. and baseline software items in a system. They may as well be the acquirer and the supplier respectively. engineers and users of the system. Verification does not alleviate the evaluations assigned to a process. At a joint review.  Audit Process . and Maintenance. Validation does not replace other evaluations. contractually established audits of a supplier's products or services. and to control storage. Production. Design and development.  Problem Resolution Process . An audit may well be conducted by the acquirer on the supplier.This process provides the evaluations related to verification of a product or service of a given activity. and Assurance of quality systems. define. but supplements them. operation or maintenance process. to record and report the status of the items and modification requests. software quality assurance is provided with the organizational freedom from persons directly responsible for developing the products or providing the services. the process requires identification and analysis of causes ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 14 .Validation determines whether the final. handling and delivery of the items. Configuration status accounting.development.This process provides the framework for independently and objectively assuring (the acquirer or the customer) of compliance of products or services with their contractual requirements and adherence to their established plans. develop. edit. To be unbiased. Configuration identification.  Documentation Process . to ensure the completeness and correctness of the items. and documentation. The four activities along with their tasks are: Process implementation. Verification determines whether the requirements for a system are complete and correct and that the outputs of an activity fulfill the requirements or conditions imposed on them in the previous activities.This process provides the mechanism for instituting a closed-loop process for resolving problems and taking corrective actions to remove problems as they are detected. and Release management and delivery. distribute and maintain those documents needed by all concerned such as managers. The process defines the activities. Product assurance.  Joint Review Process . This process consists of: Process implementation.This process provides the framework for interactions between the reviewer and the reviewee. code. the auditor assesses the auditee's products and activities with emphasis on compliance to requirements and plans.This process provides the framework for formal. to control modifications and releases of the items. the reviewee presents the status and products of a life cycle activity of a project to the reviewer for comment (or approval). At an audit. on the contrary. which plan.This process is employed to identify. In addition. The extent of validation depends upon the project's criticality.  Quality Assurance Process .

An organizational process may support any other process as well. b. tools. cannot be tailored. or supporting process. and maintenance for increasing productivity. activities. objectives. The infrastructure may include hardware. that is not contained in the standard but is pertinent to a project. The activities cover: Process establishment. an activity.The standard provides the basic. however. techniques. or a supporting process) needs to assess. Tools i. This process has the following activities: process implementation. such as the acquisition process. control. Process assessment. A process. have similar management activities. and improve its life cycle process.This process defines the activities needed for establishing and maintaining an underlying infrastructure for a life cycle process. The term "problem" includes non-conformance. and Maintenance of the infrastructure. Experiences from application of the life cycle processes on projects are used to improve the processes. viii.  Infrastructure Process . operation. vii. Tailoring in the standard is deletion of non-applicable or in-effective processes. and methods of operations. typically beyond or across projects. Therefore. and Process improvement. The process requires that a training plan be developed. For example. and training be provided to the personnel in a timely manner. estimating. The objectives are to improve the processes organization-wide for the benefit of the organization as a whole and the current and future projects and for advancing software technologies. controlling. and facilities. supply.  Improvement Process . development operation. standards. Organizational Processes (ISO 12207) 1.This process defines the generic activities and tasks of the manager of a software life cycle process. Tailoring Process (ISO 12207) 1.  Training Process . acquisition. development. Even though. Establishment of the infrastructure. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 15 . the primary processes. software. maintenance. operation process. or a task. each primary is an instantiation (a specific implementation) of the management process.and reversal of trends in the reported problems. The organization establishes these activities at the organizational level.This process may be used for identifying and making timely provision for acquiring or developing personnel resources and skills at the management and technical levels. and improving other processes. These processes help in establishing. in general. and associated tools. It should be noted that this process itself. supply process. An organization employs an organizational process to perform functions at the organizational. The activities cover: Initiation and scope definition. corporate level. analysis. training material be generated. This standard contains a set of four organizational processes. Review and evaluation. Planning. they are sufficiently different at the detailed level because of their different goals. configuration management. and Closure.  Management Process . The standard requires that all the parties that will be affected by the application of the standard be included in the tailoring decisions. maintenance process. top-level activities that an organization (that is. Execution and control. Application of tools and methods that aid in planning. and tasks. measure. may be included in the agreement or contract.

The assurance control objective as it relates to configuration management of trusted systems is to "guarantee that the trusted portion of the system works only as intended. standards. configuration control. Finally. as well as the trust placed in a trusted system. and related data in their ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 16 .  Software development libraries provide for proper handling of software code. and software/firmware to review and approval by an authorized authority. Configuration status accounting is responsible for recording and reporting on the configuration of the product throughout the change. status accounting. software modules. configuration identification. and procedures. The CM activities monitored and audited by SQA include baseline control. 3.  Software configuration authentication is established by a series of configuration reviews and audits that exhibit the performance required by the software requirements specification and the configuration of the software is accurately reflected in the software design documents. SQA also monitors and audits the software library. and delivery is compatible with the associated documentation. control.  Configuration status accounting is performed accurately including the recording and reporting of data reflecting the software's configuration identification. or changes made to the TCB do not compromise the trust of the originally evaluated system. media. SQA assures that:  Baselines are established and consistently maintained for use in subsequent baseline development and control."[1] Procedures should be established and documented by a configuration management plan to ensure that configuration management is performed in a specified manner. 1. Any deviation from the configuration management plan could contribute to the failure of the configuration management of a system entirely.  Configuration control is maintained such that the software configuration used in critical phases of testing. and associated software documents. consistent with the security policy of the system. Configuration management provides assurance that additions. hardware. 2. proposed changes to the configuration identification. The purpose of configuration management is to ensure that these changes take place in an identifiable and controlled environment and that they do not adversely affect any properties of the system. It accomplishes this by providing procedures to ensure that the TCB and all documentation are updated properly. acceptance. and auditing. Software Quality Assurance (SQA) assures that software Configuration Management (CM) activities are performed in accordance with the CM plans. deletions. the completed change can be verified to be functionally correct. CONFIGURATION MANAGEMENT (CM) .  Software configuration identification is consistent and accurate with respect to the numbering or naming of computer programs. or in the case of trusted systems. configuration status accounting. the design and requirements of the changed version of the system should be identified. For every change that is made to an automated data processing (ADP) system. and for trusted systems. do not adversely affect the implementation of the security policy of the Trusted Computing Base (TCB). Configuration management is a sound engineering practice that provides assurance that the system in operation is the system that is supposed to be in use.Configuration management consists of four separate tasks: identification. software units. SQA reviews the CM plans for compliance with software CM policies and requirements and provides follow-up for nonconformances. The control task of configuration management is performed by subjecting every change to documentation. and the implementation status of approved changes. through the process of a configuration audit.ii. documentation. SQA audits the CM functions for adherence to standards and procedures and prepares reports of its findings. and configuration authentication.

2. there are many other needs the right estimating tool can address for your organization. 3. At a minimum. and advanced workspace and build management.Long before software process improvement and the CMM were common vocabulary in the software world. analogies comparing input parameters to existing knowledge bases. Using the same proven technology. From small project teams to the global enterprise. The process that drives inputs to outputs is either cost estimating relationships derived from regression of actual data. Rational's SCM solution frees teams of all sizes to build better software faster. A sound process will improve quality. data disasters. Rational has the right size solution for your team. or desktop) 5. The operating platform of the software (commercial or military. Rational's SCM solution helps you manage complex change throughout the development lifecycle. Unite your team with a process that optimizes efficiency Process is critical to streamlining software development. Software project planning is really a balancing act between cost. ground. Approved changes to baselined software are made properly and consistently in all products. a best practices process for managing change at the activity level and controlling workflow. CM Tool Examples: i. you'll be able to select the right product today and seamlessly grow with the product tomorrow – no conversion headaches. there was wide spread recognition that software project managers needed better ways to estimate the costs and schedules of software development projects. or some other sizing metric) 2. a robust software artifact management tool. or process changes. increase development speed and ultimately enhance overall team collaboration and productivity. operating systems. version control. yet error-prone tasks associated with software development. air. Software cost-estimating tools solicit input from the users describing their software project and from these inputs the tool will derive a cost (and usually schedule) estimate for that project. space. Rational ClearCase®. schedule. The anticipated amount of reuse 3. 1. and no unauthorized changes are made. c. A quantification of the organization’s software development productivity Although cost and schedule estimates are the main deliverable of the softwareestimating tool. The type of software being developed (real time. 1. Just smooth scalability ii.etc. creates a software configuration management (SCM) solution that helps your team handle the rigors of software development. ESTIMATING . combined with Rational ClearQuest®. various forms and versions from the time of their initial approval or acceptance until they have been incorporated into the final media. By automating many of the necessary. web development. Share code effortlessly and automate error-prone processes Rational's SCM solution offers the essential functions of transparent code sharing. freeing your team from tedious tasks that inhibit productivity. In the early 70’s two concurrent research efforts resulted in two parametric software cost-estimating models available to the software development community (COCOMO and PRICE S). Function Points (FPs). algorithms derived from theoretical research. or some combination of these methodologies. Choose a solution that scales and make it your last SCM decision Rational has an SCM solution that meets the needs of all size development teams. quality and content and the right software-estimating tool can help optimize ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 17 . The size of the software (either in source lines of code (SLOC). Rational's SCM solution offers Unified Change Management (UCM).) 4. IS. the cost estimating tools ask for the user to describe. iii. the most flexible defect and change tracking tool on the market. processes and protocols.

these submodels are called the Applications Composition. rather than just development costs. It also allows a planner to easily perform "what if" scenario exploration. had been implemented in a calibrated software tool. PRICE S – A tool is distributed by Lockheed . the result is COCOMO II. CERs were determined by statistically analyzing completed projects where product characteristics and project information were known. unless otherwise explicitly indicated. It consists of three submodels. and risk analysis factors. software reuse. Project Management ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 18 . 2. software development techniques changed dramatically. Until recently. In the ensuing decade and a half. each one offering increased fidelity the further along one is in the project planning and design process. and the COCOMO II Project Affiliate Organizations. this balance. Other inputs include software function. and Postarchitecture models. and reflected the software development practices of the day. effort. This offers the project manager the capability to make trade-offs based on the total cost of ownership. complexity factors. COCOMO II is a model that allows one to estimate the cost. only the last and most detailed submodel. Many tools offer the capability ofestimating latent defects in the delivered product and then use this information to predict maintenance costs. This new. Most tools have other trade-off and analysis features as well – allowing the user to set a baseline and vary different parameters to optimize cost and schedule. operating environment.g. all further references on these web pages to "COCOMO II" or "USC COCOMO II" can be assumed to be in regard to the Post-architecture model. Successful use of the PRICE S tool depends on the ability of the user to define inputs correctly. The implemented tool provides a range on its cost. Software size may be input directly. Early Design. 1.. These changes and others began to make applying the original COCOMO model problematic. Barry Boehm in 1981. a greatly increased emphasis on reusing existing software and building new systems using off-the-shelf software components. and staffing might have on predicted costs and schedules (e. and is considered one of the first complex commercially available tools used for software estimation. by quickly demonstrating the effect adjusting requirements. v.The original COCOMO model was first published by Dr. Listed in increasing fidelity. or automatically calculated from quantitative descriptions (function point sizing). and schedule estimates. Constructive Cost Model (COCOMO) . productivity factors. and schedule when planning a new software development activity.Martin PRICE Systems. This tool was first developed in 1977. for risk management or job bidding purposes). IRUS at UC Irvine. a revised cost estimation model reflecting the changes in professional software development practice that have come about since the 1970s. or developed with expert judgment. from best case to most likely to worst case outcomes. After several years and the combined efforts of USC-CSE. These changes included a move away from mainframe overnight batch processing to desktop-based real-time turnaround. A major input to PRICE S is Source Lines of Code (SLOC). resources. effort. However. As such. and spending as much effort to design and manage the software development process as was once spent creating the software product. Another important feature that most cost-estimating tools deliver is the ability to perform a Risk Analysis.iv. so that a confidence level can accompany your estimate. The PRICE S tool is based on Cost Estimation Relationships (CERs) that make use of product characteristics in order to generate estimates. The equations used by this tool are proprietary. descriptions of the methodology algorithms used can be found in papers published by PRICE Systems. Post-architecture. It can be customized and calibrated to the needs of the user d. The solution to the problem was to reinvent the model for the 1990s. improved COCOMO is now ready to assist professional software cost estimators for many years to come.

Acquisition Process (ISO 12207) .  The presentation of machine information to the human. is now being deployed to help organizations manage all types of change. e. support. The first three activities occur prior to the agreement. ii. 1. and documentation prepared in identifying. Design i. The acquisition process begins with the definition of the need to acquire a software product or service.. The acquirer represents the needs and requirements of the users. Project management. This process consists of the following activities along with their specific tasks: Initiation. Roles/Responsibilities a. Requirements i. Request-for-Proposal preparation. Preparation of response. tasks. Acquisition i. that contractually acquires software product or service. including development and execution of plans through delivery of the service to the acquirer. The process may be initiated either by a decision to prepare a proposal to answer an acquirer's request for proposal or by signing and entering into a contract or an agreement with the acquirer to provide a software service. The organization having the need for a product or service may be the owner. Obtaining software through purchase or contract f. b. It's really the management acumen that oversees the conversion of "vision" into "reality". in defining the automated solution to satisfy the business requirements & interfaces. Contract. the last five after the agreement. Program Interfaces ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 19 . 3. Contract preparation and update. and recording the business needs and problems to be resolved by the new or enhanced system. The service may be the development of a software product or a system containing software. The process continues with the preparation and issuance of a request for proposal. Review and evaluation. techniques used. Performing tasks to manage and steer a project toward a successful conclusion. while traditionally applied to the management of projects. implement. The supply process consists of the following activities along with their specific tasks: Initiation. Also. 3.g. and management of the acquisition process through the acceptance of the system. as well as the input/output facilities. or the maintenance of a software product. and maintain software systems. and Delivery and completion. 2. and the human interaction with the machine. This life cycle process contains the activities and tasks of the supplier. the last two after the agreement. The first two activities occur prior to the agreement. Execution and control. techniques used. Planning. test. ii. Project management is the discipline (art and science) of defining and managing the vision. Tasks performed. Supply Process (ISO 12207) i. The process continues with the identification of procedures and resources needed to manage and assure the service. and resources required to complete a project. The owner may contract all or parts of the acquisition tasks to an agent. Supplier monitoring. selection of a supplier. and Acceptance and completion. Communications Interfaces  That include transmission of information between computers and remote equipment (e. Person/Machine Interfaces  Interfaces that include the operating system and the development languages that are available. prioritizing. transmission of computer data over networks).i. and documentation prepared. to assess the testability of requirements. the operation of a system with software. document.This life cycle process defines the activities and tasks of the acquirer. Understanding the documents developed in the tester’s organization to design. Tasks performed.

Principle 4 Process approach  A desired result is achieved more efficiently when activities and related resources are managed as a process. 2. 6. iv.  Through management review. or distributed across multiple tiers of the application architecture. These eight quality management principles are defined in ISO 9000:2000. Principle 7 Factual approach to decision making  Effective decisions are based on the analysis of data and information. Build and Install  Tasks performed. including installation of software. continually improve the effectiveness of the Quality Management System. Curran and Sanders indicate that this quality process must adhere to four basic principles: ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 20 . 5. Quality management systems Guidelines for performance improvements: 1. whether on the same computer. Principle 5 System approach to management  Identifying.  c. Quality management systems Fundamentals and vocabulary. 3. Quality Principles i. Software Quality: A framework for success in software development and support. ii. 4. and documentation prepared in building the automated solution to satisfy the business requirements. increase functionality (enhancement). techniques used. Interfaces for the exchange of information. 5. internal/external audits and corrective/preventive actions. Principle 6 Continual improvement  Continual improvement of the organization's overall performance should be a permanent objective of the organization.4. Principle 8 Mutually beneficial supplier relationships  An organization and its suppliers are interdependent and a mutually beneficial relationship enhances the ability of both to create value. 7. Principle 1 Customer focus  Organizations depend on their customers and therefore should understand current and future customer needs. Principle 3 Involvement of people  People at all levels are the essence of an organization and their full involvement enables their abilities to be used for the organization's benefit. Understanding the tenets of quality and their application in the enterprise’s quality program. and in ISO 9004:2000. should meet customer requirements and strive to exceed customer expectations. 8. meet changing operating environment conditions (adaptation). In their book. Principle 2 Leadership  Leaders establish unity of purpose and direction of the organization. understanding and managing interrelated processes as a system contributes to the organization's effectiveness and efficiency in achieving its objectives. Maintenance  Software modification activities performed on an operational system to resolve problems (correction). or improve operational efficiency of speed. iii. This document introduces the eight quality management principles on which the quality management system standards of the revised ISO 9000:2000 series are based. They should create and maintain the internal environment in which people can become fully involved in achieving the organization's objectives.

walkthroughs. establish a formal method of accumulating and disseminating lessons learned from past experiences and mistakes.e. and technical reviews. the system test is carried out on the basis of the results specification phase. ii. you have in effect improved the process (and recall that continuous process improvement is another key tenet in how Total Quality Management principles are applied to quality software). independent quality auditing to ensure standards and procedures are followed. Therefore. and to all key development products such as requirements. as the longer the errors go undetected. the more expensive they are to correct. At least as much effort should be placed in keeping defects out of the code as detecting their presence in the code. By the ordering of activities in time sequence and with abstraction levels the connection between development and test activities becomes clear. For example. quality controls must be put in place during all stages of the development life cycle.1. 2. The 'V' is also a synonym for Verification and Validation.) server as a base for test activities. as well as audits at the project level which will determine if project activities were carried out in accordance to the standards and procedures established in the quality process. “V” Model ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 21 . The "V" concept relates the build components of development to the test components that occur during that build phase. The "V" Concept of Software Development: i. 4. Eliminate the causes as well as the symptoms of the defects. Prevent defects from being introduced. Methods for doing this include the use of appropriate software engineering standards and procedures. designs. Ensure that defects are detected and corrected as early as possible. 3. and lastly… Independently audit the work for compliance with standards and procedures. documentation and code. d. Many of the process models currently used can be more generally connected by the 'V' model where the 'V' describes the graphical arrangement of the individual phases. This is an extension of the previous principleóremoval of the defect without eliminating the cause is not a satisfactory way to solve the problem. high quality inputs such as software tools and subcontracted software. By removing the cause. and whether those standards and procedures are adequate to ensure the quality of the project in general. These should all be subjected to rigorous review methods such as inspections. This is a two part audit conducted at the process level using SEI or SPR assessment methodologies. Oppositely laying activities complement one another (i.

Effective Methods of Software Testing proposes the 11 Step Software Testing Process. also known as the Testing 'V' Model. The 'V' Model as proposed by William E Perry – William E Perry in his book. The following figure depicts the same: ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 22 .Requirements Specifications Validate Requirements Acceptance Testing System Testing Architectural Design Verify Design Integration Testing Detail Design Unit Testing Coding iii.

Define Software Requirements Assess Development Plan and Status Develop the Test Plan Build Software Test Software Requirements Test Software Design Program Phase Testing Operate and Maintain Software Execute and Record Results Acceptance Testing Report Test Results Operate and Maintain Software Test Software Installation Test Software Changes Evalutate Test Effectiveness ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 23 .

product revision.[2] The criteria in these models are not independent. Three often used models are discussed here as examples. (IEEE) standards. Most notably are the Software Engineering Institute’s Capability Maturity Model (CMM). Quality Models and Quality Assessment A. they interact with each other and often cause conflict. ISO 12207 Standard for Information Technology . B. the Malcolm Baldrige National Quality Award. D. C. each goal having a large number of attributes.3. McCall's Model of Software Quality (The GE Model. There are many quality models and standards. Purpose of a Quality Model To satisfy business goals and objectives Requirements are imposed by a customer For competitive reasons As a guide (roadmap) to continuous improvement 2. This category will test the CQA candidate’s understanding of model objectives. structure. 1. A. SPICE (ISO 15504). ISO 9126 incorporates six quality goals. 1977) incorporates 11 criteria encompassing product operation.Software Life Cycle Processes. Boehm's Model (1978) is based on a wider range of characteristics and incorporates 19 criteria. There are many models of software product quality that define software quality attributes. A. Inc. ISO 9000. The Institute of Electrical Electronics Engineers. especially when software providers try to incorporate them into the software development process. pros and cons. and how assessments and baselines can be developed using a quality model. and the Quality Assurance Institute’s Approach to Implementing Quality. The criteria and goals(1) defined in each of these models are listed below: ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 24 . and product transition.

A. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 25 . These three models and other references to software quality use the terms criteria. Initial (Level 1) . and measuring the maturity of the processes used by software professionals. C. and success depends on individual effort and heroics. disciplined software processes. education and health care. Now considered America’s highest honor for performance excellence. the SEI and community have put in place an effective means for modeling. to recognize quality and business achievements of U. customer and market focus. service. The Capability Maturity Model for Software (CMM or SW-CMM) is a model for judging the maturity of the software processes of an organization and for identifying the key practices that are required to increase the maturity of these processes.The software process is characterized as ad hoc. D. He took a personal interest in the quality improvement act that was eventually named after him and helped draft one of the early versions. and business results. Industry Quality Models 1. To avoid confusion. defining. the National Institute of Standards and Technology designed and manages the award and the Baldrige National Quality Program. The CMM is organized into five maturity levels: i. The Software CMM has become a de facto standard for assessing and improving software processes. and heroic efforts required by individuals to successfully complete projects.S. Malcolm Baldrige was Secretary of Commerce from 1981 until his death in a rodeo accident in July 1987. we will use the terminology in ISO 9126 . metric. Awards are given in manufacturing. B.S. Baldrige was a proponent of quality management as a key to this country’s prosperity and long-term strength. and. Malcolm Baldrige National Quality Award A. slowly growing U.S. the Baldrige Award is presented annually to U. C. B. B. process management. The CMM is designed to provide organizations with guidance on how to gain control of their process for developing and maintaining software and how to evolve toward a culture of software excellence. Characterized by chaos. Congress named the award in his honor. goals and attributes interchangeably. Few if any processes in place. successes may not be repeatable. Defense Department to help improve software development processes.B. The Baldrige Award is given by the President of the United States to businesses—manufacturing and service. quality movement. Through the SW-CMM. small business. In recognition of his contributions. In conjunction with the private sector. and to publicize these organizations’ successful performance strategies. organizations by the President of the United States. starting in 1999. and occasionally even chaotic. Software Engineering Institute’s Capability Maturity Model A. information and analysis. chaotic processes to mature.goal. small and large—and to education and health care organizations that apply and are judged to be outstanding in seven areas: leadership. initiated by the U. organizations. 1) The Capability Maturity Model for Software describes the principles and practices underlying software process maturity and is intended to help software organizations improve the maturity of their software processes in terms of an evolutionary path from ad hoc. Congress established the Malcolm Baldrige National Quality Award to promote quality awareness. Few processes are defined. human resource focus. attribute. 2. It does this by serving as a model against which an organization can determine its current process maturity and by identifying the few issues most critical to software quality and process improvement. SEI = 'Software Engineering Institute' at Carnegie-Mellon University. In 1987. jumpstarting a small. strategic planning.S. periodic panics.

and functionality. Software project tracking. many of the QA processes involved are appropriate to any organization.ii. The key process areas at Level 3 address both project and organizational issues. Software Quality Assurance. Training Program. standardized. A. Software Project Planning. a Software Engineering Process Group is is in place to oversee software processes. v. It's a model of 5 levels of organizational 'maturity' that determine effectiveness in delivering quality software. B. B. and Peer Reviews. Repeatable (Level 2) . and products. Software Project Tracking and Oversight. Organization Process Definition. It is geared to large organizations such as large U.The software process for both management and engineering activities is documented. Software Product Engineering. developed by the SEI. Metrics are used to track productivity.S. A. and configuration management processes are in place. All projects use an approved.Detailed measures of the software process and product quality are collected. schedule. Standard software development and maintenance processes are integrated throughout an organization. effectiveness. measurable software process improvement. Integrated Software Management. requirements management. The key process areas at Level 5 cover the issues that both the organization and the projects must address to implement continual. The necessary process discipline is in place to repeat earlier successes on projects with similar applications. and quality is consistently high. as the organization establishes an infrastructure that institutionalizes effective software engineering and management processes across all projects. and Software Configuration Management. the empirical evidence to date supports this belief E. B.Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies. Project performance is predictable. realistic planning. They are Quantitative Process Management and Software Quality Management. and integrated into a standard software process for the organization. Software Subcontract Management. The key process areas at Level 4 focus on establishing a quantitative understanding of both the software process and the software work products being built. A. B.Basic project management processes are established to track cost. vi. and training programs are used to ensure understanding and compliance. tailored version of the organization's standard software process for developing and maintaining software. successful practices can be repeated. They are Requirements Management. iv. and if reasonably applied can be ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 26 . and Process Change Management. The Key Process Areas (KPA) at Level 2 focus on the software project's concerns related to establishing basic project management controls. Defined (Level 3) . The impact of new processes and technologies can be predicted and effectively implemented when required. Intergroup Coordination. The focus is on continouous process improvement. Defense Department contractors. Optimizing (Level 5) . and control of an organization's software processes are believed to improve as the organization moves up these five levels. Managed (Level 4) . Technology Change Management. They are Organization Process Focus. They are Defect Prevention. A. CMM = 'Capability Maturity Model'. Both the software process and products are quantitatively understood and controlled. processes. While not rigorous. However. iii. Predictability.

Based on the best current practices in the fields such as human resources and organizational development. to a mature. organize. disciplined development of the knowledge. It describes an evolutionary improvement path from ad hoc. The P-CMM consists of five maturity levels that lay successive foundations for continuously improving talent. and successfully managing the people assets of the organization.The key process areas at Level 2 focus on instilling basic discipline into workforce activities. and motivation of the work force. F. each maturity level is decomposed into several key process areas that indicate the areas an organization should focus on to improve its workforce capability. They are:  Knowledge and Skills Analysis  Workforce Planning  Competency Development  Career Development  Competency-Based Practices  Participatory Culture ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 27 . The P-CMM helps organizations to characterize the maturity of their work-force practices. Except for Level 1. develop. motivate.helpful. The P-CMM can also be used by any kind of organization as a guide for improving their people-related and work-force practices. just as the CMM describes an evolutionary improvement path for the software processes within an organization. They are:  Work Environment  Communication  Staffing  Performance Management  Training  Compensation 3) Defined .The key process areas at Level 3 address issues surrounding the identification of the organization's primary competencies and aligning its people management activities with them. The motivation for the P-CMM is to radically improve the ability of software organizations to attract. Each maturity level is a well-defined evolutionary plateau that institutionalizes a level of capability for developing the talent within the organization. and establish a culture of software engineering excellence. set priorities for immediate actions. the P-CMM provides organizations with guidance on how to gain control of their processes for managing and developing their work force. The key practices describe the infrastructure and activities that contribute most to the effective implementation and institutionalization of the key process area. integrate work-force development with process improvement. People Capability Maturity Model (P-CMM) The People Capability Maturity Model® (P-CMM®) adapts the maturity framework of the Capability Maturity Model® for Software (CMM®) [Paulk 95]. and retain the talent needed to continuously improve software development capability. developing effective teams. Each key process area is described in terms of the key practices that contribute to satisfying its goals. guide a program of continuous workforce development. Organizations can receive CMM ratings by undergoing assessments by qualified auditors. The P-CMM is designed to allow software organizations to integrate work-force improvement with software process improvement programs guided by the SW-CMM. skills. The five maturity levels of the P-CMM are: 1) Initial 2) Repeatable . inconsistently performed practices. to managing and developing an organization's work force.

under stated conditions Maintainability is the set of attributes that bear on the effort needed to make specified modifications Portability is the set of attributes that bear on the ability of software to be transferred from one environment. not just software. production. B. by a stated or implied set of users Efficiency is the set of attributes that bear on the relationship between the level of performance of the software and the amount of resources used. ISO 9126 is the software product evaluation standard that serves to eliminate any misunderstanding between purchaser and supplier.4) Managed . They are:  Personal Competency Development  Coaching  Continuous Workforce Innovation 3.Quality Management Systems: Fundamentals and Vocabulary. C. ISO 9000 family of standards presents an overview of the standards and demonstrates how they form a basis for continual improvement and business excellence. It covers documentation. development.it indicates only that documented processes are followed. design. ISO 9000 family includes ISO 9001.The ISO 9001:2000 standard (which replaces the previous standard of 1994) concerns quality systems that are assessed by outside auditors.Quality Management Systems: Requirements. (b)Q9000-2000 . installation. To be ISO 9001 certified. a third-party auditor assesses an organization. and on the individual assessment of such use. servicing. after which a complete reassessment is required. They are:  Mentoring  Team Building  Team-Based Practices  Organizational Competency Management  Organizational Performance Alignment 5) Optimizing . ISO 9000 / ISO 9004 Quality Management Principles and Guidelines on their Application A.The key process areas at Level 5 cover the issues that address continuous improvement of methods for developing competency.Quality Management Systems: Guidelines for Performance Improvements. ISO = 'International Organisation for Standardization' . testing. which is an international "quality management system" standard--a standard used to assess an organization's management approach regarding quality. D. and other processes. (c)Q9004-2000 .The key process areas at Level 4 focus on quantitatively managing organizational growth in people management capabilities and in establishing competency-based teams. ISO 9002 and ISO 9003 which were integrated into ISO 9001:2000. at both the organizational and the individual level. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 28 . Note that ISO certification does not necessarily indicate quality products . E. The full set of standards consists of: (a)Q9001-2000 . ISO 9126 is the software product evaluation standard that defines six characteristics of software quality: 1) Functionality is the set of attributes that bear on the existence of a set of functions and 2) 3) 4) 5) 6) their specified properties. and it applies to many kinds of production and manufacturing organizations. The functions are those that satisfy stated or implied needs Reliability is the set of attributes that bear on the capability of software to maintain its level of performance under stated conditions for a stated period of time Usability is the set of attributes that bear on the effort needed for use. and certification is typically good for about 3 years.

the standard is intended for two-party use where an agreement or contract defines the development.Standard for Information Technology (Software Process Improvement and Capability dEtermination) A. quality control. SOFTWARE LIFE CYCLE .) This is in contrast to a "text book approach. maintenance. joint review. and training. ISO 12207 offers a framework for software life-cycle processes from concept through retirement.Life Cycle processes A. each party has certain responsibilities. The project has three principal goals: 1) to develop a working draft for a standard for software process assessment 2) to conduct industry trials of the emerging standard 3) to promote the technology transfer of software process assessment into the software industry world-wide ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 29 .The standard establishes a top-level architecture of the life cycle of software. and sometimes retirement phase. development. maintenance.Standard for Information Technology . or operation of a software system. configuration management. installation and checkout phase.The period of time that begins when a software product is conceived and ends when the software is no longer available for use. such as management. etc. 2) Responsibility . SPICE ISO 15504 . the SPICE (ISO/IEC 15504) (Software Process Improvement and Capability dEtermination) project. as discussed later. and will continue until ISO/IEC 15504 is published as a full International Standard. The project is carried out under the auspices of the International Committee on Software Engineering Standards through its Working Group on Software Process Assessment (WG10). audit. Responsibility is one of the key principles of total quality management. quality assurance. test phase. Since 1993. measurement. that is. It divides the five processes into "activities. Field trials of SPICE-based assessment commenced in January 1995. In fact. It is especially suitable for acquisitions because it recognizes the distinct roles of acquirer and supplier.acquisition. ISO 12207 . It is not applicable to the purchase of commercial-off-the-shelf (COTS) software products. D. infrastructure. The architecture is built with a set of processes and interrelationships among these processes. supply. and problem resolution--as well as four "organizational processes"--management. (IEEE-STD-610) 5. and the evaluation of the quality of the end product (ISO 9126 ) are important and both require the presence of a system for managing quality. scheduled by 2002. requirements phase. ISO 12207 describes five "primary processes"-. and operation. SPICE (ISO/IEC 15504) is a major international initiative to develop a Standard for Software Process Assessment. implementation phase." while placing requirements upon their execution. C. In other words. The derivation of the processes is based upon two basic principles: modularity and responsibility. 1) Modularity . Software life cycle architecture . B. launched within the International Standards Organization has been developing a framework standard for software process assessment." where the life cycle functions could be studied as topics or subjects. B. An individual process is dedicated to a unique function. design. 4." and the activities into "tasks. It also specifies eight "supporting processes"-documentation. The life cycle begins with an idea or a need that can be satisfied wholly or partly by software and ends with the retirement of the software. Assurance of the process by which a product is developed (ISO 9001). The software life cycle typically includes a concept phase.A process is considered to be the responsibility of a party in the software life cycle.F. validation. verification. they are maximally cohesive and minimally coupled to the practical extent feasible. operation and maintenance phase. improvement. design phase. bringing together the major suppliers and users of assessment methods.The processes are modular.

The Seven Phase Performance Implementation Framework: 1) Improving performance is not a one-time “quick fix”. Manage by fact iv. creates standards such as 'IEEE Standard for Software Test Documentation' (IEEE/ANSI Standard 829). The Approach shows methods for improving all of the activities within IT. customers/users. QAI has developed a customizable approach that is driven by your management's style. The IEEE is a global technical professional society serving the public interest and members in electrical. weaknesses.among other things. overrun budgets. opportunities. and staff. and provide proof that IT is operating effectively and efficiently. 'IEEE Standard of Software Unit Testing (IEEE/ANSI Standard 1008). E. and develop a strategic measurement dashboard. IEEE = 'Institute of Electrical and Electronics Engineers' . 'IEEE Standard for Software Quality Assurance Plans' (IEEE/ANSI Standard 730). These five categories are: 1) Establish a quality environment within the I/S function. information & other technologies. organizational strengths. and others. The objectives of this phase are to: establish and clarify the organization’s vision. Manage by process iii. This approach is composed of five process categories. A well-defined mission statement ii. and a feedback system based upon metrics. C. 2) Align information services with corporate objectives and define the desired results to support those objectives. 3) Quality and Performance Analysis i. and threats. superior performers. inhibitors. 1) Our Approach is a business-oriented approach. D. This phase is to conduct an organization wide data gathering process to assist in: identifying strengths. Each category is segmented into specific how-to processes. It is a continuum of activity that requires the understanding and buy-in of all employees. QAI's Approach for Managing Quality in a Changing World is designed to enable IT organizations to restore credibility. processes that ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 30 . QAI'S Strategic Model 1) QAI's Strategic Model contains four processes critical to your success: i. align and deploy processes to support the defined management results. QAI subscribes to a sevenphase approach: 2) Establishing A Partnership i. specific performance objectives. 7. Manage continuous improvement 2) QAI believes that the foundation to any quality initiative is: i. 4) Establish strategic and tactical dashboards to enable management to effectively use quantitative data in their management processes. customer/user needs. B. constraints. build an enviroment where products are completed on time and within budget. organizational weaknesses. and failure to implement the needed requirements. It recognizes the close working relationship that must exist between IT management.6. The Quality Assurance Institute’s Approach to Quality Implementation A. management’s perception of “where the pain is”. A clearly defined vision B. electronics. implement. Manage toward results ii. The Institute of Electrical Electronics Engineers (IEEE) Standards A. The Approach recognizes that many organizations must first reestablish credibility because of missed schedules. 3) Establish. 5) Continuously improve the above process categories. computer. QAI has developed a detailed how-to approach to quality improvement.

B. This phase ensures that the solutions are increasing performance as defined by management’s strategic measurement dashboard. Evaluating Solutions i. the agreed upon solutions. formally designated and fixed at a specific time during the configuration item's life cycle.. Using Models for Assessment and Baselines: A. Model Selection Process 1. BASELINE a. and QAI’s improvement framework. Develop a Consensus Approach i. Using proven industry work practices. 8. Allocated Baseline. B. additional design constraints. ANSI = 'American National Standards Institute' A. interface requirements with interfacing configuration items.S. and that can be changed only through formal change control procedures. The initially approved documentation describing a system’s or configuration item’s functional characteristics and the verification tests required to demonstrate the achievement of those specified functional characteristics. and the verification tests required to demonstrate the achievement of those specified functional and interface characteristics. This final phase is to work towards building a culture and environment conducive to continuous improvement of performance enhancers. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 31 . b. A specification or product that has been formally reviewed and agreed upon. regardless of media. Criteria may include A. Implementing Solutions i. C. publishes some software-related standards in conjunction with the IEEE and ASQ (American Society for Quality). Functional Baseline. The initially approved documentation describing a configuration item’s interface characteristics that are allocated from those of the higher level configuration item or those to a lower level. Developing Solutions i. audit existing standards for applicability and status. and establish performance baselines. which are established sequentially. tools (manual or automated) in use. How an IT organization selects a model. Improving Solutions i. Need for measurable goals and objectives E. Applicability of model to the IT organization’s goals and objectives.4) 5) 6) 7) 8) are currently performed. The primary industrial standards body in the U. to accomplish the agreed upon improvement goals is the goal of this phase. Need for Baseline Assessments D. (IEEE/EIA 12207. Management commitment to include needed: a. ii. as follows: i. b. constitute the current configuration identification. plus approved changes from those baselines. Management commitment to include needed: C. Applicability of model to the IT organization’s goals and objectives. This phase is to implement within the organization’s culture and constraints. (SW-CMM (IEEE-STD-610)) A formally approved version of a configuration item. Product baselines are references points in vital areas of the application that can be used to measure development progress. For configuration management. This phase is to prepare and deliver the overall organizational assessment findings and gain the consensus agreement of the project sponsors to the findings. there are three baselines. c. that thereafter serves as the basis for further development. Baselines.0) A configuration identification document or a set of documents formally designated by the Government at a specific time during a configuration item’s life cycle.

It is management’s responsibility to establish strategic objectives and build an infrastructure that is strategically aligned to those objectives. CMM’s Process capability baseline (PCB) . Management’s Quality Directives C. and supplier selection/retention. Quality Vision i. f.Our journey is Total Quality Management--fully satisfying our customers requirements through a process of continuous improvement. It is our goal to posture our company for market expansion. any required joint and combined operations interoperability characteristics of a configuration item (including a complete summary of other service and allied interfacing configuration items or systems and equipment). Without clear quality goals agreed upon up front. including manufacturing processes and procedures. thereby providing improved job security and quality of life for all b. the selected physical characteristics designated for production acceptance testing and tests necessary for production and support of the configuration item. Product Baseline. Mission Statement a. It's critical to understand that Total Quality Management is not a short term program. Examples of Quality Goals: A. Assuring the training and the professional growth of its employees. although the initial process performance baseline will usually be derived from the organization’s process capability baselines.defined as “a documented characterization of the range of expected results that would normally be achieved by following a specific process under typical circumstances. The most important prerequisite for a successful implementation of any major quality initiative is commitment from executive management. it is impossible to determine if we have met our objectives. Quality Goals i. This Policy aims at: 1. Quality Management/Leadership A.C. iii.To that goal the management of Savaré I. ii. hashing out release criteria at the time the test plan is written is far superior to doing it at the release checklist meeting. providing a safe work environment. Unpleasant as it may be. A process performance baseline is typically established at the project level. managing our business processes. (DODD 5010. CM. ii. Explain how the vision will be achieved. commits itself to carry out a Quality Policy. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 32 . iii. The initially approved documentation describing all of the necessary physical and functional characteristics of the configuration item. materials. 10/87) A system life cycle documentation standard established to enhance program stability and provide a critical reference point for measuring and reporting the status of program implementation. e.d. It's a long term commitment aimed at continuously improving the way we work.19. Progress implies a goal. providing them with the resources that are necessary to carry out their duty to the best of their abilities.defined as “a documented characterization of the actual results achieved by following a process. If we have not met our objectives. B. which is used as a benchmark for comparing actual process performance against expected process performance. then we cannot in good conscience say that a product is ready to ship. Is a clear definition of the result you are trying to achieve. Phelps Dodge Copper Products & Refining Corporation . Quality Goals . CMM’s Process performance baseline (PPB) . Example of a Corporate Quality Vision: A. This category will test the CSQA candidate’s understanding of the management processes used to establish the foundation of a quality-managed environment. 4.

Examples Of Corporate Quality Policies: a. Newport Corporation – To be the leading supplier of high quality optics.To meet or exceed all requirements agreed to with our customers. . Honesty and integrity in all that we do D. Inc. c. Respect for our Customers. Section 4. Timely and Qualitative software solutions. instruments. Argo-Tech Corporation . Quality Principles i. and community e. Bangalore.1 of the ISO 9001 standard requires that management "shall define and document its policy for quality. b. and value in our products c. Developing its organization to foster improvements in the company services. ii. spotting and eliminating the causes of non compliance. Quality Charter i. Quality Values i. Creativity and teamwork in the workplace d. Top-notch industry know-how and specialized expertise d. – “We practice continual Improvement to achieve customer delight by providing Customer-Centric. The statement of the enterprise’s commitment to Quality. shareholders. Cost-effective. Semiconductor Equipment. Can be defined as procedures. . Can be defined as standards ii. employees. iii. to meet or exceed our commitment to those expectations by performing the correct tasks defect free. Quality Policy i. proposing actions of improvement and verifying their correct applications. suppliers. Mummert & Partners." A. Zenith Software Limited. ii. The statement of the responsibilities & authorities of all Quality function performers.” c. Convincing project results e. Promotion of quality in human-resource work d. every time. Computer Peripherals. Powerful internal organization g. Quality Management Principles provide understanding of and guidance on the application of Quality Management. Outstanding service to our Customers b. Our Quality Values and Beliefs: a." It goes on to say that the policy "shall be relevant to the supplier's organizational goals and the expectations and needs of its customers. .Our quality principles comprise the following concepts: a. Examples of Quality Principles: A.2. and Fiber-Optic Communications industries worldwide. 3. Keeping under control the products of non compliance. including objectives for quality and commitment to quality . Quality is a principle for our management and work b. Spectra-Physics Scanning Systems – “We the employees of SpectraPhysics Scanning Systems make the personal commitment to first understand our customers expectations then. Maximum quality in performing projects f. Quality Assurance Charter a. Example of Quality Values: A. quality. on time. micropositioning and measurement products and systems to the Scientific and Research. Innovation.1. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 33 . Best quality for our clients c.” b.

We will use technology innovatively and with a human touch. Quality Assurance A. team or work group. shares responsibility with IT management to market and deploy quality programs. Build and mature a quality assurance function including staffing. Leadership: A Quality Champion has a vision of the quality organization. A statement of corporate standards and quality of service. We will achieve academic and administrative excellence by encouraging and expecting the creative involvement of all staff. Educating all members in the value of quality & their responsibility for it. and by continually improving our processes. This category will test the CQA candidate’s ability to understand and apply quality assurance practices in support of the strategic quality direction of the organization. and encourages the participation of all individuals in the organization. We will do so with warmth. 3. Process Orientation: A Quality Champion leads. Interviewing b. Establishing a Function to Promote and Manage Quality: A. based upon the holiday information provided by the tour operator. Observation ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 34 . and positively leads others towards that vision by example. Co-operation: A Quality Champion embraces the spirit of teamwork by working in co-operation with others in order to achieve the desired organizational. We will by our own example. grace and with typical Temasek style. 2. Identifying or developing and using problem reports. shares that vision with others. Selling Quality i. and the like. to gather the data that can be used for the improvement of the enterprise’s information processes. People Oriented: A Quality Champion is an open minded individual who is receptive to new ideas. AITO defines ‘quality’ as “providing a level of satisfaction which. Questionnaires c. Quality Assurance is a professional competency whose focus is directed at critical processes used to build products and services. courtesy.ii. products and services. team. Quality Champion: A. so as to achieve our mission and vision for the betterment of the people of Singapore. 5. C. D. or work group performance. inspire our students and community to keep improving themselves through continuous learning. AITO is an association of independently-minded companies specialising in particular areas or types of holiday and sharing a common dedication to high standards of quality and personal service.) E. The spokesperson for quality within the IT organization. Customer Focus: A Quality Champion actively works to make the customer a priority in their workplace and perseveres regardless of the barriers that may be encountered. supports and/or participates in the development of processes which will lead to better organization. F. has a positive attitude. iii. The profession is charged with the responsibility for tactical process improvement initiatives that are strategically aligned to the goals of the organization. B. At Temasek Polytechnic. Examples of Qualit Charters: A. planning. (Customer may be internal and/or external. c. B. We at Temasek Polytechnic are committed to exceeding the expectations of our stakeholders and customers in the delivery of all our courses and services. aims to meet or exceed a customer’s reasonable expectations. B. team or work group results. Common methods are: a. regardless of the type of holiday sold or the price paid”. we will create the best environment to work and study. Data-Gathering Techniques: A. by listening to our customers and meeting their needs. 1. and plan execution.

Post-Implementation Reviews (PIR): A. A PIR is an independent. v. B. 6.. to seek out error-prone products. The main interest in this analysis is locating where key problems are occurring and the frequency of occurrence. what should be done? What are the lessons learned that will improve future performance? 8. is 100% of all code checked. vi. iv. Applying Quality Assurance to IT Technologies and IT Technical Practices ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 35 . C. Develop a tactical quality plan. How you control changes. How you track multiple development builds of the software to avoid confusion (configuration management). methods. After the implementation of any major new piece of software. incident reports. How you ensure that the product meets the requirements (validation). SOFTWARE QUALITY ASSURANCE PLAN . It defines what is meant by the various quality-related tasks in the Project Plan. 9. iii. and better manage error-detection resources. viii. Identifying activities that comprise a process so that analysis can be performed. or only the most complex parts?). but it will also help improve any future implementations. in business terms? If not. How you plan for and execute testing. Process Analysis and Understanding: A. both incrementally during development and for the entire product before delivery to EPRI. How you track and resolve defects. Analysis of the gathered data to understand a process and its strengths and weaknesses and ability to watch a process "in motion" so that recommendations can be made to remove flawintroducing actions and build upon successful flaw-avoidance and detection actions. vii. Definitions.d. It also provides a mechanism to track progress of problem resolution. Process Identification: A. ii. 7. The Quality Plan outlines how you will build quality into the software and documentation. and criteria you use to determine whether the software has passed each review. e. it is often useful to "take stock" and look back at how the process was managed. and so forth. Identifies repetitive problems and assesses how often given problems occur. Problem Trend Analysis: A. resource utilization. reviews of test scripts. Technique used to review results of projects after their completion and implementation. walk throughs. Repertory Grids Concept Mapping Joint Application Design 4. How you ensure that the product works properly (verification). How and when you conduct design reviews. objective review that is a key part of the benefits management process. expected ROI. f. etc. Examination of problem reports. B. Quality Plan: A. 5. B. B. code reviews. Quality Tools: A. reviews of test results (for example. The Quality Plan describes: i. Evaluation may include compliance with approved requirements. Understanding. C. Not only will this highlight any issues which may need resolving over the forthcoming months. using and encouraging the use of quality tools. The Quality Plan describes how a developer's overall quality process guidelines will be applied to a project. anticipate future error experience. The dates assigned to key tasks in the Quality Plan are entered into the project plan. It is used to answer the questions: Did we achieve what we set out to do.Plan which indicates the means by which the SQA requirements are met during the information system’s life cycle.

They are created by plotting the cumulative frequencies of the relative frequency data (event count data). Analyzing problems or causes by different groupings of data. Focusing on critical issues by ranking them in terms of importance and frequency. Identify the problem area b. From the Pareto Chart it is possible to see that the initial focus in quality improvement should be on reducing edge flaws. Prioritizing problems or causes to efficiently initiate problem solving.The phenomenon whereby a small number of concerns is usually responsible for most quality problems. Rank the count by frequency (using bar chart) e. Solution of what production problem will improve quality most?) c. such defects are substantially less numerous than the edge flaws. Cause and Effect Diagram (Fishbone) ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 36 . an Italian economist who found that a large percentage of wealth was concentrated in a small proportion of the entire population. Pareto Principle . and in an orderly format. Which problem with Product X is most significant to our customers?) b. The principle is named for Vilfredo Pareto. (ex. iv. in decending order. Pareto charts are extremely useful because they can be used to identify those factors that have the greatest cumulative effect on the system. (ex. the most essential factors for the analysis are graphically apparent. Count the named incedences d. Validate reasonableness of the Pareto analysis b. (ex. Ideally. Although the print quality is also of some concern. Analyzing the before and after impact of changes made in a process. Management Tools: a. v. by process) d. Steps for preparing a Pareto analysis: a. Pareto Chart i. by machine. When this is done. Pareto Charts are used for: a. this allows the user to focus attention on a few important factors in a process. Name the events/items/causes that will be analyzed c. and thus screen out the less significant factors in an analysis. (ex. iii.A. The initiation of a quality improvement program reduced the number of defectives?) ii.

b. it is essential to establish a system that will continuously promote quality in all aspects of its operation B. Thus. This type of diagram is useful in any analysis. is used to associate multiple possible causes with a single effect. Discovery Analysis Improvement Monitoring Implementation Verification d. This diagram. f. check sheets. In many instances.a. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 37 . d. Furthermore. the tools should be utilized to ensure that all attempts at process improvement include: a. c. Problem Identification Tools: Demonstrate an understanding of tools such as flow charts. In any case. it is important to note that the mere use of the quality control tools does not necessarily constitute a quality program. given a particular effect. b. The primary branch represents the effect (the quality characteristic that is intended to be improved and controlled) and is typically labelled on the right side of the diagram. The tools listed above are ideally utilized in a particular methodology. also called an Ishikawa diagram (or fish bone diagram). a Cause and Effect diagram of the related process is created to help the user see the entire process and all of its components. i. and brainstorming. Thus. e. to achieve lasting improvements in quality. Minor branches correspond to more detailed causal factors. the diagram is constructed to identify and organize possible causes for it. as it illustrates the relationship between cause and effect in a rational manner. attempts to find key problem areas in a process can be a hit or miss proposition. However. it was decided to collect data on the curetimes of the material c. Each major branch of the diagram corresponds to a major cause (or class of causes) that directly relates to the effect. Having decided on which problem to focus on. In this instance. other methodologies may need to be developed to allow for sufficient customization to a certain specific process. which typically involves either reducing the process variability or identifying specific problems in the process. c.

ii. i. By breaking down the process into a series of steps. Flowcharts are pictorial representations of a process. Problem Analysis Tools: Demonstrate an understanding of tools such as histograms. in some instances. Flowchart i. b. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 38 . scatter diagrams. control charts. However. by incorporating a depiction of the system under analysis into the form. and force field analysis. Additional data collection checksheet examples demonstrate the utility of this tool. The function of a checksheet is to present information in an efficient.a. By breaking the process down into its constituent steps. The data collected will be used in subsequent examples to demonstrate how the individual tools are often interconnected ii. This may be accomplished with a simple listing of items. Checksheet i. C. iii. A defect location checksheet is a very simple example of how to incorporate graphical information into data collection. the utility of the checksheet may be significantly enhanced. the flowchart simplifies the analysis and gives some indication as to what event may be adversely impacting the process. graphical format. flowcharts can be useful in identifying where errors are likely to be found in the system.

but is bimodal instead. Histograms provide a simple. ii. Data for a test of curetimes was collected and analyzed using a histogram. histograms provide the easiest way to evaluate the distribution of data. Scatter diagrams are graphical tools that attempt to depict the influence that one variable has on another. iii. graphical view of accumulated data. Deviations from a normal distribution in a histogram suggest the involvement of additional influences in the process. it is possible to conclude that by establishing a standard curetime within this range. From this chart. Scatter Diagram i. the curetime distribution does not appear to be a normal distribution as might be expected. Control Chart ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 39 .c. some degree of quality improvement is likely. it is possible to see that there are very few defects in the range of approximately 29.5 to 37. including its dispersion and central tendancy. Applying curing time test data to create a scatterplot. ii. Histogram i. Thus. e. A common diagram of this type usually displays points representing the observed value of one variable corresponding to the value of another variable. In addition to the ease with which they can be constructed.0 minutes. d.

Forcefield Analysis ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 40 . iii. On the other hand. but run charts focus more on time patterns while a control chart focuses more on acceptable limits of the process. ii. it helps determine whether or not a process is operating consistently or if a special cause has occurred to change the process mean or variance. Run charts are useful in discovering patterns that occur over time. Applying statistical formulas to the data from the curetime tests of base material.i. ii. The control chart is the fundamental tool of statistical process control. it was possible to construct X-bar and R charts to assess its consistency. as it indicates the range of variability that is built into a system (known as common cause variation). Run charts evolved from the development of these control charts. f. Run charts are used to analyze processes according to time or order. Run Charts i. As a result. which can typically be found and eliminated. g. we can see that the process is in a state of statistical control. The bounds of the control chart are marked by upper and lower control limits that are calculated by applying statistical formulas to data from the process. Data points that fall outside these bounds represent variations due to special causes. Thus. improvements in common cause variation require fundamental changes in the process.

Backup and Recovery i. Security i. Brochure ware. however. E. A method of analyzing a situation by looking at all the forces and factors affecting the issue. or could help. B. Identifying risks of distributed processing. storefront. It should be used whenever a change or improvement is needed. E-Commerce. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 41 . Web based systems i. Reducing development cycle time with disciplined processes. ii. Protecting access to your organization’s technology assets. Client server. distributed processing refers to localarea networks (LANs) designed so that a single program can run simultaneously at various sites. Most distributed processing systems contain sophisticated software that detects idle CPUs on the network and parcels out programs to utilize them. D. On the right side.i. B. list the forces that are helping. On the left side. Ensuring customer’s confidential data is not compromised. in which a single computer uses more than one CPU to execute programs. or processor. More often. or a selling channel. to run an application. Improvement will happen only if the encouraging factors are strengthened or the inhibiting factors are weakened. list the forces that are hindering the situation. ii. i. A technique which helps us to achieve improvement by considering those factors or forces that encourage change or those which work against change. The objective of this skill is to identify where and how the Quality Assurance professional can control IT technologies and technical practices such as: A. or could get in the way of reaching the goal. 2. This includes parallel processing. 1. drive the group towards the goal. ii. Distributed Processing . Restart application after problems are encountered.Refers to any of a variety of computer systems that use more than one computer. Web-based systems integration is the art of combining multiple systems (including Legacy systems and proprietary software applications) into a new system that is accessible through a Web browser. Privacy i. i. C. F.

and logistics from raw materials to customer satisfaction. conveys knowledge embedded in its users to others in the organization. Brochure ware . parts purchasing. internal customers and suppliers. providing customer service. interacting with suppliers. Storefront . Knowledge Management(KM). and the distribution of these finished products to customers. Enterprise Resource Planning (ERP). C. i. operations. all the modern features of a shopping cart system. A new business strategy built around demand and trust. ii. Supply Chain Management (SCM). process and technology to maximize all of your relationships – with your day-to-day customers. iii. although the complexity of the chain may vary greatly from industry to industry and firm to firm.A website that is little more than a corporate brochure. B. C. ii. Customer Relationship Management(CRM). field support and other functions that touch your customer. or other corporate media. Supply chains exist in both service and manufacturing organizations. That is why ERP is often referred to as back-office software. A. maintaining inventories. This is an accurate definition. Supply Chain Management focuses on globalization and information management tools which integrate procurement. marketing. and the subsequent ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 42 . G. Your customers will want easy navigation of your product catalog. iii. D. created. E. A. The umbrella under which products are ordered. ERP's best hope for demonstrating value is as a sort of battering ram for improving the way your company takes a customer order and processes it into an invoice and revenue—otherwise known as the order fulfillment process. E-Business. distribution channel partners. The right CRM strategy integrates people. i. but doesn't give us much insight into the excitement surrounding eBusiness and eCommerce. including product planning.The software you use to build and manage your online store is critical to the overall success of your e-commerce venture. transformation of these materials into intermediate and finished products. flexible payment options and clear confirmation that their order has been received. video. B.ii. It is the information technology available to "enable" business transactions electronically H. ERP (Enterprise resource planning) is an industry term for the broad set of activities supported by multi-module application software that helps a manufacturer or other business manage the important parts of its business. a simple check-out process. Customer Relationship Management (CRM) is the seamless coordination between sales. A supply chain is a network of facilities and distribution options that performs the functions of procurement of materials. iii. and delivered. where the interaction is enabled by information technology. eBusiness is an interaction with business partners. B. Understanding how to build a partnership with your most valuable customers. The process. ERP attempts to integrate all departments and functions across a company onto a single computer system that can serve all those different departments' particular needs. once institutionalized. customer service. and tracking orders. Knowledge management involves the identification and analysis of available and required knowledge assets and knowledge asset related processes. A.

and access to an application (that is commercially available) from a centrally managed facility. The ASP makes the application available to customers everywhere via the Internet. A contractual service offering hosting. In many cases. operates and maintains the servers that run the application." (Journal for Quality & Participation. The ASP owns and operates a software application. C.. Knowledge Management caters to the critical issues of organizational adaption. the term ASP has come to denote companies that supply software applications and/or software-related services over the Internet. One key attribute beginning to rapidly evolve is the emphasis on Webbased e-business application management as an important differentiator from the more traditional outsourced client-server application management services. the ASP can provide the service for free or will even pay the customer. D. The terms "ASP" and "Application Service Provider" are applied specifically to companies that provide services via the Internet. B. A repository of historical data used to make decisions. The term data warehousing generally refers to combine many different databases across an entire enterprise H. C. Data warehouses contain a wide variety of data that present a coherent picture of business conditions at a single point in time. Application Service Providers (ASP). A. this interpretation explicitly addresses the strategic distinction between knowledge and information explained earlier. however. Essentially. Here are the most common features of an ASP: A. i. managing. Furthermore. either in a browser or through some sort of "thin client. The ASP also employs the people needed to maintain the application. G. Data Warehousing (DW). unlike most prevailing definitions. The ASP bills for the application either on a per-use basis or on a monthly/annual fee basis. it embodies organizational processes that seek synergistic combination of data and information processing capacity of information technologies. Data Warehousing . Outsourcing. this conception is better related to the new model of business strategy discussed earlier. an ASP is a service provider whose specialization is the implementation and ongoing operations management of one or more networked applications on behalf of its customer. A." D. In most cases.. Development of a data warehouse includes development of systems to extract data from operating systems plus installation of a warehouse database system that provides managers flexible access to the data. Simply stated. (Information Strategy: The Executive's Journal) F. Its primary focus is on How can knowledge management enable business strategy for the new world of business? and What strategic outcomes should knowledge management try to achieve? rather than What goes into the nuts and bolts of the machinery that supports knowledge management? It relates more closely to the dynamic view of business strategy as driver of the corporate information strategy. B..A collection of data designed to support management decision making. Unlike most conceptions of knowledge management proposed in information systems research and in trade press. B.C. and Asian Strategy Leadership Institute Review). ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 43 . Hewlett-Packard Executive Intelligence. and the creative and innovative capacity of human beings. survival and competence in face of increasingly discontinuous environmental change. planning and control of actions to develop both the assets and the processes so as to fulfil organisational objectives. The ASP owns.

The act of hiring an outside source. processes and applications for access via a virtual private network or an Internet-based browser. is typically used for design products and untested code. the less costly it is to correct. can perform quality control. and psychology of the technique. The technique. A method for exposing people to new areas of the system. D. and/or the customer. rules. Management Controls A. An opportunity to monitor adherence to standards. Quality control comprises all methods employed to detect the presence of defects. ranging from informal peer reviews to structured reviews for the purpose of early error detection for removal. Quality Control A. iii.RFP). the process for selecting a service provider. and reporting on the operation of the information system function. and to ensure through governance. Inspections i. C. financial integrity. i. Additional benefits of conducting walkthroughs (actually significant benefits in our situation) are: A. 6. This category will test the candidates understanding of quality control principles and methods. The primary purpose of a walkthrough is to identify defects in the product as early in the Systems Life Cycle as possible. structured and easy to maintain products. Verification Methods A. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 44 . Application Controls A. B. an independent group or person. A method for disseminating new concepts and conventions. rationale. iii. A method for improving group communication. accounting. D. Knowledge should cover principles. and systems performance. System of Internal Control 1. Ensure readable. and the easier it is to take corrective action. Walkthroughs i. usually a consultant or application service provider. Planned and formal technique used to verify compliance of specific development products against their documented standards and requirements. E. to transfer components or large segments of an organization's internal IT structure. The producer of the product or service. ii. B. ii. staff. A basic understanding of typical manual and automated controls within an information system designed to ensure data integrity. security. Knowledge of the methods and procedures used by management to provide direction to their staff. Verification and Validation 1.A. and processes to manage and control expectations and status. Quality control should occur both during the build of a product or service and after completion. Quality control is a component of internal control. Outsourcing . Internal System Controls A. B. Knowledge of the subset of management controls focused on assuring a completed project meets the user’s true needs E. Quality Control Practices A. Knowledge of how software applications are controlled. Developing a process to solicit service providers (Request For Proposal . process integrity. The earlier a defect is identified. In Process Reviews A.

Software Inspections are a disciplined engineering practice for detecting and correcting defects in software artifacts. Code inspections -. v. 4-5 people participate and expend 1-2 hours of preparation and 1-2 hours of conduct each. Are all requirements allocated? G. What mission need is addressed by a requirement? B. The development and use of Requirements Tracing techniques originated in the early 1970s to influence the completeness. The Return on Investment for Software Inspections is defined as net savings divided by detection cost. Is this requirement necessary? D. Where is a requirement implemented? C. The goal is to identify and remove bugs before testing the code. The net savings then are up to nine times for major defects and up to three times for minor defects. Formal code inspections are one of the most powerful techniques available for improving code quality. availability. Is this design element necessary? I. and roles of participants. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 45 . Are we done? L. What design decisions affect the implementation of a requirement? F. How do I interpret this requirement? E. Methods to ensure that requirements are implemented correctly during each software development life cycle phase. Software Inspections were introduced in the 1970s at IBM.ii. consistency. viii. This cost of 10 to 20 hours of total effort per session results in the early detection of 5-10 defects in 250-500 lines of new development code or 1000-1500 lines of legacy code. A minor defect may cost two to four times to detect and correct. Why is the design implemented this way and what were the other alternatives? H. a Key Process Area (kpa) of the CMM. Typically. and traceability of the requirements of a system. the population of Software Inspections adopters ranges from level 1 to 5. structure (including rules). vi. iv. What acceptance test will be used to verify a requirement? K. Software Inspections are a rigorous form of peer reviews. Savings result from early detection and correction avoiding the increased cost that comes with the detection and correction of defects later in the life cycle. which pioneered their early adoption and later evolution.peers reviewing code for bugs -complement testing because they tend to find different mistakes than testing does. The detection cost is the cost of preparation effort and the cost of conduct effort. and preventing their leakage into field operations. Is the implementation compliant with the requirements? J. ii. and many organizations limit their software process improvement agenda to the kpas for the maturity level they are seeking to achieve. Although peer reviews are part of achieving CMM level 3. Software Inspections provide value in improving software reliability. and maintainability. Knowledge should cover purpose. Requirements Tracing i. An undetected major defect that escapes detection and leaks to the next phase may cost two to ten times to detect and correct. The cost of performing Software Inspections includes the individual preparation effort of each participant before the session and the conduct effort of participants in the inspections session. They provide an answer to the following questions: A. vii. iii. What is the impact of changing a requirement? C. Code inspections are even more useful when inspectors hunt for specific errors rather than casually browse for bugs.

Black Box testing (Functional testing) attempts to find discrepancies between the program and the user’s ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 46 . Test Concepts A. White box testing does not account for errors caused by omission.e. design. B. and specialized tables or matrices that track the cross references). progress. and others as appropriate. assumption-based truth maintenance networks. A. B. Validation Methods A.Also known as glass box.The documentation is restructured in terms of an underlying network or graph to keep track of requirements changes (e. Unlike black box testing. programming. text. numbering. start-up. in both a forward and backward direction (i.g. Incremental (top-down and bottom-up). build. At the end of a project they are commonly called “Post Mortems Review” 2... The test is accurate only if the tester knows what the program is supposed to do. A walkthrough gives assurance that no major oversight lies concealed in the material. Testing techniques i. design. b. WHITE BOX . Decisions to proceed with development. Cross referencing . and the phases typically take their names from these items: requirements. Should cover topics such as purpose. such as human (walkthroughs/inspections). from its origins. clear box and open box testing. A formal written report of the findings and recommendations is normally provided. stage gates. are usually a part of these reviews. A software testing technique whereby explicit knowledge of the internal workings of the item being tested are used to select the test data. the completion of each phase of development.g. structural. and regression. etc. chaining mechanisms. white box testing uses specific knowledge of programming code to examine outputs. Review of products and the processes used to develop or maintain systems occurring at. e.. and methods for designing and conducting. C. and all visible code must also be readable. c. black box (data driven). white box (logic driven). Requirements traceability is defined as the ability to describe and follow the life of a requirement. He or she can then see if the program diverges from its intended goal. or kill points. BLACK BOX . and through periods of ongoing refinement and iteration in any of these phases). and propagation). Restructuring .A presentation of developed material to an audience with a broad cross-section of knowledge about material being presented. Specialized templates and integration or transformation documents These are used to store links between documents created in different phases of development. a. based on cost.iii. turnover. The majority of these items are related to the primary phase deliverable. B. Knowledge of the various techniques used in testing.This involves embedding phrases like "see section x" throughout the project documentation (e. or near. constraint networks. There is no required preparation on the part of the audience and limited participation. Phase-End Reviews A. through its development and specification. Each project phase normally includes a set of defined work products designed to establish the desired level of management control. tagging.g. to its subsequent deployment and use. risk. or indexing of requirements. schedule. These phase-end reviews are often called phase exits.. WALKTHROUGH . C..

description of what the program should do. Unit tests are written from a programmer's perspective. Methods of types such as unit/program. iii. The test cases are difficult to design. if any. whether or not successful. integration. C. A. Testing every possible input stream is unrealistic because it would take a inordinate amount of time. a. A. The tester does not need knowledge of any specific programming languages. Test cases can be designed as soon as the specifications are complete. D. c. iv. REGRESSION ACCEPTANCE TEST . The disadvantages of this type of testing include: i. Testing is done from the user's point-of-view. either because the initial execution did not proceed successfully to its conclusion or because a flaw was discovered in the system or subsystem being tested. ii. performance (subsets include volume and stress. system.It may be necessary to execute a planned acceptance. Each test confirms that a method produces the expected output when given a known input. The test can be redundant if the software designer has already run a test case. string. Subsequent executions.d. The first execution of a planned test. and its outputs are verified for conformance to specified behavior. Software users are concerned with functionality and features of the system. and controls) and recovery. not the designer. For example. or unit test more than once. It is ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 47 . B. Unit tests tell a developer that the code is doing things right. many program paths will go untested. A software testing technique whereby the internal workings of the item being tested are not known by the tester. in a black box test on a software design the tester only knows the inputs and what the expected outcomes should be and not how the program arrives at those outputs. The test is unbiased because the designer and the tester are independent of each other. iii. b.An essential aspect of unit testing is to test one feature at time. B. Testing methods i. are termed regression tests. The advantages of this type of testing include: i. security. A test of an application software unit. The test is done from the point of view of the user. therefore. Unit Testing . Unit Testing may be defined as the verification and validation of an individual module or 'unit' of software. The tester does not ever examine the programming code and does not need any further knowledge of the program other than its specifications. ii. is termed an initial test. They ensure that a particular method of a class successfully performs a set of specific tasks. Also known as functional testing. It subjects the program or system to inputs. functional tests tell a developer that the code is doing the right things.

which test the behavior of a single class. Security Test & Evaluation E. Security Testing . Stress Testing . Log Review G. Functional Tests . etc B. output. links. Scalability Testing . A. Virus Detection ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 48 .The primary reason for testing a system is to identify potential vulnerabilities and subsequently repair them A. Unit testing may require developing test driver modules or test harnesses. Also. The purpose of Volume Testing is to find weaknesses in the system with respect to its handling of large amounts of data. push buttons. F.Determines the breaking point or unacceptable performance point of a system to reveal the maximum service level it can achieve. In addition. Integrity Checkers H. Vulnerability Scanning C. These tests confirm that the system does what users are expecting it to.Determines the behavior of a system with expanded workloads simulating future production states such as added data and an increased amount of users. “Routine” Unit Testing includes identifying all fields and testing for input. E.the most "micro" scale of testing for testing particular functions or code modules. unit testing often requires detailed knowledge of the internal program design.seeks to verify the physical and logical limits to a system's capacity and ascertain whether such limits are acceptable to meet the projected capacity of the application's required processing. E. Testing allows an organization to accurately assess their system’s security posture. upper and lower boundaries. A. A load test simulates user activity and analyzes the effect of the real-world user environment on an application. A. B. thus providing additional insight and advantage. Password Cracking F. G. C. etc. using the techniques recommended in this report. The following types of security testing: A. allows an organization to view its network the same way an attacker would. server requests. Volume Testing . Unlike UnitTests. These include scroll bars. a company can identify problematic parts of a Web application before it is accessed by hundreds or thousands of users. Load Testing . as well as calculations when appropriate.written from a user's perspective. FunctionalTests test the entire system from end to end. D. Penetration Testing D. All standard GUI elements should be identified and validated. testing. By load testing a Web application throughout development.Determines the response time of a system with various workloads within the anticipated normal production range. Network Mapping B.

v. A test of an entire application software system conducted to ensure that the system meets all applicable user and design requirements. System testing specifically goes after behaviors and bugs that are properties of the entire system as distinct from properties attributable to components (unless. D. the component in question is the entire system). delivered by the development team. regression testing is initiated after a programmer has attempted to fix a recognized problem or has added source code to a program that may have inadvertently introduced errors.000 numbers in a matter of days. This is important because after all the units are tested individually we need to ensure that they are tested progressively. system. modules. The simplest definition of Integration Testing that I could find states that "An integration test verifies that all the parts of an application "Integrate" together or work as expected together". Selective testing of an item. War Dialing A. The approach of using personnel not involved in the development of the product or system in its testing. and systems have been properly designed and implemented. The software is of high quality. All will provide a report on the . of course. The functionality. Examples of system testing issues: resource loss bugs. Verification that current changes have not adversely affected previous functionality. Independent i. Also referred to as verification testing. ii. the software will replace/support the intended business functions and achieves the standards required by the company for the development of new systems. F. ii. performance. iii. This process is called war dialing. iv. iii. Regression testing i. security. is as specified by the business in the Business Design Specification Document and the Requirements Documentation.I. iii. or component to verify thatmodifications have not caused unintended effects and that the item. transaction synchronization bugs (often misnamed "timing bugs"). Testing that is focused on an entire end-to-end business process. System Test i. or componentcomplies with its specified requirements. ii. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 49 . Certain war dialers will even attempt some limited automatic hacking when a modem is discovered. Test which verifies that interfaces and interdependencies of products. system. The selective retesting of a software system that has been modified to ensure that any bugs have been fixed and that no other previously-working functions have failed as a result of the reparations and that newly added features have not created problems with previous versions of the software. A computer with four modems can dial 10. Integration Test i. numbers with modems. There are several software packages available (see Appendix C) that allow hackers and network administrators to dial large blocks of phone numbers in search of available modems. throughput bugs. recovery. C. The software delivered interfaces correctly with existing systems. subsystems. E. It is a quality control measure to ensure that the newly-modified code still complies with its specified requirements and that unmodified code has not been affected by the maintenance activity.discovered.

iv. System Acceptance i.iv. An Integration Test will thus allow "flaws" in the application between different Objects. If the problem were to discovered in a system test at the end of the Development cycle it would probably require more resources to correct than during the cycle. security and control tests. Often there is a deadline that drives businesses to develop new applications. The QA Team does not have to wait for the entire system to be completed before Testing is implemented but can take the various units after they have been developed and ensure that they function correctly together. and in an effort to preempt the market the time for Development and of course testing is generally shortened as the project matures. In cases where an acceptance test is conducted. and Modules etc to be uncovered while the Application is still being developed and the developers are still conceivably working in the same portion of the application. Components. if applicable. Formal testing conducted to determine whether a system satisfies its acceptance criteria and to enable the customer to determine whether to accept the system. This test phase verifies compliance with the system design objectives and tests each module/program/system against the functional specifications using the system test environment. Establishing the test in the operational environment requires coordination between the System Developer and the Information Processing Centers and is used to validate any additional impacts to the operating environment. environmental impact on the operating systems. One of the ways that the QA team contributes to the project is to perform Integration Tests on the various units as they are developed. operability tests. the user is involved in validating the acceptability of the system against acceptance criteria using the operational test environment. Test Program Development A. disaster recovery tests. iii.The Software Acceptance Test is used to test effectiveness of the documentation. the training plan. The completion of the SAT should result in the formal signing of a document accepting the software and establishes a new baseline. (SW-CMM (IEEE-STD-610)) SOFTWARE ACCEPTANCE TEST (SAT) . ii. and security. stress testing.A test of an application software system that is performed for the purpose of enabling the system sponsor to decide whether or not to accept the system. v. Testing of the system to demonstrate system compliance with user requirements. Many individuals use the terms System Testing and Integration Testing interchangeably and for simple applications that do not have many components the criteria and test scripts required to perform testing are similar. This is especially important in today's market where the drive is to be the first to market a product. But as an application increases in complexity. and size and users demand new functionality and features the need to perform Integration Test becomes more obvious. it is not conducted in lieu of a system test but in addition to a system test. and. a volume test. SOFTWARE QUALIFICATION TEST i. at the sponsor’s option. an acceptance test may or may not be conducted. Planning (Test Plan) ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 50 . B. For a given release of an application software system. Upon completion of all units a complete "System Test" is performed to ensure that data 'flows' from the beginning to the end of the Application. ACCEPTANCE TESTING . G. The SQT should include a performance test. In this test phase. a data conversion test H.

Selection of techniques and methods to be used to validate the product against its approved requirements. Data i. Test scripts describe how to perform the test. they are software testing software. ii. D.i. approaches for verification. ii. Cases i. Performance/Load Testing D. E. Many large projects require a Master Test Plan which establishes the test management process for the overall project. For many programs. Test scripts also contain expected results. ii. as well as level-specific test plans which establish protocol for each required level of testing. In fact. or other authorized entity. The chaining approach for automated software test data generation which builds on the current theory of execution-oriented test data generation. Analysis techniques used to evaluate results of testing. Development of test objective (cases). customer. Automated test scripts are sometimes referred to as test procedures. In addition to the master test plan. iii. lists items to be tested and serves as a communication device between all members of the project team. A QA Team typically creates a test plan and uses it to guide the QA team's efforts. and expected results that determine whether the software being tested meets functional requirements. System Testing C. iii. Development of test data. Test cases describe what you want to test. A distinction must also be made between manual and automated test scripts. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 51 . these projects may include test plans for: A. You can also carry this concept to manual test scripts by keeping the test script free of specific test data Development. In the chaining approach. a test script may be placed in a loop and read many different test cases from test data files. A test plan provides an overview of the project. includes planning for regression testing. These automated test scripts or procedures closely resemble source code. User Acceptance Testing The criteria that a system or component must satisfy in order to be accepted by a user. execution. test data are derived based on the actual execution of the program under test. the execution of the selected B. C. The plan also identifies sufficient and proper tests to assure that previously tested related functions will execute properly. execution conditions. Acceptance Criteria i. A specific set of test data and associated inputs. and evaluation of procedures used for testing. ii. Test Procedure . including techniques. ii. In test automation. but usually in more general terms. and validation of cases.Defines the procedures to be followed when applying a test suite to a product for the purposes of conformance testing. (SW-CMM (IEEE-STD-610)) “Acceptance Criteria" means the written technical and operational performance and functional criteria and documentation standards set out in the project or test plan. Unit Testing B. A test case typically describes detailed test conditions that are designed to produce an expected result. Procedures i. Tools related to generation of test data. What is the difference between Test Cases and Test Scripts? A. iii. Rule-based software test data generation is proposed as an alternative to either path/predicate analysis or random data generation.

The main benefit of a test script is that it predefines a procedure to follow in performing a test. and what subsets of features comprise a useful solution. D. Specifications i. Improve customer satisfaction since their expectations are met or exceeded. ii. iii. A well-understood specification reduces unplanned features and informs developers where future features will be needed. Analysis Techniques i. The level of predictability of how a user will interact with a web interface. The intensity of the test. Sometimes you want the randomness of user actions. ii. so the design can allow for them. At the same time. Documentation of the steps to be performed in testing. The degree of freedom a user is intended to have in interacting with a web interface. Tools and methods to access test results. This is the classic tradeoff between test scripts and test cases. Focus should be on the purpose and preparation.. Scripts i. B. The test case specifications should be developed from the test plan and are the second phase of the test development life cycle. data dependence analysis automatically identifies statements that affect the execution of the selected statement. and confirming these subsets. The importance of documenting a test of a specified sequence. preparation. Test scripts describe how to perform the test. The test specification should explain "how" to implement the test cases described in the test plan. you want to know in advance the conditions to be tested and how they should behave. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 52 . The existing methods of test data generation may not efficiently generate test data for these types of programs because they only use control flow information of a program during the search process.statement may require prior execution of some other statements.e. Knowledge should cover purpose. iv. The decision to use test cases versus test scripts depends on: A. By dividing the full feature set into useful subsets. A good software specification document can: A. and usage. H. The chaining approach uses these statements to form a sequence of statements that is to be executed prior to the execution of the selected statement. The importance of sequence in the user’s correct performance of a task. i. It describes the features the new product should have. you can better plan a staged delivery that will test your assumptions and validate your design. This can also be its greatest curse. C. The chaining approach uses data dependence analysis to guide the search process. Determine what features are most important. B. Creation of test specifications. iii. C. F. E. D. Reduce the time needed to complete the project by determining the usability of the system and providing the customers with a realistic expectation of what the system will do — before it is built. The experiments have shown that the chaining approach may significantly improve the chances of finding test data as compared to the existing methods of automated test data generation. G. A software specification document is crucial for a successful project.

and interrupt handlers. Condition Coverage . Test Logs/Traces Identify any remaining (open) deficiencies. for each recommendation. Provide any recommended improvements in the design. iv. Data flowanalysis provides interesting information about the structure of the code that can be used for deducing static properties of the code and for deriving coverage information. Code coverage analysis is sometimes called test coverage analysis. Knowledge of purpose. Structural testing examines how the program works. this measure includes coverage of switch-statement cases. Structural testing is also called path testing since you choose test cases that cause paths to be taken through the structure of the program. v. Management Summary B. Do not confuse path testing with the path coverage measure. Test Results Analysis C. If no recommendation is provided state. separated by logical-and ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 53 .The following sections should be included in the results analysis: A. branch or path level. operation. iv. taking into account possible pitfalls in the structure and logic. and test coverage tools used for monitoring the execution of software and reporting on the degree of coverage at the statement. Functional testing examines what the program accomplishes. Provide an assessment of the manner in which the test environment may be different from the operational environment and the effect of this difference on the test results. vi. iii. The Test Results Analysis Report is an analysis of the results of running tests. Code coverage analysis is a structural testing technique (AKA glass box testing and white box testing). C. Structural testing compares test program behavior against the apparent intention of the source code. without regard to how it works internally. B. Problem/change reports may be used to provide deficiency information. exception handlers.Condition coverage reports the true or false outcome of each boolean sub-expression. iii. Additionally. c. The entire boolean expression is considered one true-or-false predicate regardless of whether it contains logical-and or logical-or operators. limitations. For each remaining (open) deficiency. The results analysis provide management and the development team with a readout of the product quality. The two terms are synonymous. Decision Coverage . which compares test program behavior against a requirements specification. or constraints that were detected by the testing performed. or constraint.This measure reports whether each executable statement is encountered. including identification of requirements not met 2) The impact on system design to correct it 3) A recommended solution/approach for correcting it.ii. Test Completion Criteria A. describe: 1) Its impact on system performance. or testing of the system tested. Likewise. A large variety of coverage measures exist.This measure reports whether boolean expressions tested in control structures (such as the if-statement and while-statement) evaluated to both true and false. C. indicate: None. This contrasts with functional testing (AKA black-box testing). Describe the impact on the system. v. Code coverage i. a coverage analyzer is sometimes called a coverage monitor. The academic world more often uses the term "test coverage" while practitioners more often use "code coverage". ii. limitation. Statement Coverage . methods. Here is a description of some fundamental measures: A.

This "linear" sequence can contain decisions as long as the control flow actually continues from one line to the next at run-time. Predicate coverage views paths as possible combinations of logical conditions.Condition/Decision Coverage is a hybrid measure composed by the union of condition coverage and decision coverage. Also known as predicate coverage. Broad. One disadvantage is that this measure does not include decision coverage. F. B.vi. Decision coverage includes statement coverage since exercising every branch must lead to exercising every statement. Researchers refer to the coverage ratio of paths of length n LCSAJs as the test effectiveness ratio (TER) n+2. and logical-or if they occur. Condition/Decision Coverage .This measure reports whether each of the possible paths in each function have been followed. without requiring a flow graph.Also known as MC/DC and MCDC.Multiple condition coverage reports whether every possible combination of boolean sub-expressions occurs. The advantage of this measure is that it is more thorough than decision coverage yet avoids the exponential difficulty of path coverage. Condition coverage measures the subexpressions independently of each other. the sub-expressions are separated by logical-and and logical-or. C. Multiple Condition Coverage . Linear Code Sequence and Jump (LCSAJ) Coverage . E.This variation of path coverage considers only the sub-paths from variable assignments to subsequent references of the variables. The advantage of this measure is the paths reported have direct relevance to the way the program handles data. Call Coverage . As with condition coverage. when present. The test cases required for full multiple condition coverage of a condition are given by the logical operator truth table for the condition.This variation of path coverage considers only sub-paths that can easily be represented in the program source code. A path is a unique sequence of branches from the function entry to the exit. Condition/decision coverage includes decision coverage and condition coverage (by definition). BullseyeCoverage measures function coverage. D. H. The hypothesis is that faults commonly occur in interfaces between modules. shallow testing finds gross deficiencies in a test suite quickly. Data Flow Coverage . Predicate coverage includes path coverage and multiple condition coverage. This measure requires enough test cases to verify every condition can affect the result of its encompassing decision. Path coverage includes decision coverage. Also known as call pair coverage C. A. The disadvantage is that it does not avoid infeasible paths. Function Coverage . as well as most other measures. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 54 . It is useful during preliminary testing to assure at least some coverage in all areas of the software. An LCSAJ is a sequence of source code lines executed in sequence. Path Coverage . D. G. This measure was created at Boeing and is required for aviation software by RCTA/DO-178B.This measure reports whether you executed each function call. Here is a description of some variations of the fundamental measures and some less commonly use measures: A. You can compare relative strengths when a stronger measure includes a weaker measure. B. D.This measure reports whether you invoked each function or procedure. Modified Condition/Decision Coverage . Sub-paths are constructed by concatenating LCSAJs.

It is useful for testing multi-threaded programs such as in an operating system.This measure is similar to relational operator coverage but much more general. An SRE is used to identify and categorize specific program risks emanating from product.This measure reports whether you executed each loop body zero times. The SEI Software Risk Evaluation (SRE) Service is a diagnostic and decision-making tool that enables the identification. It works by reporting coverage of conditions derived by substituting (mutating) the program's expressions with alternate operators. resources. For example.This measure reports whether each machine language conditional branch instruction both took the branch and fell through. iii. process. Researchers have proposed numerous variations. Race Coverage . Loop Coverage . management. Requirements. information not reported by others measure. Each of these categories is subdivided further. >.This measure reports whether multiple threads execute the same code at the same time. and between local and global variables. Knowledge of risk assessment and risk abatement techniques used in the testing process. and constraints. One way of identifying software project risks is using a questionnaire such as the SEI TaxonomyBased Risk Identification Questionnaire. loop coverage reports whether you executed the body exactly once. It reports whether test cases occur which would expose the use of wrong operators and also wrong operands. analysis. Relational Operator Coverage .Another disadvantage is complexity. Scale) asks: "Is the system size and complexity a concern?". and Program Constraints. pointers also present problems. As with data flow analysis for code optimization. It helps detect failure to synchronize access to resources. such as "-" substituted for "+". ii. variations distinguish between the use of a variable in a computation versus a use in a decision. The valuable aspect of this measure is determining whether while-loops and for-loops execute more than once. I. >=). The hypothesis is that boundary test cases find off-by-one errors and mistaken uses of wrong relational operators such as < instead of <=. The Software Risk Evaluation (SRE) is a service that helps projects establish an initial baseline set of risks and mitigation plans¾ one of the key first steps for putting risk management in place.This measure reports whether boundary situations occur with relational operators (<. Object Code Branch Coverage . analysis. and more than once. Risk i. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 55 . E. <=. mitigation. all of which increase the complexity of this measure. B. Risks to a software project must first be identified. For example. The Taxonomy-Based Questionnaire is structured into three main areas of software risk: Product Engineering. This measure gives results that depend on the compiler rather than on the program structure since compiler code generation and optimization techniques can create object code that bears little similarity to the original source code structure. and more than once (consecutively). and communication of risks in software-intensive programs. and mitigation of risks facing their own development effort. Weak Mutation Coverage . the thirteenth question on the Questionnaire (Product Engineering. F. Development Environment. and with alternate variables substituted. For do-while loops. narrowing the focus on particular aspects of risk. exactly once. The program's own personnel participate in the identification. H. tracking. G.

iii. Cost risk E. C. project performance. Specify why the risk is important? B. The cumulative average time that a manufacturer estimates between failures or occurrences in a component. The risk must be analyzed After analyzing software risks. and/or the potential impact of the risk may be mitigated by dealing with the problem early in the project. Process improvement focuses on building the product right. Technical risk (performance related) B. and schedule can be further segmented into five risk areas. Who is responsible for the Risk Management activity? D. implement strategies to deal with those risks Risk management is currently a key process area (KPA) in the Systems Engineering CMM® and the Software Acquisition CMM. Next. and tools for managing risks in a project. The five risk areas are: A. v. A trigger should be set up. ix. develop a mitigation/contingency plan or accept the risk. Quantify your risks by determining the likelihood of an event and the potential consequence to the ISS. A detailed plan of how the risk will be prevented and/or mitigated is created. the process must continue. expressed in MTBE (Mean Time Between Errors) and availability expressed as MTBF (mean time between failures). activity improvement. Planning stages should cover the following: A. Reliability. Action planning can be used to mitigate the risk via an immediate response. managing variability. The probability of the risk occurring. Risk management and process improvement are complementary. Programmatic risk (environment related) D. and uncertainty. It is a Process Area (PA) at Maturity Level 3 in the CMM IntegrationSM (CMMISM) staged model. F. Supportability risk (performance related) C. conformance. assess continuously what could go wrong (risks) B. Schedule risk Risk Identification What key technical area or process is at risk? Risk Analysis Determine the root cause of the risk. vi. innovation.iv. xi. ii. Risk Management is a practice with processes. and control Software Risk factors that impact a product’s performance. Understanding of mean time between errors as a criterion for test completion. The reliability of a system is measured by the probability of errors. What resources are needed to perform the activity? E. What information is needed to track the status of the risk? C. Once risks have been identified in some manner. determine which risks are important to deal with C. G. iv. Risk Communication Provide status of the risks on a regular basis. methods. the contingency plan is put in effect. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 56 . and if the trigger is reached. It provides a disciplined environment for proactive decision making to A. However. viii. x. a plan should be formulated to address each risk. managing change. Contingency planning can be used to monitor the risk and invoke a predetermined response. vii. any given risk may have an impact in more than one area. Risk Abatement What can you do about a risk? Identify possible solutions. Error rate i. Risk management focuses on building the right product. cost.

teams can speed time to market with efficient parallel development. and guidelines).The "down-time" during which the managed application is unavailable due to failure. production b.v. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 57 . As PVCS Professional facilitates communication. and PVCS Configuration Builder for automated software builds. Software change control is both a managerial as well as technical activityand is essential for proper software quality control. Change control over the project management environment that projects function within (i. hardware and operating system (support) software change control. Tracks and communicates issues across the enterprise iii. implementation and operations i. integration. New software and changes to existing software should be prepared in the Development Environment by appropriately authorized development or applications support staff. PVCS Tracker. d. implemented. 2. Migration of software between environments should only be undertaken after obtaining the appropriate sign-offs as specified in the Software Change Control Procedures. three separate environments should be maintained for each strategic system : i. scheduled. closed. automate software configuration management tasks. etc. Reliability is a measure that indicates the probability of faults or the mean time between errors. Once assessed as satisfactory. i. Environment. Example Tool: a. Applications should be specified. approved or rejected. Where possible.. the new or modified software should be transferred to the Testing Environment for systems and acceptance testing by an appropriate testing group. A. Merant PVCS Professional gives teams the power to organize and manage software assets. design. Changes to software are not permitted in the testing environment. Professional combines PVCS Version Manager. testing iii. 1. of a particular software project. plans. F. Also. storing and changing system items that pass through development. according to an agreed test procedure. standards. manages and protects software assets ii. Change control over the requirements. development ii. Mean Time Between Failures . vi. and manage workflow tasks involved in team collaboration. A complete package for software configuration management. coordinates tasks and manages changes. evaluated. and tracked. Organizes. procedures. PVCS Professional enables organizations to protect development assets. designed and coded according to systems development methodology. Project. PVCS Professional combines PVCS Version Manager. track and communicate issues and standardize the software build process. code. The process by which a software change is proposed. ii. 1. Automates software builds for standardized. At the project level these activities should be included as part of the project plan or in a software change control plan. Software Change Control 1. Change control procedures cover the establishment of methods for identifying. c. Change Control Environment a.e. PVCS Tracker and PVCS Configuration Builder in an integrated suite for software configuration management. 2. documentation.. more code re-use and fewer errors. repeatable development success.

the version number will be identified by two or three digits e. problems within the team development environment stem from the inability to work comfortably in a setting sensitive to their projects and source code. Visual SourceSafe also provides many advanced features for Web site management. c. b. Java class/package or even complete projects has which functionality. Team members can also reconcile conflicts between different versions of the same file by using a visual merge capability. Share and linking capabilities promote the reuse of code and components across projects and simplify code maintenance by propagating changes across all shared and linked files whenever a file is updated. its second point release and its first mini release or patch. software developers have a generally agreed code of practice with regard to software versioning.2. the costs and overhead associated with file-based version control often outweigh the benefits. including extensive deployment support. Historically. Although not a global standard per se. A contingency plan to enable the software to be restored to its previous version in the event that the implementation is unsuccessful should be prepared where appropriate. Visual SourceSafe enables teams to develop with the confidence that their projects and files will be protected. which provides a point-and-click interface for uniting files and avoids potential loss of valuable changes. enable teams to fork the development process into parallel projects and files. and the software life cycle grows. Following successful completion of testing and approval by the appropriate systems custodian. Defect Management 1. (version) 1. d. all changes and documentation are secured by Visual SourceSafe. In SVC you can specifacally tell which version of a program. such as branching. the new or modified software should be transferred to the Production Environment for implementation under the control of IT Operations staff. Versioning features. and types of reporting. Visual SourceSafe can be used to create site maps and check hyperlinks. Version Control. By providing project-oriented software management. As revisions are made. Knowledge of controlling multiple releases of configuration items. Difference reporting provides quick access to changes across separate versions of the same file.g. Parallel development features.NET. G. files are added and modified. Corrective Action on Defects. Visual SourceSafe 6. 2. Defect Recording. Identification of the most common sources of information and the different methods. individual projects. iii. 3. module. 1. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 58 . 1. Tracking And Correction i. easily accessible to even the novice user. Maybe version management is more appropriate. Defect Reporting and Tracking. a. Additionally. provide snapshots of a project for the quick retrieval of any previous version in the software life cycle. creating identical copies that inherit all versioning documentation but may be tracked as new. enabling a deeper degree of software reliability. enabling developers to know immediately what lines of code have changed.e. a. In general. providing an audit trail for every file and every project. such as labels.0c is the ideal version control system for any development team using Microsoft Visual Studio® . frequency.1 This example indicates that the software is in its first major release. While every project requires an adequate level of software management.

j. B. well documented. continually updated and followed. Dynamic economic. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 59 . Today’s auditors are becoming more proactive and coactive. and joint projects with the IT audit section of the organization’s internal auditing department. Helps in improving processes IT management must accept the responsibility for nonconformance i. Requirements documents are defect prone iii. delivered at any time. Produce products and services on-time at the lowest possible cost i. standards and procedures must be developed. Everyone must ‘buy in’ into the quality principles & methods Reduce product inspections and testing by building processes that produce defect-free products. F. i. IT policies. rather than reactive. Knowledge of IT audit function and how to effectively coordinate with work schedules.i. Analyzing problem data and using problem-solving principles such as identification of problems. Customer is always right ii. worldwide demand for knowledge. IT Auditing Coordination A. the risks and exposures involved and how they affect audit plans and the audit. In an atmosphere like this it is critical that all auditors understand the impact of Information Systems on control and auditing. Uniqueness of information technology ii. will not satisfy customers Create enthusiasm and cooperation between management and staff for quality i. h. it can not be controlled The goal of IT management and staff must be to produce defect-free products & services Non-conformance must be detected as early as possible. Auditors must know what the new technologies are. Quality is a binary state ii. If Quality is not measured. establishment of applicable objectives. Today’s auditors must be fully integrated auditors. Cannot survive without customers b. Identify customer’s true needs and update the requirements document. e. f. high tech has created new roles and responsibilities for everyone in the audit function. 80% of all defects are directly attributable to ineffective processes The customer’s view of Quality is the correct view of Quality i. No longer the exclusive domain of the Information Systems department or even Information Systems Auditors. c. Quality must be defined quantitatively i. if needed. understanding information systems and able to function effectively within a technical environment. 2. Everybody’s responsibility ii. recorded and measured i. g. In their broadened role they participate with management in strengthening the overall control framework. political and social forces are creating an urgent. Quality at any cost. Economic issue ii. response to implementation of IT audit recommendations. and defining/documenting and applying the appropriate solution. Meet Customer’s True Quality Needs i. d. QAI Recommended Quality practices a.

Tailoring processes i. An implementation of the standards group in which the members are not part of a fixed group. Implementing a process i. The purpose of a process is to produce results such as products or services. Measurements of process performance are used to evaluate the ability of a process to produce products or services with the characteristics we desire. We measure the results and the ways in which they are delivered to determine quality. b. c. quantity.the state of chaos iv. and how to define a process and how to continuously improve process capability. We must constantly improve our ability to produce quality products that add value to your customer base. energy. and procedures working together in a specified manner to produce an intended result. ii. Standards needs assessment i. across some or all of the enterprise. but come from the other standardized portions of the enterprise to perform the standards function. Nonconforming and unpredictable -. Implement.g. The group provides a safe and supportive framework for employees at all levels to learn to communicate openly and to work constructively in any environment where the achievement of common goals is important. and timeliness of the products and services. selection of processes to build. using on-line terminals rather than paper manuals. Determination of the needs of the enterprise for new or modified standards. In other words. cost. The methods for process mapping. and others. customers are more knowledgeable and demanding. c. acquire. The four possibilities for any Process: A. Process development process i. and install process. Build.” a. The identified group within the enterprise that has the responsibility and authority to identify.the ideal state B. This category will test the candidate’s understanding of process components. Compliance and Enforcement a. d. Maintenance of approved processes in an automated environment. and quality and speed of delivery are now critical needs. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 60 . A process is defined as any set of conditions or set of causes that work together to produce a given result. Modifying existing processes to better match the needs of a project or environment..the threshold state C. equipment. b. Process development group i. Conforming and predictable -. or generate.the brink of chaos D. Implementing a newly defined process is as complex and risk laden as defining the process and includes training. Administering Processes a. b. The group method improves working practice and outcomes and is likely to contribute to far greater job satisfaction. Conforming yet unpredictable -. Use statistical tools to indicate the degree to which a process is “in control. materials. c. a. e. or for the elimination of obsolete or non-beneficial standards. The world is constantly changing. and Improve Work Processes A.7. Nonconforming and predictable -. Developing/Building Processes b. and the procedures to build processes. On-line standards i. Defining and continuously improving work processes enables you to maintain the pace of change without negatively impacting the quality of your products and services. Waiver b. Define. help to define process performance. d. The Process Development Group has bee developed by Richard Reynolds at Indigo Rose as a highly effective way of enabling people to be more productive working in a team. a process is a system of causes: the people. These characteristics. Process committee i.

e. process. and product and service quality. W3C is a forum for information. process capability. or the process itself. product quality. Quantitative Methods A. Standards developed outside the enterprise that may affect the operation.g. and ISO/IEC 15504 b. 8. Technical standards i. A. A properly established measurement system is used to help achieve missions. This category will test the candidate’s understanding of measures and how to build an effective measurement program. IEEE). and objectives. Establish process measures 1. SPC is a method of monitoring a process during its operation in order to control the quality of the products while they are being produced -.. The quality assurance professional must ensure quantitative data is valid and reliable and presented to management in a timely and easy-to-use manner. customer satisfaction. In some cases. This is done in order to ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 61 . B. products. the tools can correct the noncompliance. Probability and Statistics 1.. Knowledge of where to find standards developed outside the enterprise (e. and quality management at all levels.rather than relying on inspection to find problems after the fact. 2. Process Improvement Methods a. FAA. i. evaluating them. Automated process enforcement i. f.provides objective information about. B. c. and providing management with the information and techniques to introduce beneficial modifications to the process. Use of measures and other information allow organizations to learn from the past in order to improve performance and achieve better predictability over time. industry standards and specifications.. statutory regulations. The method by which release from the requirements of a specific process may be obtained for a specific situation. effectiveness and efficiency of processes. What gets measured gets done. The World Wide Web Consortium (W3C) develops interoperable technologies (specifications. iii. DOD. Measurement data is most reliable when it is generated as a by-product of producing a product or service. communication.c. on a near real-time basis so that the operator can take action on the process. project performance. goals. i. Statistics are used for both determining whether or not the processes under control (i. Acquisition and customization i. Statistical methods used to monitor process performance. Acquiring externally developed standards and adapting them for beneficial use within the enterprise. Sources i. ii. which may be useful to the enterprise. within acceptable variance from standards) and to help identify the root cause of process problems that are causing defects. and tools) to lead the Web to its full potential.g. and visibility into. commerce. Measurement can be used to measure the status of processes. and other public domain standards). Software development standards include: IEEE. Measurement . The Capability Maturity Model (CMM) certainly affirms this viewpoint and represents measurement practices as critical components of project. or opportunities of the enterprise (e. Locating potential process improvements. Collecting measurement data on process performance in use of process improvement. software. and as a tool for management to use in their decision-making processes. Statistical process control (SPC). process performance. It involves gathering information about the product.e. The use of precompilers and other tools to detect noncompliance. Externally Developed Standards a. and collective understanding. guidelines.

It is hard to detect and reduce. equipment. b. the so-called "assignable causes. Managers need to determine whether a production system is undergoing only random fluctuations in its operation. There is an increasing demand for managers and workers who understand and are able to apply Statistical Process Control methods. are called variation. Fluctuations in the process performance come from two sources. Even a superior product or service can be improved on. Statistical Process Control. who had worked with Walter Shewhart. Random variation occurs. the process may be subject to large and unusual changes in the cause system from time to time which result in non-random variation in the process performance. as its name implies. W. etc. Notice that improvement is a never-ending cycle. Dr. is a tool that businesses and industries use to achieve quality in their products and/or services. Assignable variation is easy to detect and easier to reduce than random variation because its causes are known. C. or whether non-random deviations. This is not to say that one day management decided to make inferior goods. or machine to machine. 80 percent of all quality problems are due to management. a. Variation .) are responsible for random variation in the process performance and is referred to by Shewhart as common cause variation. According to Dr. On the other hand. The problem solving cycle shown below illustrates the process of continual improvement. Random variation is inherent in a system. a set point change. Random and assignable causes A. Universally. poor calibration of the instruments. physical and mental reactions of people. operator." are occurring—to be tracked down and eliminated. due to random causes or chance. businesses and industries use mathematics and statistical measurements to solve problems. or the process output over time. assignable causes can almost always be eliminated. environment. Today US businesses are in the process of implementing SPC to build quality into products and services.Differences exist from product to product. Dr. Define Problem Proposal Pareto chart Implemen t Solution Analyse Data Collect Data Define Process List Possible Flow Chart Analyse Charts Brainstorm Ranking Fishbone Sampling D. Statistical methods used to differentiate normal variance in the operation of processes (random). Deming. These differences among products. B. thus bringing the process under statistical control and reducing variation. bad bearings. by its methods of operation. etc. has built defects into the process. To assist management. the presence of an impurity. Assignable causes also can include factors too minor to bother about. Assignable causes in converting operations could include a change in raw materials. or other specific causes. taught SPC to the Japanese after World War II. procedure. out-of-round rolls. Fluctuations over time in the inherent process cause systems (differences in material. Once all assignable causes are removed from a process. SPC for short. Top management. C. Such variation is referred to as assignable or special cause variation since the variation is generally due to causes that could have been ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 62 . Top managers make important decisions for companies and have the most influence on the future of the business. D. Deming has created 14 points to serve as a guideline. then the process is in statistical control. Random causes rarely can be eliminated. person to person. from variances that are associated with the root cause (assignable). 2. Edwards Deming.identify special causes of variation and other non-normal processing conditions. Assignable variation in the product performance occurs due to a change in machine setup. chemicals.

When assignable causes are removed. B. mean is the value where a process is expected to operate or the target value.prevented. F.. mean and standard deviation. as a statistical limit. meaning that they can be derived from examination of the software itself (usually in the form of source or object code. and are due to sudden or persistent anomalies within one or more components of the cause system. Other measures can be characterized as dynamic. When variation in process performance includes assignable cause variation. Standard deviation. project milestones. coding. Measures and Metrics a. Problem characteristic analysis A. or pre-control charts. Product size: count lines of code.e. In the industrial environment. number of requirements. object classes. When the cause system is constant. Defects: count the number found by testing and by customers and their type. Normal distribution is characterized by two parameters. requirements specification. the observed distribution of the process performance variation tends to approach. meaning that they can only be derived from observation of the execution of the softwareComputer scientists and software engineers have done a lot of research ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 63 . When process performance is limited to common cause variation. Average values are plotted to monitor the process output using tools such as trend charts. Process mean. Common cause variation of process performance is characterized by fluctuations that are random. the normal distribution is used to predict the probability of producing defective product. The limits are empirically derived from measurements of the variation in the process performance over time 3. corrective) d. i. Statistical methods used to accumulate and analyze problems incurred as a result of operating processes. and vary within predictable bounds. The control charts employ upper and lower control limits (UCL and LCL) to delineate or filter assignable cause variation from common cause variation. the standard deviation. C. A. and overall product development c. Characteristics of measures and methods i. testing) and maintenance activities (adaptive. Standard deviation is the square root of the mean sum of the squares of the deviations from the mean. In manufacturing operations. Some software measures can be characterizes as static. perfective. in statistical control or stable. B. Shewhart's control charts are the primary vehicle used to analyze process performance variation. and status (open or closed) B. Work effort distribution: record the time spent in development activities (project management. function points. Standard deviation is widely used to quantify the variability of a process. or perhaps in terms of a design document). The standard deviation is a measure of inconsistency in a process. Calculating probabilities using the normal distribution requires the estimate of the process mean. a distribution function of some sort. Estimated and actual duration (calendar time) and effort (labor hours): track for individual tasks. G. process variation will decrease with future execution of the process and the process will become stable and predictable. A process capability is defined as six times standard deviation. severity. A. The total variation that may be observed in process performance is expressed by the equation: Total variation = Common cause variation + assignable cause variation E. design. the process is no longer predictable. Type of software measurements: a. control charts. it will be within a distribution function and is therefore predictable. or GUI elements b. Assignable cause variation arises from events that are not part of the normal process. The definitions and concepts.

man-month. academia. There are a few other software properties that are generally believed to be important but which we don’t yet know how to measure very well. e. i. it must be total commitment. and so conclude which of the two measures is more desirable. identified four direct measures and several indirect measures that software engineering organizations can use to improve their software development processes. For example. This includes allocating staff as well as tools. usability.trying to define the important measures of software engineering. The calculation of the measure is repeatable and the result is insensitive to minor changes in environment. and government. staff-week. such as zero for number of errors. one measure alone is insufficient to measure the features of the design paradigm or to accomplish the objectives of the software project. Before examining the details of software measures (often called metrics). c. many of the potential benefits that an organization can derive from a sound measurement program is often not achieved due to a half-hearted commitment by managers to a measurement program. assisted by more than 60 specialists from industry. effort (labor-month. let's consider which properties of a measure. control. The policy must be followed with the allocation of resources to the measurement program. and portibility. a federally funded research and development center at Carnegie Mellon University. Among these are reliability and complexity. The measure should be robust. The properties or attributes of software that are directly measurable are size (source lines of code (SLOC)). Software metrics are measurements made on a software artifact. C. which can be defined in several ways. A measure is a numerical value computed from a collection of data. staff-hour). There is a scale upon which we can make a comparison of two measures of the same type. then smaller is better for all other types of measures in the suite. The commitment cannot be just a policy statement. and quality(freedom from defects. The measure should suggest a norm. and bounds. Finally. One of the most significant efforts was undertaken over the last four years at the Software Engineering Institute (SEI). there are other attributes of software that seem important but that we don’t know how to measure at all. A suite of measures should be consistent. Clearly. or observer. a performance measure is a dynamic software measure. and the process of collecting the data for the measure is objective. If a smaller value is better for one type of measure in the suite. and monitoring of projects • Better understanding of both the software development process and the development environment • Identification of areas of potential process improvement as well as an objective measure of the improvement efforts • Improved communication E. A measure should have the following characteristics to be of value to us: a. A suite of measures adds an additional consideration. This suggests that a collection or suite of measures is needed to provide the range and diversity necessary to achieve the software project's objectives. stability for use). Another property whose measure is widely regarded as fundamentally important is performance. d. Researchers at the SEI. D. in general. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 64 . and there should be a rationale for collecting data for the measure. Often. The measure is precise. The measure relates to the product. schedule. Organizations with successful measurement programs report the following benefits: • Insight into product development • Capability to quantify tradeoff decisions • Better planning. The measure should be meaningful. These include maintainability. there is a realistic lower bound. tool. scale. However. that are reasonable. b.

F. Measure - n. A standard or unit of measurement; the extent, dimensions, capacity, etc.
of anything, especially as determined by a standard; an act or process of measuring; a result of measurement. v. To ascertain the quantity, mass, extent, or degree of something in terms of a standard unit or fixed amount, usually by means of an instrument or process; to compute the size of something from dimensional measurements; to estimate the extent, strength, worth, or character of something; to take measurements. Measurement - The act or process of measuring something. Also a result, such as a figure expressing the extent or value that is obtained by measuring. Techniques or methods that apply software measures to software engineering objects to achieve predefined goals. A measure is a mapping from a set of software engineering objects to a set of mathematical objects. Measurement goals vary with the software engineering object being measured, the purpose of measurement, who is interested in these measurements, which properties are being measured, and the environment in which measurement is being performed. Examples of measures include software size, Halstead's software science measures, and McCabe's cyclomatic complexity. Associated models include sizing models, cost models, and software reliability models. Data Definition Frameworks (DDF) are primarily used to define measurements as well to communicate more effectively what a set of measurements represent. Secondary DDF uses include assistance for: identifying issues that can be used to focus data analysis designing databases for storing measurement data developing data collection forms A DDF can be used to define a set of measurements. For example, a single DDF can be used to identify a line of code measurement, i.e., identify what is to be counted. A DDF can also be used to help communicate what has been counted. A DDF does this by allowing a user to identify specifically what was included and excluded in a measurement. For example, if I have a count of lines of code, say 317,300 lines of code. The DDF helps me communicate what that number represents by identifying what types of code were counted and included in that number and what types of code were specifically not counted, i.e., excluded.

G. H.

I.

b. Complexity measurements
i. Quantitative values accumulated by a predetermined method that measures the complexity of a software product, such as code and documentation. ii. Software complexity is one branch of software metrics that is focused on direct measurement of software attributes, as opposed to indirect software measures such as project milestone status and reported system failures. There are hundreds of software complexity measures, ranging from the simple, such as source lines of code, to the esoteric, such as the number of variable definition/usage associations. iii. An important criterion for metrics selection is uniformity of application, also known as "open reengineering." The reason "open systems" are so popular for commercial software applications is that the user is guaranteed a certain level of interoperability-the applications work together in a common framework, and applications can be ported across hardware platforms with minimal impact. The open reengineering concept is similar in that the abstract models used to represent software systems should be as independent as possible of implementation characteristics such as source code formatting and programming language. The objective is to be able to set complexity standards and interpret the resultant numbers uniformly across projects and languages. A particular complexity value should mean the same thing whether it was calculated from source code written in Ada, C, FORTRAN, or some other language. The most basic complexity measure, the number of lines of code, does not meet the open reengineering criterion, since it is extremely sensitive to programming language, coding style, and textual formatting of the source code. The cyclomatic complexity measure, which measures the amount of decision logic in a source code function, does meet the open reengineering criterion. It is completely independent of text formatting and is nearly independent of programming language since the same fundamental decision structures are available and uniformly used in all procedural programming languages. iv. Ideally, complexity measures should have both descriptive and prescriptive components. Descriptive measures identify software that is error-prone, hard to understand, hard to modify,
____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 65

hard to test, and so on. Prescriptive measures identify operational steps to help control software, for example splitting complex modules into several simpler ones, or indicating the amount of testing that should be performed on given modules. v. There is a strong connection between complexity and testing, and the structured testing methodology makes this connection explicit. A. First, complexity is a common source of error in software. This is true in both an abstract and a concrete sense. In the abstract sense, complexity beyond a certain point defeats the human mind's ability to perform accurate symbolic manipulations, and errors result. The same psychological factors that limit people's ability to do mental manipulations of more than the infamous "7 +/- 2" objects simultaneously apply to software. Structured programming techniques can push this barrier further away, but not eliminate it entirely. In the concrete sense, numerous studies and general industry experience have shown that the cyclomatic complexity measure correlates with errors in software modules. Other factors being equal, the more complex a module is, the more likely it is to contain errors. Also, beyond a certain threshold of complexity, the likelihood that a module contains errors increases sharply. Given this information, many organizations limit the cyclomatic complexity of their software modules in an attempt to increase overall reliability. B. Second, complexity can be used directly to allocate testing effort by leveraging the connection between complexity and error to concentrate testing effort on the most error-prone software. In the structured testing methodology, this allocation is precisethe number of test paths required for each software module is exactly the cyclomatic complexity. Other common white box testing criteria have the inherent anomaly that they can be satisfied with a small number of tests for arbitrarily complex (by any reasonable sense of "complexity") software.

vi. Cyclomatic Complexity Metric (McCabe & Associates, Inc.) A. Cyclomatic Complexity is a measure of the complexity of a module's
decision structure. It is the number of linearly independent paths and therefore, the minimum number of paths that should be tested. B. Cyclomatic complexity measures the amount of decision logic in a single software module. It is used for two related purposes in the structured testing methodology. First, it gives the number of recommended tests for software. Second, it is used during all phases of the software lifecycle, beginning with design, to keep software reliable, testable, and manageable. Cyclomatic complexity is based entirely on the structure of software's control flow graph. C. Cyclomatic complexity measures branches in the control flow of a program. In the simplest possible code, there are 0 branches and cyclomatic complexity equals 1. For every branch, a value of 1 is added to the complexity total.

vii. Halstead Software Metrics (Dr. Maurice Halstead) A. Program Length
The total number of operator occurrences and the total number of operand occurrences.

B. Program Volume C. D. E. F. G.
The minimum number of bits required for coding the program. Program Level and Program Difficulty Measure the program's ability to be comprehended. Intelligent Content Shows the complexity of a given algorithm independent of the language used to express the algorithm. Programming Effort The estimated mental effort required to develop the program. Error Estimate Calculates the number of errors in a program. Programming Time The estimated amount of time to implement an algorithm.

____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 66

H. I. J. K. L.

Line Count Software Metrics Lines of Code Lines of Comment Lines of Mixed Code and Comments Lines Left Blank

c. Size measurements
i. Methods developed for measuring the (primarily software) size of information systems, such as lines of code, function points, etc. Also effective in measuring software development productivity. ii. The most widely used size measure is a count of source lines of code (SLOC). Unfortunately, there are as many definitions of what to count as there are people doing the counting. Some people count executable statements but not comments; some include declarations while others exclude them; some count physical statements and others count logical statements. Published information on software measures that depend on this measure is therefore difficult to interpret and compare. One SEI report says this about measurement of source code size: “Historically, the primary problem with measures of source code size has not been in coming up with numbers —anyone can do that. Rather, it has been in identifying and communicating the attributes that describe exactly what those numbers represent.” The precision of a measurement of source lines of code does not depend on the numbers used in counting (everyone agrees to use the nonnegative integers), so it must depend on what we choose to count. A comprehensive definition of what kinds of statements or constructs in a program to count is necessary before precise measurement is possible.

iii. Function Points
A. Allan Albrecht (Reference 1), in collaboration with John Gaffney, Jr. (Reference 2), designed FPs as a direct measure of functionality. FPs are a weighted sum of the number of inputs, outputs, user inquiries, files, and interfaces to a system. The latest counting rules are defined in Release 3.0 (1990) of "Function Point Counting Practices Manual," by the International Function Points Users Group (IFPUG). B. Function Points and the Function Point Model are measurement tools to manage software. Function Points, with other business measures, become Software Metrics. C. Basic function points quantify the size and complexity of an application based on that application's inputs, outputs, inquiries, internal files, and interfaces. The resulting count is then adjusted based on the complexity of the system defined by a set of general system characteristics. Since function points are independent of language, operating system, platform or development process, it avoids the problems that arise from the use of source lines of code (SLOC) to measure the size of an application. Function points have been gaining in popularity and usage in recent times. At the 1993 International Conference on Applications of Software Measurement, it was announced that function points had become the most widely used metric in the world. D. Function Points measure Software size. Function Points measure functionality by objectively measuring functional requirements. Function Points quantify and document assumptions in Estimating software development. Function Points and Function Point Analysis are objective; Function Points are consistent, and Function Points are auditable. Function Points are independent of technology. Function Points even apply regardless of design. But Function Points do not measure people directly. Function Points is a macro tool, not a micro tool. Function Points are the foundation of a Software Metrics program. E. Software Metrics include Function Points as a normalizing factor for comparison. Function Points in conjunction with time yield Productivity Software Metrics. Function Points in conjunction with defects yield Quality Software Metrics. Function Points with costs provide Unit Cost, Return on Investment, and Efficiency Software Metrics, never before available.

____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 67

This is often illustrated with an analogy to the building trades. Function Points support Customer Satisfaction measures to create Value Software Metrics. or user. Function Points can even measure the Corporate Vision and generate Software Metrics to report progress toward meeting it. Other factors.Many executives have come to the conclusion that regardless of their core business. Function Points and Usage or Volume measures create Software Metrics that demonstrate an organization's ability to Leverage software's business impact. Measure productivity -. Function Point analysis can be used for: a. Function points are not a perfect measure of effort to develop an application or of its business value. point of view. The size is measured from a functional.) Without them your organization is only two thirds whole. like location and number of bedrooms. Calculating several variations on the function points produced per month theme tells them how well they are doing in this regard. (The other two are time and money. technology or capability of the project team used to develop the application. The fact that Albrecht originally used it to predict effort is simply a consequence of the fact that size is usually the primary driver of development effort. and the Software Metrics they generate. J. It is independent of the computer language.F. H. Function points are a measure of the size of computer applications and the projects that build them. might also make the smaller house more valuable as a residence. However. All of the above Software Metrics can prove your organization is Doing Things Right! But the real and biggest value of Function Points and Software Metrics is proving you are Doing The Right Things! G. the Function Point Model. The Leverage of E Commerce is obvious. Supplemental Software Measures. but until now unmeasured. many attributes like marble bathrooms and tile floors might actually make the smaller house more expensive. are only the third measure that transcend every part of every organization. Function Point Analysis. The function points measured size. Function Points and Skill measures provide Software Metrics for Employee Service Level Agreements to meet current and future company skill needs. development methodology. A three thousand square foot house is usually less expensive to build one that is six thousand square feet. It is important to stress what function points do NOT measure. I. they are also in the software business. Function Points. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 68 . although the size in function points is typically an important factor in measuring each.

PC screens of data. At best. How then can we consider any sizing metric without considering the algorithm complexity characteristic? We approached the algorithm problem in a manner not unlike that of the late Dr. While at Purdue University. function points have been used as an estimating technique. Estimating is obviously necessary for the cost benefit analysis that justifies application development. N. Monitor outsourcing agreements -. Logical operators: AND. d. originated by Allan Albrecht in 1979 while he was with IBM. d. A+B+((C/G-E)*F)=D 4. we see the results shown in Figure 1.To put them in perspective. subtract. Even for strategic projects that need no quantitative justification. Since function points are based on screens. this measure takes the users' view Basic function points. c. Estimate development and support -. Algorithm 1. divide.000 function point system are much easier to take. some are simplistic. greater than. take longer to develop. relational operators. accurate estimation is required for proper staffing. like CSC and IBM Global Services. in effect. Just as he observed that all software contains four basic characteristics. treating them all equally. c. A*(B-D)/C = E 3. Outsourcers. but only counted the number of algorithms used. The size in function points is an attribute that needs to be tracked for each application and project. K.Since the beginning. A + B = C 2. Designing. The same 100 delivered defects on a 10. multiply. Relational operators: equal. L. reports and other external objects.Companies must analyze their portfolios of applications and projects. or any variable name. They. less than. we observed that all algorithms contain four basic characteristics: elements. arithmetic operators. other measures frequently require the size in function points. NOT. coding. debugging.b. and correctly executing those algorithms are critical to the applications and add to the complexity of the development effort. Elements: A.Companies outsourcing significant parts of their IS requirements are concerned that the outsourcing entity deliver the level of support and productivity gains that they promise. Maurice Halstead. and printed reports. Arithmetic operators: add. They may be defined as follows: a. he identified the commonality of software characteristics in his software science methods [1]. this will allow decisions regarding the retaining.C. and logical operators. Along with other data. M. For example. If we count the number of elements and operators in any given algorithm. IF (D lt A/B*((C/G-E*F)) AND A/B gt 0 THEN D=0 5.D. some are more complex. b. Drive IS related business decisions -.B. OR. were designed primarily for business applications using disk files. The evolution of feature points included algorithms. IF (A AND B) OR Arithmetic Relational Logical Elements Operators Operators Operators Total 3 1 1 0 5 5 3 1 0 9 7 5 1 0 13 10 9 6 0 3 1 1 7 20 17 ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 69 . counting lines of code measures software from the developers' point of view. Normalize other measures -. applications algorithms abound. In embedded software engineering. unique and total operators and unique and total operands. frequently use function points to demonstrate compliance in these areas. e. exponents. retiring and redesign of applications to be made. 100 delivered defects on a 100 function point system is not good news.

usually related to system size. Values associated with numbers or types of defects.C). It is not the "silver bullet" of metrics. P. The system can be used to provide a productivity rate and used as a planning tool for new projects to provide more accurate and achievable schedules and resource allocation. and reproducible manner. It is as effective with a one-person team with a small project as it is with a 100-person team with a very large project. It provides management with a simple. reproducible method to predict how large a software project will be before it has ever been designed or coded. It is a more definitive sizing metric than using SLOC that is difficult to define and is not consistent across multiple software languages. As the algorithms increase in complexity. and consistent functional results in accordance with required action. process. we can. or development process. In the first example. or data definition in a computer program. The tracking mechanism reflects any requested changes and immediately shows the impact and the effort needed to keep the current time schedule intact. d. flexible tool to track development progress against the projected plan utilizing deliverables instead of tracking staff hours used or number of dollars left in the budget for the project. A software defect is a manifestation of a human (software producer) mistake. It avoids extensive intrusion into the developer's time allowing more opportunity for the creative function and requires little configuration. one relational operator (=). This system provides a consistent. a defect is frequently referred to as a fault or a bug. Software faults and other measures of software quality can be known only at the point the software has finally been retired from service. over time. iii. the greater the effort needed to deliver a quality product. The EFPs are then added to the Engineering Function Point Summary Sheet and included in the overall calculation of unadjusted engineering function points. consistent. efficient execution timing. collection. Only then can it be said that all the relevant faults have been isolated and removed from the software system. not all human mistakes are defects. easy to use. ii. build models based on observed relationships between faults and some other measurable software attributes. one arithmetic operator (+). We will define a software defect to be any flaw or imperfection in a software work product or software process. but it is a positive step in the right direction for measuring software in an early. such as defects/1000 function points.B. however. Q. O. A fault is an incorrect program step. One of the fundamental tenets of the statistical approach to software test is that it is possible to create fault surrogates. we can see that there are three elements (A. It is independent of software language used. Defect measurements i. so do the total EFPs. Engineering function points and its tracking system provide a sizing metric very early in the software development cycle when it can be the most useful and least expensive and can be used throughout the development cycle to check progress as it is made. and posting time. While we cannot know the numbers and locations in faults. We can consider that the larger the point total.((C AND (D OR (E OR F))) AND (G OR H)) THEN X=1 Figure 1: Algorithm Complexity. Faults are defects that have persisted in software until the software is executable. This provides us with a consistent and reproducible method of counting and differentiating algorithms when counting function points in an engineering environment. and no logical operators for a total of five engineering function points (EFPs). hardware platform. When found in executable code. not just from the standpoint of writing code. but for correct element definition and usage. nor are all defects the result of human mistakes. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 70 . It can then be tracked to ensure completion of the project on time.

Find inefficient code such as unnecessary Variants. v. Enforce programming standards. The ACSI is produced through a partnership of the University of Michigan Business School. the ACSI tracks trends in customer satisfaction and provides valuable benchmarking insights of the consumer economy for companies. the American Society for Quality (ASQ). Functionality. Achieving 100% efficiency in defect removal has only been observed twice out or more than 10. Fix that spaghetti. Project Analyzer is a complete code review and quality control tool for Visual Basic. The overall average for software defects in the United States for 2001 hovers around 5 defects per function point from initial requirements through one year of production. An optimal suite of formal inspections and test stages can top 99% in cumulative defect removal efficiency. Determination of the level of service perceived by the customer including the ability to meet requirements and overall expectation. Customer Quality Evaluation Measurement Methods a.000 projects examined. D. length of names." is published quarterly in the Wall Street Journal and provides a benchmark for success in the private sector. and have topped 85%. source code. comment to code ratio. The identification and removal of software defects constitutes the basis of the software testing process. Somewhat surprisingly. With Project Analyzer's problem detection feature.S. so the volume of delivered defects averages about 0. Estimate the quality of your code with metrics such as logical lines of code. a fact that inevitably places increased emphasis on defect related software measurements. industry trade associations. and that methodical analysis of these relationships can provide a road map for improving them. while Defect Age. vi. Errors can enter software applications from a variety of sources. Customer satisfaction i. designs. A. and check for error prone places in your project. e. A powerful economic indicator. The best organizations in terms of overall quality use synergistic combinations of formal inspections. you remove unnecessary code. often referred to as "the voice of the nation’s consumer. including requirements themselves. get recommendations for better coding style. and government agencies. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 71 . Automated Complexity Tools (example): i. formal testing. those removing more than 95% of defects before release have shorter schedules and lower costs than those removing less than 85%. Style.iv. ii. The American Customer Satisfaction Index (ACSI) was developed to provide business with this analytical tool.75 defects per function point. depth of conditional nesting. Best-in-class organizations create only about half as many defects. The U. The ACSI. The most efficient defect removal activities yet measured are formal design and code inspections. Some of the most advanced thinking in the business world recognizes that customer relationships are best treated as assets. Detect dead code and decrease your . Defect Distribution.exe by up to 100s of kB. average for removing defects prior to delivery is about 85%. Are you sure all the forms resize? How about error handling? B. vii. most forms of testing are less than 50% efficient in finding defects. the American Customer Satisfaction Index (ACSI) is a uniform and independent measure of household consumption experience. and very complete defect measurements. Defect Detection Rates and Defect Response Time metrics allow for pinpointing software inspection and testing process shortcomings. and can remove more than 96% of them before delivery to clients. and the international consulting firm. A. in that at least half of latent defects remain after the testing is finished. Code coverage and testing effort measurements complement the defect metrics and provide additional software product as well as process quality indicators. cyclomatic complexity. Established in 1994. It is important to note that excellence in software quality has a very positive return on investment. CFI Group. When otherwise similar projects are compared. Metrics. and “bad fixes” or secondary defects introduced during defect repairs. Optimization. Defect Density and Defect Type metrics allow the quantification of the quality on software modules. These activities average around 65% in removal efficiency.

For example. SLAs generally include minimum standards related to the following: A. Performance/service measures (i. Defines help desk support. A SLA contains a set of definitions that identifies what the service deliverables are. availability or downtime. Information systems are becoming increasingly more complex. The SLA is the key to setting the users’ service expectation. the ISO 9000 quality management standard requires that registered companies measure customer satisfaction. network response time. repeatable functions and processes––the output of which is delivered to users in accordance with pre-agreed performance levels.Documents service objectives. A contract between a network service provider and a customer that specifies the services the network service provider will furnish. help-desk response time. payment of "failure credits. the responsibilities of the service provider and the customer. Price or charge to the customer for use of the service vii.iii. ii. conformance requirements. iii. Supported Environment: defines the hardware. and invalid statistic analysis. The establishment of a contract with the customer to maintain an agreed upon service level for the customer’s application(s). A service-level agreement (SLA) is an informal contract between a carrier and a customer that defines the terms of the carrier's responsibility to the customer and the type and extent of remuneration if those responsibilities are not met. procedures. the following points should be addressed in the SLA: A. Chargeback systems compare costs to expected benefits and provide benefits (i. Scope: includes the purpose. the Service-Level Agreement (SLA) serves as a valuable tool in meeting this challenge by documenting the success of the system in meeting needs and expectations. Customer Satisfaction Measurement for ISO 9000 explains in a clear and simple manner how to conduct a professional customer satisfaction survey that will produce a reliable result--as well as be consistent with the requirements of ISO 9001:2000. while ensuring that end users have a factual understanding of network realities. and dependencies) C. v. and callback/repair-dispatch response time. SLAs allow organizations to measure stated objectives by comparing actual performance to the performance levels or standards specified by the SLA. (International Engineering Consortium) iv. The SLA should not describe how the service is delivered. A SLA defines the acceptable levels of information systems (IS) performance.e. and user satisfaction) B. Service-level agreements i.. the state of systems management remains constant. SERVICE LEVEL AGREEMENTS . the customer’s definition of downtime may be quite different than the definition used by the service provider. and statistics to be provided.. Customers and service providers are encouraged to discuss a number of different scenarios that may lead to compensation by the service provider if expected outcomes are not met. Customers should ensure they have an accurate understanding of the terms of the agreement vi. Many customer surveys produce misleading results due to poor questionnaire design. when they are delivered and where. background information. At a minimum. The service can originate from another part of the user’s enterprise or from a third party. major players. and reference documents B. Customers should understand what the service provider considers an acceptable level of performance. and the criteria and metrics for measuring performance. rules and regulations." monetary or otherwise) to clients if the service provider fails to achieve the agreed upon service levels.e. A SLA is a set of broadly defined. SLAs should be short yet precisely define the services and the level of services to be provided. ISPs often provide SLAs for its customers. software and network C. Technical Support: provides an overview of the support services provided during business hours and after hours. The SLA is an insurance policy of sorts. b. Services SLAs may specify often include the percentage of time services will be available. It ensures that the organization understands and works in sync with business goals. network infrastructure ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 72 .. While information systems are evolving. reliability. Constraints (i. objectives. workload. number of users that can be served simultaneously. Customers can set requirements for IS and network services and weigh them against the service cost. typically to include response time. inappropriate data collection methods. network availability. For the first time.e. serviceability.

Performance: the system functions properly. the more likely a problem or the more serious the impact of a problem. You can prioritize in a number of ways. F. Service-level Partnerships: outline the relationships established with providers c. iii. Robustness: common errors are handled improperly. the system fails. either due to usage profiles or the technical risk of the problem? c. support and other operational problems Response-time Goals: includes hardware and software orders. The priority of a risk to system quality arises from the extent to which that risk can and might affect the customers’ and users’ experiences of quality. we can group risks into four categories. Customer service departments should be staffed with resource managers and technology coordinators. hangs. understanding of the customer's problem. kills sessions. vi. and so forth. network connections. e. A basic and effective baseline customer satisfaction survey program should focus on measuring customer perceptions of how well the company delivers on the critical success factors and dimensions of the business as defined by the customer. etc. The findings of company performance should be analyzed both with all customers and by key segments of the customer population E. archive-data-special file recovery and server support Support Staffing: (within IS operations) should include security administration and customer services. The most obvious is functionality: Does the software provide all the intended capabilities? Other classes of quality risks: i. b. In other words. Quality risks: How might the product lack customer-satisfying behaviors or possess customerdissatisfying behaviors? Testing allows us to assess the system against the various risks to system quality. data back-up and recovery. Localization: problems with supported languages. Financial risks: How might the project overrun the budget? ii. Priority: How much does a failure of the system in this area compromise the value of the product to customers and users? iii. viii. (A common method is a questionnaire. courtesy of staff. vii. support and server maintenance. Use cases: working features fail when used in realistic sequences. etc. which allows the project team to manage and balance quality risks against the other three areas. One approach I like is to use a descending scale from one (most risky) to five (least risky) along three dimensions. Feature risks: How might we build the wrong product? iv. Likelihood: What are the odds that a user will encounter a failure in this area. Reliability: too often—especially at peak loads—the system crashes. permitting objective evaluation.D. Using quantitative data to manage i. i. Evaluation methods i. service promptness. Volume/capacity: at peak or sustained loads. Schedule risks: How might the project exceed the allotted time? iii. f. An understanding and application of the basic techniques needed to provide consistent reliable results. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 73 . for example. but too slowly. E. It’s important for test professionals to remember that many kinds of quality risks exist. the more testing that problem area deserves. Severity: How dangerous is a failure of the system in this area? ii. i. Methods for using quantitative data as a management tool. responsiveness. In any software development project. d. time zones. Data quality: a database becomes corrupted or accepts improper data. v. currencies. iv.) ii. Risk Analysis a. ii. Usability: the software’s interface is cumbersome or inexplicable.

TEST PLAN .Files. C.. (4) schedules task starts and completions. performance. schedule. Prototyping is a complement to other methodologies. Extra Information A. a hypothetical user environment intentionally constructed to be sufficiently diverse and complex to support execution of all relevant test cases. I. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 74 . response time). PROTOTYPE . a..The individual or group who will use the system for its intended operational use when it is deployed in its environment. analysts. Examples of technical requirements include functional. G. development. evaluation criteria. TEST CASE . E. you cannot substitute any prototype for a paper specification. J.Any requirements related to software. A test architecture includes: (1) a structural blueprint. design specifications. or maintenance work (e.A tool for directing the software testing which contains the orderly schedule of events and list of materials necessary to effect a comprehensive test of a complete application. CRITICAL PATH . A formal or informal plan for carrying out a particular test that: (1) defines tasks to be performed. and the cost of any new computer equipment and software.9. It provides the framework within which detailed test and evaluation plans are generated. D.An active model for end-users to see touch feel and experience. However. It details the test strategy. Those requirements that describe what the software must do and its operational constraints. the truth of which must be demonstrated through testing in order to conclude that the entity meets established user/design requirements. i. derived from the Operational Requirements Document. and (3) a definition of the overall processing sequence for the test. There is no standard format for test data. TEST and EVALUATION PLAN . key performance parameters and operational performance parameters (threshold and objective criteria). and data elements created by users. and resource requirements for test and evaluation. test management strategy and structure. supply. H. (2) a definition of the test time dimension (the time span covered by the test and the division of that time span into discrete periods. It is the working equivalent to a paper design specification with one exception . and quality requirements F.g. TEST ARCHITECTURE . computer usage. Samples of live data may be used for test data if they are analyzed. In addition. costs associated with the installation and start-up of the new system must be calculated. to determine completeness in terms of all conditions which can occur. and (5) links.The high-level design of a planned application software test. Those parts of the document directed toward the user staff personnel should be presented in noncomputer-oriented language.Development costs include personnel costs. records.A series of dependent tasks for a project that must be completed as planned to keep the entire project on schedule. DEVELOPMENT COSTS .e. (3) defines resources required to accomplish each task. B. and developers to test requirements. training. TEST DATA .An assertion concerning the functioning of an application software entity.A system life cycle documentation standard that identifies high-level requirements and defines the objectives and overall structure of the test and evaluation for a system. and equipment costs. and those parts of the document directed toward other personnel should be presented in suitable terminology. (2) specifies sequential dependencies among the tasks. and critical technical parameters. and supplemented as necessary. END USER . TECHNICAL REQUIREMENTS . via an initial traceability matrix. test tasks to pertinent user/design requirement.errors can be detected earlier. interface. and software code. and required resources to critical operational issues. It relates program schedule. and major decisions.

plans procedures. So. depending upon the service provider. SEI-CMM or DoD. For exsiting project we will take only those phases which are complete. COMPLETE TESTING . i. required execution procedures. . and all the installation activities are over at the client's site. We have confirmed this with SEI and the same is indicated in SEI's web site as well. to demonstrate which type of testing is to be executed in parallel with the phases of the SDLC. V-Model is only an approach for testing.com”) How to prepare a Project Closure Report: (courtsey of Advait. in the report you can have columns to check whether all the reuqirements are met. Given a profile with ISO 9001:2000 certificate and Process focus in the organization.K. M.a set of activities. The notion is specific to a test selection criterian: i.e. It defines the test cases to be executed. testing is "complete" when the tests specified by the criterion have been passed. This is part of the traceability matrix. otherwsie to make the new MBR we should have atleast 4-8 datapoints it means that project for same nature or technolgy. TEST SCRIPT . Test requirements should generally exist at levels corresponding to the requirements. A project closure report can be prepared either by formal project closure report or it can be in the form of a checklist. organizations who have been assessed for CMM Level 5 are finding that they need about 15 months to acheive a proper CMMI Level 5 implementation. This is misinformation prevailing in the market.A system life cycle documentation standard that is the design specification for a test run. It is not correct that CMM is being phased out by end of 2003. A CMMI Level 4 initiative could run for much longer (say 18 months and above). you can fill the report and get it counter signed by the client.any artifact created as part of the software process includingcomputer programs. 3. costs for a CMMI assessments would be around 45000 USD while that for CMM would be around 28000 USD. Currently.  “advaitslele@indiatimes. required set up procedures. L. VERIFICATION . TEST REQUIREMENTS .2nd ed.. and whether software installed is running without hiccups. N. Software process . and transformations that people use to develop and maintain software work products [CMU/SEI 91]. by Boris Beizer..  Can anyone tell me the procedure to arrive at the metrics baselines report? I believe that it depends upon the organisation to organisation when they will update the MBR.erroneously used to mean 100% branch coverage. practices. IEEE. NIST. 5. You also have standards set by British for their own companies to follow.e. and required evaluation procedures. Postings from CQAfolks (Yahoo)  Testing Standards: Testing Standards mainly are those set by ISO.A description of the test which must be executed to verify a system/software requirement. CMM is definitely on till end of 2005. after a careful examination ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 75 . Software Work Product . (ISBN 0-442-20672-0) O. and associated documentation and data [CMU/SEI 91]. methods. 2. when you are in the installation phase. 6. 4. V-Model is not a standard. if not later than that. a CMM Level 4 initiative can reach you to a tangible milestone at an early date (say 9 to 12 months).Ankur Handa  CMM & CMMI certain facts: CMM or CMMI? how long will it take? can transition between from ISO to CMMI can happen? 1. P.* * .The process of evaluating software to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase (IEEE-STD-610).Quoted from the Glossary/Index of Software Testing Techniques . A CMM Level 4 initiative would produce tangible results for you and you can reach a milestone in a comparative lesser duration and can act as a strong springboard to launch your further initiatives. Absolutely complete testing is impossible. For revision we can take the existing & closed project in that defined duration let say in my orgainsation we do it after 3 months.

A prototype is a scled down version of the full system and serves for further clarification of the requirements. 3) Tools used : Tools used in the coding phase . Closure of Project report needs to address following topics 1) General Information : Project Description . The following is a quick list of these models: Linear Sequential Model Also known as waterfall model as well as SDLC = Software Development Life Cycle.of the system at client's site after installation (u can get the client to sit besides u when this check is being performed) you can fill up the report / checklist. Team size etc. then start all over The RAD Model ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 76 . checklist . Testing tools. Software Engineering Paradigms Also known Software Process Models . Listen to customer 2. Testing The Prototyping Model This model skips the rigorous steps involving requirements analysis and specifications since the customer is unsure about what is required and the developer would like to make sure the implementation is not going to be a big problem. Analysis 2. Coding 4. various models emphasizing the product or process views to one degree or another have been proposed and discussed in the literature. guidelines in any. Metrics like : Productivity achieved v/s Planned . After a cursory requirements gathering and a quick design. Project management tool . Have customer test-drive the new prototype. Design 3. Duration . Build/revise mock-up 3. They mainly involve the question as to what software development process is all about and how it can best be controlled. these are various strategies for successful solution of software engineering problems. this model involves welldefined steps or phases in the software development process. Tailoring the templates. CM tools etc 4) Risk Management : Risk assessment details . These steps are: 1. Quality of product (Include Acceptance Defects ) Customer Complaint received and resolution 2) Process Details : Methodology used. Contingency plan etc 5) Metrics for all the phases a) Effort Estimation b) Schedule Phase wise c) Test Defects d) Review defects e) Paretto analysis f) Cause and effect analysis g) Defect Leakage across phase h) Defect density I) Any other metrics as required for the customer 6) SWOT analysis of the project 7) Overall conclusion of the project . Its a cyclic process as shown below: 1. Deviations in the process. Technology used. the developer build a prototype. Through the years.

this model is based on keeping track of many phases of the project simultaneously. this is an evolutionary model which combines the iterative nature of prototyping with the systematic aspects of life cycle model. (ISO 9004: 1987. 3.This Rapid Application Development model is essentially an SDLC with extremely short life cycle. It is a highspeed adaptation of SDLC using component based construction approach. Since it emphasizes the evolutionary nature of the software development proces where customer and developer both understand and react better at each step. The activities associated with various phases are grouped together and defined as a state. using the ptotyping philosophy. Rather than defining as simple sequence of activities. Testing and turnover The Evolutionary Models These models are based on evolutionary nature of software development process. State transition diagrams are use d for keeping track of and controlling various phases. It relies heavily on software reuse which is a big issue in and of itself.1) Customer: Ultimate consumer. (ISO 9126: 1991. In later increments. limited versions are introduced to meet the competitive requirements. this model is not a panacea. and requires careful assesment and analysis of risks at each step. Software is developed in a series of incremental releases. This approach involves the following phases: 1. 3. When applied to client-server models. package or product. But.4) Defect: The nonfulfilment of intended usage requirements. client. The early iterations may be a paper or simple prototypes. (ISO 8402: 1986. Application generation 5. The Concurrent Development Model Also known as concurrent engineering. Business and product requirements change and the details and extensions of the core software needs to adapt to these changes. Data Modeling 3. (ISO 9126: 1991. as with the earlier models. This model is fairly new and has not yet been assessed for efficacy. 3. this model highly realistic. It is an attempt to produce the iterative versions of the product from prepackaged sotware components corresponding to object classes. Therefore. package or product for the purpose of determining acceptance or release of the software module. The Spiral Model originally proposed by Boehm. more sophisticated versions are devloped with new features. In essence. Business Modeling 2.for different components of a large system at the same time. user. The Incremental Model The first increment is normally referred to as core product and thereafter. this model defines activities in two dimensions: The system dimension and the component dimension. beneficiary or second party.2) ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 77 . In an iterative manner. this model is based on using appropriate models --as above-. The following list defines terms that could be relevant for evaluation purposes: Assessment: An action of applying specific documented assessment criteria to a specific software module. increasingly more complex versions of the software is developed. Process Modeling 4.21) Features: Features are identified properties of a software product which can be related to the quality characteristics. The Component Assembly Model This is essentially a spiral model with technical framework based on object technologies. 3. increasingly complete versions are engineered. this model defines a network of clusters of activities.

(ISO 9126: 1991. A. (ISO 8402: 1986. (ISO 8402: 1986.The basic difference between `nonconformity' and `defect' is that specified requirements may differ from the requirements for the intended use. These levels are called rating levels. 3.14) Level of performance: The degree to which the needs are satisfied.5) Nonconformity: The nonfulfilment of specified requirements.4) Liability (product/service): A generic term used to describe the onus on a producer or others to make restitution for loss related to personal injury. examining.1.3) Inspection: Activities such as measuring.2) Security: Attributes of software that bear on its ability to prevent unauthorized access. 3. 3.2. (ISO 8402: 1986. (ISO 8402: 1986. rules and any associated documentation pertaining to the operation of a data processing system.20) Quality: The totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs. 3.6) Quality control: The operational techniques and activities that are used to fulfill requirements for quality.6.4. testing. conditions. (ISO 8402: 1986. The computer program and data contained in firmware are classified as software. (ISO 9000-3: 1991. 3.19) Measurement: The action of applying a software quality metric to a specific software product. 3. 3.11) Rating: The action of mapping the measured value to the appropriate rating level.4) Resource behaviour: Attributes of software that bear on the amount of resources used and the duration of such use in performing its function. whether accidental or deliberate. The term `reliability' is also used as a reliability characteristic denoting a probability of success or a success ratio. 3. Used to determine the rating level associated with the software for a specific quality characteristic. 3. represented by a specific set of values for the quality characteristics. 3.7) Rating level: A range of values on a scale to allow software to be classified (rated) in accordance with the stated or implied needs. A. processes.2. 3. A. Managers or Developers. 3.1) Quality assurance: All those planned and systematic actions necessary to provide adequate confidence that a product or service will satisfy given requirements for quality. (ISO 9126: 1991. 3. (ISO 8402: 1986. Users.2. gauging one or more characteristics of a product or service and comparing these with specified requirements to determine conformity.3) Reliability: The ability of an item to perform a required function under stated conditions for a stated period of time.2.2. (ISO 9126: 1991. (ISO 9126: 1991. (ISO 9126: 1991. property damage or other harm caused by a product or service.18) Replaceability: Attributes of software that bear on the opportunity and effort of using it in the place of specified other software in the environment of that software. the circuitry containing the computer program and data is classified as hardware. procedures.8) Recoverability: Attributes of software that bear on the capability to re-establish its level of performance and recover the data directly affected in case of a failure and on the time and effort needed for it. 3. (ISO 9126: 1991. and analysis of records in relation to stated references to ensure that specified requirements for quality are being met. to programs and data. (ISO 8402: 1986. (ISO 8402: 1986.e. (ISO 9126: 1991. (ISO 9126: 1991.5) Software: Intellectual creation comprising the programs.20) NOTE -. products and services. (ISO 8402: 1986.7) Quality surveillance: The continuing monitoring and verification of the status of procedures. Appropriate rating levels may be associated with the different views of quality i. 3. methods. (ISO 9126: 1991. A.Firmware: Hardware that contains a computer program and data that cannot be changed in its user environment.1) ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 78 .

12) Software quality characteristics: A set of attributes of a software product by which its quality is described and evaluated. by a stated or implied set of users.22) Stability: Attributes of software that bear on the risk of unexpected effect of modifications.2. The quality is represented by the set of rated levels associated with the software product. (ISO 9126: 1991.6 ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 79 . A.1) Understandability: Attributes of software that bear on the users' effort for recognizing the logical concept and its applicability. (ISO 9126: 1991.3. (ISO 9126: 1991. (ISO 9126: 1991. 3.2.2.4. procedures and associated documentation and data designated for delivery to a user.2. A. (ISO 9126: 1991.3) Suitability: Attribute of software that bears on the presence and appropriateness of a set of functions for specified tasks.5. (ISO 9000-3: 1991. 3. (ISO 8402: 1986. and on the individual assessment of such use.5.2.14) Specification: The document that prescribes the requirements with which the product or service has to conform. 3.2) Software item: Any identifiable part of a software product at an intermediate step or at the final step of development. (ISO 9000-3: 1991.7) Verification (for software): The process of evaluating the products of a given phase to ensure correctness and consistency with respect to the products and standards provided as input to that phase.4) Time behaviour: Attributes of software that bear on response and processing times and on throughput rates in performing its function.1) Usability: A set of attributes that bear on the effort needed for use. 3. (ISO 9126: 1991. 3. A. (ISO 9126: 1991.11) Software quality assessment criteria: The set of defined and documented rules and conditions which are used to decide whether the total quality of a specific software product is acceptable or not. 4. A.Software product: Complete set of computer programs.1. (ISO 9126: 1991. 3. 3. A software quality characteristic may be refined into multiple levels of sub-characteristics. (ISO 9000-3: 1991. 3. 3. A.1) Testability: Attributes of software that bear on the effort needed for validating the modified software.13) Software quality metric: A quantitative scale and method which can be used to determine the value a feature takes for a specific software product.3) Software quality: The totality of features and characteristics of a software product that bear on its ability to satisfy stated or implied needs.3) Validation (for software): The process of evaluating software to ensure compliance with specified requirements. (ISO 9000-3: 1991. (ISO 9126: 1991. (ISO 9000-3: 1991.