This action might not be possible to undo. Are you sure you want to continue?
A. Before an organization can begin to assess the quality of its products and services and identify opportunities for improvement, it first must have a working knowledge of quality principles. This category will test the CSQA candidate’s understanding and ability to understand and apply these principles.
B. Definitions of Quality: 1. Quality
i. Totality of characteristics of an entity that bear on its ability to satisfy stated and implied needs. The term "quality" should not be used as a single term to express a degree of excellence in a comparative sense, nor should it be used in a quantitative sense for technical evaluations. To express these meanings, a qualifying adjective should be used. ii. QUALITY - The degree to which a system, component, or process meets specified requirements, or customer or user needs or expectations. iii. QUALITY FACTORS - The characteristics used to formulate measures of assessing information system quality. iv. The New 2000 ISO 9000 Standards - The four primary standards are as follows: ISO 9000: Quality management systems - Fundamentals and vocabulary ISO 9001: Quality management systems - Requirements ISO 9004: Quality management systems - Guidance for Performance Improvement ISO 19011: Guidelines on Quality and Environmental Auditing v. Quality software is reasonably bug-free, delivered on time and within budget, meets requirements and/or expectations, and is maintainable. However, quality is obviously a subjective term. It will depend on who the 'customer' is and their overall influence in the scheme of things. A wide-angle view of the 'customers' of a software development project might include end-users, customer acceptance testers, customer contract officers, customer management, the development organization's management/accountants/testers/salespeople, future software maintenance engineers, stockholders, magazine columnists, etc. Each type of 'customer' will have their own slant on 'quality' - the accounting department might define quality in terms of profits while an end-user might define quality as user-friendly and bug-free. vi. In Quality Is Free, Phil Crosby describes quality as "conformance to requirements.” vii. J.M. Juran’s definition of quality. He spends a goodly portion of an early chapter in Juran on Planning for Quality discussing the meaning of quality, but he also offers a pithy definition: fitness for use. In other words, quality exists in a product—a coffee maker, a car, or a software system—when that product is fit for the uses for which the customers buy it and to which the users set it. A product will be fit for use when it exhibits the predominant presence of customersatisfying behaviors and a relative absence of customer-dissatisfying behaviors.
2. Producer’s View of Quality
i. A more objective view ii. Conformance requirements iii. Costs of quality (prevention, appraisal, scrap & rework, warranty costs) Prevention costs: training, writing quality procedures Appraisal costs: inspecting and measuring product characteristics Scrap and Rework costs: internal costs of defective products Warranty costs: external costs for product failures in the field
3. Customer’s View of Quality
i. A more subjective view ii. Quality of the design (look, feel, function) iii. Consider both feature and performance measures to asses value
____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 1
Value = quality / price (determined by individual customers) International Standards Organization document ISO 9126. i. This standard proposes that the quality of a software system can be measured along six major characteristics: Functionality: Does the system provide the required capabilities? Reliability: Does the system work as needed when needed? Usability: Is the system intuitive, comprehensible, and handy to the users? Efficiency: Is the system sparing in its use of resources? Maintainability: Can operators, programmers, and customers upgrade the system as needed? Performance: Does the system fulfill the users’ requests speedily?
C. Quality Concepts: 1. Cost of Quality
i. ii. iii. iv. Prevention costs – maintaining a quality system Appraisal costs – maintaining a quality assurance system Internal failures – manufacturing losses, scrap, rework External failures – warranty, repair, customer, product service Jim Campenella illustrates a technique for analyzing the costs of quality in Principles of Quality Costs. Campenella breaks down those costs as follows:
Cquality=Cconformance+Cnonconformance vi. Conformance costs include prevention costs and appraisal costs. Prevention costs include money
spent on quality assurance—tasks like training, requirements and code reviews, and other activities that promote good software. Appraisal costs include money spent on planning test activities, developing test cases and data, and executing those test cases once. Nonconformance costs come in two flavors: internal failures and external failures. The costs of internal failure include all expenses that arise when test cases fail the first time they’re run, as they often do. A programmer incurs a cost of internal failure while debugging problems found during her own unit and component testing The costs of external failure are those incurred when, rather than a tester finding a bug, the customer does. These costs will be even higher than those associated with either kind of internal failure, programmer-found or tester-found. In these cases, not only does the same process described for tester-found bugs occur, but you also incur the technical support overhead and the more expensive process of releasing a fix to the field rather than to the test lab. In addition, consider the intangible costs: angry customers, damage to the company image, lost business, and maybe even lawsuits. The flip side of the quality approach is Philip Crosby's “quality if free" idea. Basically, Crosby's thesis is that bad quality is very expensive. If you add up the costs of scrap, rework, delays in scheduling, the need for extra inventories to compensate for schedule changes, field service costs, product warranty expenses, and most of all customer dissatisfaction with your product, that can cost one heck of a lot of money. Companies need to get their arms around these costs of quality and quantify their impact. Most companies have estimated that their cost of quality is 25% to 35% of product cost. With that as the incentive, a firm can start to go to work and attack the root causes that result in those bad quality costs, reduce them, and end up at the same place as the Deming approach which is to produce high quality goods and eliminate screw-ups in the manufacturing process. They both have the same objective, to get the quality up throughout the whole process instead of waiting and inspecting the product at the end.
i. A Problem Solving Process ii. The well known Deming cycle instructs us to Plan, Do, Study and then Act upon our findings, in order to obtain continuous improvement. For a software organisation this might be rephrased as "Plan the project, Develop the system, Scrutinise its implementation and Amend the process".
____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 2
iii. The continuous improvement uses a process that follows the plan-do-check-act cycle. The situation is analyzed and the improvement is planned (Plan). The improvement is tried (Do). Then data is gathered to see how the new approach works (Check or study) and then the improvement is either implemented or a decision is made to try something else (Act). This process of continuous improvement makes it possible to reduce variations and lower defects to near zero.
Plan – The Change
1) Step 1: Identify the Problem
i. ii. iii. iv. 2) Step 2: i. ii. iii. iv. v. vi. vii. viii. ix. Select the problem to be analyzed Clearly define the problem and establish a precise problem statement Set a measurable goal for the problem solving effort Establish a process for coordinating with and gaining approval of leadership Analyze the Problem Identify the processes that impact the problem and select one List the steps in the process as it currently exists Map the Process Validate the map of the process Identify potential cause of the problem Collect and analyze data related to the problem Verify or revise the original problem statement Identify root causes of the problem Collect additional data if needed to verify root causes
Do – Implement The Change
1) Step 3: Develop Solutions i. Establish criteria for selecting a solution ii. Generate potential solutions that will address the root causes of the problem iii. Select a solution iv. Gain approval and supporters of the chosen solution v. Plan the solution 2) Step 4: Implement a Solution i. Implement the chosen solution on a trial or pilot basis ii. If the Problem Solving Process is being used in conjunction with the Continuous Improvement Process, return to Step 6 of the Continuous Improvement Process iii. If the Problem Solving Process is being used as a standalone, continue to Step 5
Check – Monitor and Review The Change
1) Step 5: Evaluate The Results i. Gather data on the solution ii. Analyze the data on the solution
Act - Revise and plan how to use the learning’s
1) Step 6: i. ii. iii. iv. v. Standardize The Solution (and Capitalize on New Opportunities) Identify systemic changes and training needs for full implementation Adopt the solution Plan ongoing monitoring of the solution Continue to look for incremental improvements to refine the solution Look for another improvement opportunity
____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 3
A Six Sigma opportunity is then the total quantity of chances for a defect. The central idea behind Six Sigma is that if you can measure how many "defects" you have in a process. ii. As the process sigma value increases from zero to six. The end result is greater customer satisfaction and lower costs. data-driven approach and methodology for eliminating defects (driving towards six standard deviations between the mean and the nearest specification limit) in any process -. iii. A Six Sigma defect is defined as anything outside of customer specifications. a process must not produce more than 3. Six Sigma is a highly disciplined process that helps us focus on developing and delivering nearperfect products and services. Six Sigma i. The objective of Six Sigma Quality is to reduce process output variation so that ±six standard deviations lie between the mean and the nearest specification limit. To achieve Six Sigma.’ Decrease your process variation (remember variance is the square of your process standard deviation) in order to increase your process sigma.4 defect Parts Per Million (PPM) opportunities. iv.from manufacturing to transactional and from product to service. Process sigma can easily be calculated using a Six Sigma calculator. The statistical representation of Six Sigma describes quantitatively how a process is performing. the variation of the process around the mean value decreases. Why "Sigma"? The word is a statistical term that measures how far a given process deviates from perfection.Plan-Do-Check-Act Cycle 3. What’s Involved In a Six Sigma Initiative? ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 4 . to be produced. This will allow no more than 3. also known as Defects Per Million Opportunities (DPMO). v. With a high enough value of process sigma.4 defects per million opportunities. you can systematically figure out how to eliminate them and get as close to "zero defects" as possible. the process approaches zero variation and is known as 'zero defects. Six Sigma is a disciplined.
document. Define the customers. knowledge and organization. They typically also manage the individuals who are selected for Black Belt training.000 percent improvement! No one person and no one area can accomplish this alone. one must note that we cannot achieve a Six Sigma level in one go. whereas 6 sigma brings defect levels to 3. 1) Define. Sigma is a metric that indicates how well the process is performing. Pg 34. Generate and determine potential solutions and plot them on a small scale to determine if they positively improve process performance. Considering the enormous reduction in defect level and variations. They act as coaches and mentors for Black Belts. the philosophy is called the Six Sigma philosophy. Master Black Belts: These are the full-time trainers for a company’s Six Sigma efforts. As sigma increases. customer satisfaction goes up while at the same time cycle time goes down and costs plummet. and the key process that affects that customer. Training these individuals to become Black Belts and “change agents” is critical to successful implementation of Six Sigma Problem Solving Technology. This process is to be repeated until we reach a defect level of 3. George Eckes. Every time we improve the process. 3) Analyze. and maintaining improvement. it takes more than technology. their requirements. However. These tools apply a refined methodology of measurement and discovery to gain a comprehensive understanding of performance and key variables affecting the quality of a company’s products and services. Going from Four to Six Sigma is almost a 2. Analyze the data collected as well as the process to determine the root causes for why the process is not performing as desired. The Six Sigma Revolution. determining their root causes. the sigma level goes up. The challenge to leadership is to harness the ideas and energy of many people across functions. and implement a plan to ensure that performance improvement remains at the desired level. 2) Measure. formulating ideas around what would result in improvement. Supervisory-Level Management: These managers play a pivotal role because they own the processes of the business and must ensure that improvements to the process are captured and sustained. A level of Six Sigma represents the peak of quality — the virtual elimination of defects from every product and process within an organization. marketing. quality. Develop. and must understand the challenges facing them as well as be willing and empowered to remove any roadblocks to progress. and execute the plan for data collection. there are six distinctive roles in the implementation process. The essence of this method centers on identifying problems. sites. Listed below is a high level overview of the Six Sigma improvement methodology that various companies have used to practice its process improvement initiative. Implementing Six Sigma requires mobilizing people resources and arming them with the tools they need to accomplish the goal of quality improvement and impressive financial results. Achieving Six Sigma performance across an organization is an enormous challenge.4 only per one million opportunities. To become a Six Sigma company. From within this group. and even business groups. 4) Improve. sales. This quantum leap in quality needs people to make it happen. service. Six Sigma is the disciplined application of statistical problem-solving tools that show you where wasteful costs are and points you toward precise steps to take for improvement. a senior management leader or leaders is assigned to provide day-to-day top management leadership during implementation. These individuals are referred to as Champions. A three Sigma indicates a level of 66. 5) Control. the team charter. Six Sigma is a philosophy to eliminate variation in process. the Data Collection Plan for the process in question. executive managers provide overall leadership and must own and drive Six Sigma.807 defects per one million opportunities.4 or lower for every million opportunities. While all employees need to understand the vision of Six Sigma and use some of its tools to improve their work. They will grow them from the ranks ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 5 . It can be applied to all disciplines: production. Identify the key measures. Six Sigma Champions: As a group. testing those improvements.
thus completing more projects in a given period of time. leading and facilitating the completion of Six Sigma projects. it allows the Black Belt to work on overlapping projects. the best Black Belts become the Master Black Belts. such as fitness for use. sharing. Continuous improvement. “the teams”. In summary. These eight principles are: Customer focused organization Leadership Involvement of people Process approach System approach to management Continual improvement Factual approach to decision making Mutually beneficial supplier relationship D. of the Black Belts with the help of the Six Sigma partner’s team of experts.3. and using knowledge and best practices. management should define objectives pertaining to key elements of quality. ii. performance. Reduce development costs/improve time-to-market capability iii. Improve Customer Satisfaction ii. Project Team Members: These are the project. BENCHMARK . 3) A measurement system. in regard to organizational quality and performance.specific. (Source ISO 9004: 1987. Best Practices i. Green Belts also work on smaller projects inside their functional areas. that provide process and cross-functional knowledge. and financial advantage. ii. 4.1. Benchmarking i. Six Sigma is: 1) A measure of variation that achieves 3. ii. Quality Objectives: 1.A standard against which measurements or comparisons can be made. The revisions of ISO 9001 and 9004 are based on eight quality management principles that reflect best management practices. safety and reliability.) i. Quality Attributes: 1. they help Black Belts complete projects and extend the reach of Black Belts. (ISO 9001) A complete cycle to improve the effectiveness of the quality management system. part-time people resources. It focuses on how to improve any given business process by exploiting top-notch approaches rather than merely measuring the best performance. 2) A cultural value or philosophy toward your works. Reliability ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 6 . For the corporate quality policy. The process of identifying. (SWCMM (IEEE-STD-610) 5. 4) A goal 4. and eventually take 100 percent ownership of a Black Belt project. operational. Improve Processes E. They help sustain the gains achieved by Six Sigma projects. 6. focuses on improving customer satisfaction through continuous and incremental improvements to processes. To sustain a program.4 defects per million opportunities. Green Belts: As part-time resources. Finding. Continuous Improvement i. When a Black Belt has access to the time and expertise of Green Belts. including by removing unnecessary activities and variations. studying and implementing best practices provides the greatest opportunity for gaining a strategic. Black Belts: Full-time employees who are 100 percent focused on identifying.
Proper documentation of standards and procedures is necessary since the SQA activities of process monitoring. ii. On the other hand. 6. Effort required to couple or interface one system with another. System versatility (or system robustness) is the capability of the system to allow flexible procedures to deal with exceptions in processes and procedures. and the system can produce the desired result correctly. while correctness would score high. ii. ii. Correctness i. RELIABILITY . They define legal language structures. then the correctness quality factor would be rated perfect. and application support personnel perspectives. control. 3. product evaluation. Effort required to learn. Standards are the established criteria to which the software products are compared. reliability may score low. such as using wrong program versions. Maintainability i. Flexibility i. and product documentation and provide consistency throughout a project. if an input transaction is entered perfectly. Effort and response time required to enhance an operational system or program. Forms are misinterpreted. etc. The NASA Data Item Descriptions (DIDs) are documentation standard.Automated applications are not run in a sterile environment. and misinterpreted by the people that receive them. mislaid. ii. and test the system or project enhancement from user. The ability of software and hardware on different machines from different vendors to share data. and internal code documentation ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 7 . that same system which processed using imperfect input may fail to produce correct results. Documentation Standards specify form and content for planning. style conventions. System adaptability is the capability to modify the system to cope with major changes in business processes with little or no interruption to business operations. Extent to which a system or release can be expected to perform its intended function with required precision and without interruption to execution and delivered functionality. 5. Thus. People operate the system and make mistakes. production control. Interoperability i. misdirected. The reliability factor measures the consistency with which the system can produce correct results.i. Code Standards specify the language in which the code is to be written and define any restrictions on use of language features. and auditing rely upon unequivocal definitions to measure project compliance. Design Standards specify the form and content of the design product. operate. 4. They provide rules and methods for translating the software requirements into the software design and for representing it in the design documentation. rules for data structures and interfaces. For example. Procedures are the established criteria to which the development and control processes are compared. the SQA role is to ensure their existence and adequacy. Effort required to implement new enhancements or fix operational errors. Standards and procedures establish the prescribed methods for developing software. Conformance to accepted software standards to include additional enhancements to the standards. The characteristics of software that allow or enable adjustments or other changes to the business process. Outputs may be lost. 2. not including all of the input or adding input which should not be included. Busy people prepare input and make mistakes in input preparation. and users of systems experiment with input transactions. instructions are unknown. since these provide the framework from which the software evolves. Standardization i. All of these affect the correctness of the application results. maintain. Effort required to implement zero-defect functionality. Establishing standards and procedures for software development is critical.
HELP. and so on. Quality Metric Criteria: Criteria Accuracy Clarity Quality Measurement Those attributes of the software products that provide the required precision in calculations and output products and fully meet the functional. Usability i. However. Testability i. and consistency. Usability is a combination of factors that affect the user's experience with the product or system. like the other quality factors. attention must focus on clarity. should be discussed and negotiated with the user responsible for the application. ii. and tutorial explanations requires the ability to write clear English and avoid jargon. but to actually take full advantage of it. An application is portable across a class of environments to the degree that the effort required to transport and adapt it to a new environment in the class is less than the effort of redevelopment. Portability i. an extension of hypertext. 12. Pictures may also be extremely helpful. Usability: To create good user interfaces. They should be used where appropriate to make applications easy to learn and use. The wording should be considered carefully. (2) The degree to which a requirement is stated in terms that permit establishment of test criteria and performance of tests to determine whether those criteria have been met (IEEE-STD-610). Those attributes of the software that provide for useful inputs and outputs which are readily assimilated. and operational requirements. 8. pull-down menus. mobile technology. Hyper documents. the amount of resources allocated to testing can vary based on the degree of reliability that the user demands from the project. Performance i. there should be consistent use of screen layouts. are also an appropriate form of on-line documentation. a software application. performance. Scalability i. Usability is the measure of the quality of a user's experience when interacting with a product or system — whether a Web site. COMPREHENSION: Developing on-line documentation. (1) The degree to which a system or component facilitates the establishment of test criteria and the performance of tests to determine whether those criteria have been met. Testing. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 8 . Security 14. comprehension. ii. or any user-operated device. 9. ii. CLARITY: The screen layout needs to be clear and uncluttered. 10. It is the ability of a computer application or product (hardware or software) to continue to function well as it (or its context) is changed in size or volume in order to meet a user need. It is the ability not only to function well in the rescaled situation. That attribute of a computer system that characterizes the timeliness of the service delivered by the system. 11. The choice of words and colors should be consistent.7. The resources that need to be utilized to test the system to ensure the specified quality has been achieved. Effort required to prepare for and test a system or program to assure it performs its intended functionality without degraded operational performance. multiple-choice mechanisms. function keys. CONSISTENCY: A new application should look and feel as familiar as possible to users. Availability 13.
Quality Assurance All those planned and systematic activities implemented to provide adequate confidence that an software package will fulfill requirements for quality 1) Data Gathering Problem Trend Analysis Process Identification Process Analysis Process Improvement 2) Focus on Process. changing. Those attributes of the software products that provide continuity of operation under non-nominal conditions. Pioneer of Modern Quality Control. Reactive. Quality Assurance vs. etc. Those attributes of the software products that provide for simplicity and completeness in presentation and for implementation of a function with a minimum amount of code. Software objects that protect themselves and their associated data. Walter A. Find Defects ii. Quality Control: 1. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 9 . also those attributes of the software products that can be delivered before the event horizon. Staff Function. Consistency is measured for report and screen formats. Shewart i. Self-descriptiveness Those attributes of the software that provide explanation of the maintenance and implementation of a function. Communicativeness Those attributes of the software products that provide useful inputs and outputs that can be assimilated. Expandability Modularity Those attributes of the software products that provide for increasing. Conciseness Consistency Data commonality Encapsulation Error Tolerance Execution efficiency Those attributes of the software that provide for minimum execution processing time without decrease in functionality. For Year 2000 projects. lending itself to simplicity and modularity. Proactive.A planned and systematic pattern of all actions necessary to provide adequate confidence that a software work product conforms to established technical requirements G. Those attributes of the software products that are delivered on time or run on schedule. and customizing functionality. Simplicity Timeliness Those attributes of the software products that provide maintenance and implementation of the functions in the most understandable manner. programming techniques. Those attributes of the software that provide the use of standard data representations and structures. Prevent Defects iii. Line Function. Quality Control The operational techniques and activities that are used to fulfill requirements for quality 1) Problem Identification Problem Analysis Problem Correction & Feedback to QA 2) Focus on Product. Those attributes of the software products that provide uniform design and implementation techniques.Communications Those attributes of the software that provide the use of commonality standard protocols and interface routines. ii. SOFTWARE QUALITY ASSURANCE . ease of maintenance. lending itself to simplicity. Recognized the need to separate variation into assignable and unassignable causes. eliminating random effects on other objects in the same system when encountering error conditions. ISO 9000 Definitions: i. and future expansion. JCL coding. Those attributes of the software products that provide a structure of highly independent modules with each serving a particular function and accordingly. F. Quality Pioneers: 1.
vi. 4) Measurement of quality is the cost of nonconformance. not close enough. Directed most of his work at executives and the field of quality management. Enlightened the world on the concept of the “vital few. trivial many” which is the foundation of Pareto charts. 7) Break down barriers between departments. Stressed a systems approach to quality (all organizations must be focused on quality) ii. Originator of the plan-do-check-act cycle. Studied under Shewart at Bell Laboratories ii. 10) Institute modern methods of supervision. 2. Philip Crosby i. 14) Put everybody in the company to work to accomplish the transformation.) 6. 3) Cease dependence on mass inspection to achieve quality. 11) Institute modern methods of training. Main contribution is his Fourteen Points to Quality. 9) Eliminate work standards (quotas). warranty. 8) Eliminate numerical goals. Edwards Deming i. We are in a new economic age. v. 13) End the practice of awarding business on price tag. engineering. W. Perhaps the first to successfully integrate statistics. objective quality: quality of a thing independent and subjective quality. 3) Performance standards of zero defects. and quality improvement. and failures (scrap. quality control. Developed concept of true and substitute quality characteristics True characteristics are the customer’s view Substitute characteristics are the producer’s view Degree of match between true and substitute ultimately determines customer satisfaction ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 10 . Joseph Juran i. 5. 6) Drive out fear. iv. Quality management Four absolutes of quality including: 1) Quality is defined by conformance to requirements. appraisal. Kaoru Ishikawa i. 4) Constantly and forever improve the system. Costs of quality may be separated into costs for prevention. subjective quality: quality relative to how people perceive it.iii. 2) System for causing quality is prevention not appraisal. Contributions: Well known for helping Japanese companies apply Shewart’s statistical process control. 2) Adopt the new philosophy. Arman Feigenbaum i. and economics. 12) Institute a program of education and retraining. 4. 3. 5) Remove barriers. Contributions: Also well known for helping improve Japanese quality. Founder of the control chart. Defined quality in terms of objective and subjective quality. etc. Developed the “Juran Trilogy” for managing quality: 1) Quality planning. The 14 points are: 1) Create constancy of purpose toward improvement of product and service.
and aiming at long-term success through customer satisfaction and benefits to all members of the organization and to society. and responsibilities. ii. Total quality management is the management approach of an organization. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 11 . Quality loss function (deviation from target is a loss to society) iii. quality control. quality assurance. Genichi Taguchi i. and maintenance cycles while changing organizational culture. Quality planning covers product planning. All activities of the overall management function that determine the quality policy. Quality planning embodies the concepts of defect prevention and continuous improvement as contrasted with defect detection. centered on quality. It maintains that organizations must strive to continuously improve these processes by incorporating the knowledge and experiences of workers. and implement them by means such as quality planning. iii. Quality Improvement Cycle i. Advanced the use of quality circles (worker quality teams) iv. focus is on “engineering design” robust design/parameter design H. and breakthroughs with development. Quality Management i. objectives. improvement. based on the participation of all of its members. customer service …) to focus on meeting customer needs and organizational objectives. ii. Developed the concept of Japanese Total Quality Control Quality first – not short term profits. Next process is your customer. Quality Vocabulary: 1. Total Quality Management is a structured system for satisfying internal and external customers and suppliers by integrating the business environment. 1960s – 1980s ii. ii. production. ii. 2. aimed at continually improving performance over the long term by focusing on customers while addressing the needs of all stakeholders. A quality improvement cycle is a planned sequence of systematic and documented activities aimed at improving a process. A comprehensive and fundamental rule or belief. Promoted the use of parameter design (application of Design of experiments) or robust engineering Goal: develop products and processes that perform on target with smallest variation that are insensitive to environmental conditions. continuous improvement. Use facts and data to make presentations.ii. TQM (Total Quality Management) i. Quality Planning i. Respect for humanity as a management philosophy – full participation 7. Improvements can be effected in two ways: By improving the process itself By improving the outcomes of the process. for leading and operating an organization. 4. managerial and operational planning. and quality improvement within the quality system. and the preparation of quality plans. It views organizations as a collection of processes. finance. A management philosophy which seeks to integrate all organizational functions (marketing. 3. Advocate of the use of the 7 tools iii. design. engineering. The activities that establish the objectives and requirements for quality and for the application of quality system elements.
iii. Quality Assurance i. Many companies have a separate department devoted to quality assurance.. Software Development. Once this guideline is in place and the quality assurance procedures are implemented. and increased market share. Today's quality assurance systems emphasize catching defects before they get into the final product. 6. a company's management team decides quality assurance policies and objectives. and to enable a company to better compete with others. reduction in the total product development cycle. Roughly. Advanced (Product) Quality Planning (AQP / APQP) is a structured process for defining key characteristics important for compliance with regulatory requirements and achieving customer satisfaction. 2. if this particular stuff is broken then whatever you're testing fails. the company is certified as in conformance with the standard. It is oriented to 'prevention'. Quality Control i. increased customer satisfaction. 5. A system of management which assures that planning is carried out such that ALL staff know what is expected and how to achieve the specified results. Conformance to ISO 9000 is said to guarantee that a company delivers quality products and services. AQP includes the methods and controls (i.e. processes. an outside assessor examines the company's quality assurance system to make sure it complies with ISO 9000.a planning tool for incorporating customer quality requirements through all phases of the product development cycle. Quality control is a formal (as in structured) use of testing. The concept of quality control in manufacturing was first advanced by Walter Shewhart. ii. Quality Function Deployment (QFD) . Operational techniques and activities that are used to fulfill requirements for quality. procedures. the company or an external consultant formally writes down the company's policies and requirements and how the staff can implement the quality assurance system. and resources needed to implement quality management. tests) that will be used in the design and production of a specific product or family of products (i. in effect. Quality control describes the directed use of testing to measure the achievement of a specified standard. iii. quality assurance is any systematic process of checking to see whether a product or service being developed is meeting specified requirements. Once the problems are corrected. iii.e. Quality System i. The organizational structure.monitoring and improving the process. It involves techniques that monitor a process and eliminate causes of unsatisfactory performance at all stages of the quality loop. 7. to improve work processes and efficiency. and with quality control you set limits that say. although it often used synonymously with testing. iii. you test to see if something is broken. Quality assurance was initially introduced in World War II when munitions were inspected and tested for defects after they were made. and the company agrees to correct any problems within a specific time. In developing products and services. Quality control is a superset of testing. A detailed report describes the parts of the standard the company missed. To follow ISO 9000. Next. and ensuring that problems are found and dealt with. ii. ISO 9000 is an international standard that many companies use to ensure that their quality assurance system is in place and effective. Key benefits to this approach are product improvement. The planned and systematic activities implemented within the quality system and demonstrated as needed to provide adequate confidence that an entity will fulfill requirements for quality. materials). A quality assurance system is said to increase customer confidence and a company's credibility. ii. measurements. making sure that any agreed-upon standards and procedures are followed. Software QA involves the entire software development PROCESS . Acquisition and Operation Processes ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 12 .. parts.
migration. System design. or an activity may be recursed to offset any implied or default Waterfall sequence. The operation of the software is integrated into the operation of the total system. Software requirements analysis. Software integration. Maintenance Process (ISO 12207) 1. The CQA candidate must understand how quality software is built to be effective in assuring and controlling quality throughout the software life cycle 2. and User support. total system. as needed. by the acquisition.1. iv. or the need for an improvement or adaptation. System operation. the development process is invoked to effect and complete the modifications properly. Modification implementation. The development process consists of the following activities along with their specific tasks: Process implementation. or a combination of these) for a project or an organization. the Waterfall. Development Process (ISO 12207) 1. The term development denotes both development of new software and modification to an existing software. The process covers the operation of the software and operational support to users. a deficiency. Understanding the processes used in the organization to develop. This process consists of the following activities along with their specific tasks: Process implementation. This process is activated when a system undergoes modifications to code and associated documentation due to an error. System qualification testing. Operational testing. The positional sequence of these activities does not necessarily imply a time order. A supporting process supports any other process as an integral part with a distinct purpose and contributes to the success and quality of the project. a problem. Software detailed design. vi. These activities may be iterated and overlapped. System requirements analysis. The objective is to modify an existing system while preserving its integrity. and Software retirement. This life cycle process contains the activities and tasks of the operator of a software system. the Spiral. v. Supporting Processes (ISO 12207) 1. All the tasks in an activity need not be completed in the first or any given iteration. Software coding and testing. incremental. ISO 12207 – Software Life-Cycle standard iii. These activities and tasks may be used to construct one or more developmental models (such as. Software architectural design. Software qualification testing. The maintenance process contains the activities and tasks of the maintainer. System integration. Operation Process (ISO 12207) 1. Problem and modification analysis. Software installation. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 13 . ii. but these tasks should have been completed as the final iteration comes to an end. and Software acceptance support. or other. The process ends with the retirement of the system. This process consists of the following activities along with their specific tasks: Process implementation. Maintenance review/acceptance. `This life cycle process contains the activities and tasks of the developer of software. The development process is intended to be employed in at least two ways: (1) As a methodology for developing prototypes or for studying the requirements and design of a product or (2) As a process to produce products. A supporting process is invoked. Process Knowledge a. supply. Operation and Maintenance Processes i. evolutionary. operate and maintain software systems. This process provides for developing software as a stand-alone entity or as an integral part of a larger. Software Development. Whenever a software product needs modifications.
This process consists of: Process implementation. Configuration identification. Configuration status accounting. The four activities along with their tasks are: Process implementation. They may as well be the acquirer and the supplier respectively. The reviews are at both management and technical levels. handling and delivery of the items. Problem Resolution Process . Process assurance.This process provides the framework for interactions between the reviewer and the reviewee. operation or maintenance process. develop. and Assurance of quality systems. and Maintenance. but supplements them. integration. design. as-built system fulfills its specific intended use. code. engineers and users of the system. Design and development. on the contrary. Joint Review Process .This process provides the framework for independently and objectively assuring (the acquirer or the customer) of compliance of products or services with their contractual requirements and adherence to their established plans. define. Verification does not alleviate the evaluations assigned to a process. To be unbiased. the auditor assesses the auditee's products and activities with emphasis on compliance to requirements and plans. Product assurance. or another supporting process. distribute and maintain those documents needed by all concerned such as managers.This is a process for recording information produced by a life cycle process. This process consists of: Process implementation. Quality Assurance Process . Verification determines whether the requirements for a system are complete and correct and that the outputs of an activity fulfill the requirements or conditions imposed on them in the previous activities. Verification Process .Validation determines whether the final. software quality assurance is provided with the organizational freedom from persons directly responsible for developing the products or providing the services. it supplements them.development. edit. and Release management and delivery. the process requires identification and analysis of causes ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 14 . requirements. At a joint review. Configuration evaluation. Configuration Management Process . The process defines the activities. At an audit. which plan. The extent of validation depends upon the project's criticality. In addition.This process provides the mechanism for instituting a closed-loop process for resolving problems and taking corrective actions to remove problems as they are detected. to record and report the status of the items and modification requests. and to control storage. and documentation. contractually established audits of a supplier's products or services. to control modifications and releases of the items. to ensure the completeness and correctness of the items. Documentation Process . Validation Process .This process is employed to identify. and baseline software items in a system. design. The process covers verification of process.This process provides the framework for formal.This process provides the evaluations related to verification of a product or service of a given activity. Configuration control. the reviewee presents the status and products of a life cycle activity of a project to the reviewer for comment (or approval). Audit Process . Validation does not replace other evaluations. Production. An audit may well be conducted by the acquirer on the supplier.
Infrastructure Process . Tools i. Organizational Processes (ISO 12207) 1. or a supporting process) needs to assess. or a task. viii.This process defines the activities needed for establishing and maintaining an underlying infrastructure for a life cycle process. supply process. maintenance. The process requires that a training plan be developed. Process assessment. A process. This standard contains a set of four organizational processes. Improvement Process . Training Process . or supporting process. they are sufficiently different at the detailed level because of their different goals. Establishment of the infrastructure. An organizational process may support any other process as well. and associated tools. The term "problem" includes non-conformance. maintenance process. Management Process . and facilities. For example. that is not contained in the standard but is pertinent to a project. b. Review and evaluation.The standard provides the basic. analysis. and Closure. however. techniques. training material be generated. software. Tailoring in the standard is deletion of non-applicable or in-effective processes. acquisition. objectives. such as the acquisition process.This process defines the generic activities and tasks of the manager of a software life cycle process. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 15 . and improve its life cycle process. The standard requires that all the parties that will be affected by the application of the standard be included in the tailoring decisions. tools. corporate level. controlling. These processes help in establishing. Experiences from application of the life cycle processes on projects are used to improve the processes. may be included in the agreement or contract. operation process. The infrastructure may include hardware. and improving other processes. have similar management activities. Planning. standards. It should be noted that this process itself. supply. The activities cover: Process establishment. typically beyond or across projects. the primary processes. activities. Application of tools and methods that aid in planning. development operation. and methods of operations. Even though.and reversal of trends in the reported problems. top-level activities that an organization (that is. and maintenance for increasing productivity. The organization establishes these activities at the organizational level. This process has the following activities: process implementation. and tasks. configuration management. measure. an activity. in general. control. The objectives are to improve the processes organization-wide for the benefit of the organization as a whole and the current and future projects and for advancing software technologies. and Process improvement. and Maintenance of the infrastructure. Tailoring Process (ISO 12207) 1. cannot be tailored. vii. operation. Therefore.This process may be used for identifying and making timely provision for acquiring or developing personnel resources and skills at the management and technical levels. estimating. development. Execution and control. An organization employs an organizational process to perform functions at the organizational. and training be provided to the personnel in a timely manner. The activities cover: Initiation and scope definition. each primary is an instantiation (a specific implementation) of the management process.
configuration control. software units. and related data in their ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 16 . Configuration management provides assurance that additions. and software/firmware to review and approval by an authorized authority. the design and requirements of the changed version of the system should be identified. control. as well as the trust placed in a trusted system. and auditing." Procedures should be established and documented by a configuration management plan to ensure that configuration management is performed in a specified manner. standards. Software configuration identification is consistent and accurate with respect to the numbering or naming of computer programs. Software configuration authentication is established by a series of configuration reviews and audits that exhibit the performance required by the software requirements specification and the configuration of the software is accurately reflected in the software design documents.Configuration management consists of four separate tasks: identification. configuration identification. For every change that is made to an automated data processing (ADP) system. SQA also monitors and audits the software library. The assurance control objective as it relates to configuration management of trusted systems is to "guarantee that the trusted portion of the system works only as intended. or in the case of trusted systems.ii. status accounting. consistent with the security policy of the system. Any deviation from the configuration management plan could contribute to the failure of the configuration management of a system entirely. SQA assures that: Baselines are established and consistently maintained for use in subsequent baseline development and control. The control task of configuration management is performed by subjecting every change to documentation. and delivery is compatible with the associated documentation. 2. hardware. and configuration authentication. and procedures. media. Finally. documentation. Software Quality Assurance (SQA) assures that software Configuration Management (CM) activities are performed in accordance with the CM plans. Configuration management is a sound engineering practice that provides assurance that the system in operation is the system that is supposed to be in use. proposed changes to the configuration identification. Configuration status accounting is responsible for recording and reporting on the configuration of the product throughout the change. Software development libraries provide for proper handling of software code. Configuration control is maintained such that the software configuration used in critical phases of testing. through the process of a configuration audit. Configuration status accounting is performed accurately including the recording and reporting of data reflecting the software's configuration identification. deletions. The CM activities monitored and audited by SQA include baseline control. 1. software modules. The purpose of configuration management is to ensure that these changes take place in an identifiable and controlled environment and that they do not adversely affect any properties of the system. or changes made to the TCB do not compromise the trust of the originally evaluated system. do not adversely affect the implementation of the security policy of the Trusted Computing Base (TCB). SQA reviews the CM plans for compliance with software CM policies and requirements and provides follow-up for nonconformances. and associated software documents. the completed change can be verified to be functionally correct. SQA audits the CM functions for adherence to standards and procedures and prepares reports of its findings. 3. configuration status accounting. CONFIGURATION MANAGEMENT (CM) . and for trusted systems. acceptance. It accomplishes this by providing procedures to ensure that the TCB and all documentation are updated properly. and the implementation status of approved changes.
Long before software process improvement and the CMM were common vocabulary in the software world. algorithms derived from theoretical research. In the early 70’s two concurrent research efforts resulted in two parametric software cost-estimating models available to the software development community (COCOMO and PRICE S). or some other sizing metric) 2. there was wide spread recognition that software project managers needed better ways to estimate the costs and schedules of software development projects. The anticipated amount of reuse 3. At a minimum.) 4. By automating many of the necessary. or process changes. A quantification of the organization’s software development productivity Although cost and schedule estimates are the main deliverable of the softwareestimating tool. freeing your team from tedious tasks that inhibit productivity. The type of software being developed (real time. data disasters. 2. Rational's SCM solution helps you manage complex change throughout the development lifecycle. you'll be able to select the right product today and seamlessly grow with the product tomorrow – no conversion headaches. a robust software artifact management tool. or some combination of these methodologies. combined with Rational ClearQuest®. Rational's SCM solution frees teams of all sizes to build better software faster. or desktop) 5. A sound process will improve quality. Using the same proven technology. quality and content and the right software-estimating tool can help optimize ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 17 . schedule. The operating platform of the software (commercial or military. c. Function Points (FPs). Rational's SCM solution offers Unified Change Management (UCM). analogies comparing input parameters to existing knowledge bases. space. 1. the most flexible defect and change tracking tool on the market. increase development speed and ultimately enhance overall team collaboration and productivity. version control. creates a software configuration management (SCM) solution that helps your team handle the rigors of software development. Share code effortlessly and automate error-prone processes Rational's SCM solution offers the essential functions of transparent code sharing. CM Tool Examples: i. ground. Software cost-estimating tools solicit input from the users describing their software project and from these inputs the tool will derive a cost (and usually schedule) estimate for that project. ESTIMATING . Software project planning is really a balancing act between cost. 1. yet error-prone tasks associated with software development. IS. Rational has the right size solution for your team. processes and protocols. operating systems. Approved changes to baselined software are made properly and consistently in all products. From small project teams to the global enterprise. air. there are many other needs the right estimating tool can address for your organization. Unite your team with a process that optimizes efficiency Process is critical to streamlining software development. 3. and advanced workspace and build management. various forms and versions from the time of their initial approval or acceptance until they have been incorporated into the final media.etc. a best practices process for managing change at the activity level and controlling workflow. Just smooth scalability ii. the cost estimating tools ask for the user to describe. The process that drives inputs to outputs is either cost estimating relationships derived from regression of actual data. web development. Choose a solution that scales and make it your last SCM decision Rational has an SCM solution that meets the needs of all size development teams. iii. Rational ClearCase®. and no unauthorized changes are made. The size of the software (either in source lines of code (SLOC).
. Early Design. It consists of three submodels. Most tools have other trade-off and analysis features as well – allowing the user to set a baseline and vary different parameters to optimize cost and schedule. complexity factors. effort. by quickly demonstrating the effect adjusting requirements. Many tools offer the capability ofestimating latent defects in the delivered product and then use this information to predict maintenance costs. 1. This offers the project manager the capability to make trade-offs based on the total cost of ownership. This new. However. This tool was first developed in 1977. After several years and the combined efforts of USC-CSE. Software size may be input directly.iv. Constructive Cost Model (COCOMO) . It can be customized and calibrated to the needs of the user d. and staffing might have on predicted costs and schedules (e.g. only the last and most detailed submodel. and risk analysis factors. for risk management or job bidding purposes). resources. and Postarchitecture models.The original COCOMO model was first published by Dr. and is considered one of the first complex commercially available tools used for software estimation. These changes included a move away from mainframe overnight batch processing to desktop-based real-time turnaround. A major input to PRICE S is Source Lines of Code (SLOC). this balance. descriptions of the methodology algorithms used can be found in papers published by PRICE Systems. or automatically calculated from quantitative descriptions (function point sizing). Successful use of the PRICE S tool depends on the ability of the user to define inputs correctly. v. and reflected the software development practices of the day. the result is COCOMO II. These changes and others began to make applying the original COCOMO model problematic. Another important feature that most cost-estimating tools deliver is the ability to perform a Risk Analysis. The equations used by this tool are proprietary. or developed with expert judgment. unless otherwise explicitly indicated. so that a confidence level can accompany your estimate. from best case to most likely to worst case outcomes. The implemented tool provides a range on its cost. operating environment. and schedule when planning a new software development activity. a greatly increased emphasis on reusing existing software and building new systems using off-the-shelf software components. Listed in increasing fidelity. CERs were determined by statistically analyzing completed projects where product characteristics and project information were known. and the COCOMO II Project Affiliate Organizations. In the ensuing decade and a half. The PRICE S tool is based on Cost Estimation Relationships (CERs) that make use of product characteristics in order to generate estimates. all further references on these web pages to "COCOMO II" or "USC COCOMO II" can be assumed to be in regard to the Post-architecture model. had been implemented in a calibrated software tool. software reuse. Post-architecture. these submodels are called the Applications Composition. effort. 2. PRICE S – A tool is distributed by Lockheed . and spending as much effort to design and manage the software development process as was once spent creating the software product. Barry Boehm in 1981. As such. each one offering increased fidelity the further along one is in the project planning and design process. improved COCOMO is now ready to assist professional software cost estimators for many years to come. It also allows a planner to easily perform "what if" scenario exploration. Other inputs include software function. The solution to the problem was to reinvent the model for the 1990s.Martin PRICE Systems. Until recently. rather than just development costs. Project Management ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 18 . a revised cost estimation model reflecting the changes in professional software development practice that have come about since the 1970s. COCOMO II is a model that allows one to estimate the cost. IRUS at UC Irvine. software development techniques changed dramatically. productivity factors. and schedule estimates.
1. The presentation of machine information to the human. The organization having the need for a product or service may be the owner. ii. The first two activities occur prior to the agreement. and Delivery and completion. Obtaining software through purchase or contract f. the last five after the agreement. The acquirer represents the needs and requirements of the users. The first three activities occur prior to the agreement. 2. Project management is the discipline (art and science) of defining and managing the vision. Contract. and Acceptance and completion. Request-for-Proposal preparation. The service may be the development of a software product or a system containing software. The process continues with the identification of procedures and resources needed to manage and assure the service. The acquisition process begins with the definition of the need to acquire a software product or service. Tasks performed. as well as the input/output facilities. Program Interfaces ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 19 . Also. in defining the automated solution to satisfy the business requirements & interfaces. Planning. to assess the testability of requirements. and the human interaction with the machine. document. and recording the business needs and problems to be resolved by the new or enhanced system. This process consists of the following activities along with their specific tasks: Initiation. The process continues with the preparation and issuance of a request for proposal.g. The process may be initiated either by a decision to prepare a proposal to answer an acquirer's request for proposal or by signing and entering into a contract or an agreement with the acquirer to provide a software service. Contract preparation and update. the operation of a system with software. and management of the acquisition process through the acceptance of the system. transmission of computer data over networks). prioritizing. is now being deployed to help organizations manage all types of change. The owner may contract all or parts of the acquisition tasks to an agent. b. Communications Interfaces That include transmission of information between computers and remote equipment (e. Supplier monitoring. Acquisition i. while traditionally applied to the management of projects. Requirements i. Roles/Responsibilities a. techniques used. and resources required to complete a project. implement. and documentation prepared. 3. the last two after the agreement. Execution and control. This life cycle process contains the activities and tasks of the supplier. Review and evaluation. ii. It's really the management acumen that oversees the conversion of "vision" into "reality". Acquisition Process (ISO 12207) . Design i. Tasks performed. Supply Process (ISO 12207) i. 3.This life cycle process defines the activities and tasks of the acquirer. techniques used. and maintain software systems. The supply process consists of the following activities along with their specific tasks: Initiation. support. test. including development and execution of plans through delivery of the service to the acquirer. selection of a supplier. tasks. that contractually acquires software product or service. Person/Machine Interfaces Interfaces that include the operating system and the development languages that are available. Project management. and documentation prepared in identifying. or the maintenance of a software product. Preparation of response. Performing tasks to manage and steer a project toward a successful conclusion. Understanding the documents developed in the tester’s organization to design.. e.i.
iii. 4. Principle 5 System approach to management Identifying. continually improve the effectiveness of the Quality Management System. Understanding the tenets of quality and their application in the enterprise’s quality program. Build and Install Tasks performed. 5. Curran and Sanders indicate that this quality process must adhere to four basic principles: ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 20 . They should create and maintain the internal environment in which people can become fully involved in achieving the organization's objectives. Software Quality: A framework for success in software development and support. Principle 3 Involvement of people People at all levels are the essence of an organization and their full involvement enables their abilities to be used for the organization's benefit. increase functionality (enhancement). Quality management systems Fundamentals and vocabulary. 7. Principle 7 Factual approach to decision making Effective decisions are based on the analysis of data and information. whether on the same computer.4. techniques used. Quality Principles i. Principle 1 Customer focus Organizations depend on their customers and therefore should understand current and future customer needs. 3. understanding and managing interrelated processes as a system contributes to the organization's effectiveness and efficiency in achieving its objectives. Interfaces for the exchange of information. 6. In their book. c. Principle 2 Leadership Leaders establish unity of purpose and direction of the organization. Principle 8 Mutually beneficial supplier relationships An organization and its suppliers are interdependent and a mutually beneficial relationship enhances the ability of both to create value. ii. Through management review. and in ISO 9004:2000. 8. or improve operational efficiency of speed. internal/external audits and corrective/preventive actions. These eight quality management principles are defined in ISO 9000:2000. Principle 4 Process approach A desired result is achieved more efficiently when activities and related resources are managed as a process. iv. meet changing operating environment conditions (adaptation). or distributed across multiple tiers of the application architecture. 2. Maintenance Software modification activities performed on an operational system to resolve problems (correction). 5. This document introduces the eight quality management principles on which the quality management system standards of the revised ISO 9000:2000 series are based. should meet customer requirements and strive to exceed customer expectations. Quality management systems Guidelines for performance improvements: 1. including installation of software. and documentation prepared in building the automated solution to satisfy the business requirements. Principle 6 Continual improvement Continual improvement of the organization's overall performance should be a permanent objective of the organization.
e. documentation and code. These should all be subjected to rigorous review methods such as inspections. The "V" Concept of Software Development: i. 4. The 'V' is also a synonym for Verification and Validation. “V” Model ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 21 . d. high quality inputs such as software tools and subcontracted software. For example.) server as a base for test activities. and to all key development products such as requirements. quality controls must be put in place during all stages of the development life cycle. and whether those standards and procedures are adequate to ensure the quality of the project in general. 2. Methods for doing this include the use of appropriate software engineering standards and procedures. the more expensive they are to correct. the system test is carried out on the basis of the results specification phase. walkthroughs. 3. designs. and lastly… Independently audit the work for compliance with standards and procedures. independent quality auditing to ensure standards and procedures are followed. establish a formal method of accumulating and disseminating lessons learned from past experiences and mistakes. By removing the cause. Therefore.1. By the ordering of activities in time sequence and with abstraction levels the connection between development and test activities becomes clear. This is a two part audit conducted at the process level using SEI or SPR assessment methodologies. and technical reviews. Many of the process models currently used can be more generally connected by the 'V' model where the 'V' describes the graphical arrangement of the individual phases. Eliminate the causes as well as the symptoms of the defects. Ensure that defects are detected and corrected as early as possible. as the longer the errors go undetected. Oppositely laying activities complement one another (i. as well as audits at the project level which will determine if project activities were carried out in accordance to the standards and procedures established in the quality process. you have in effect improved the process (and recall that continuous process improvement is another key tenet in how Total Quality Management principles are applied to quality software). The "V" concept relates the build components of development to the test components that occur during that build phase. ii. At least as much effort should be placed in keeping defects out of the code as detecting their presence in the code. This is an extension of the previous principleóremoval of the defect without eliminating the cause is not a satisfactory way to solve the problem. Prevent defects from being introduced.
also known as the Testing 'V' Model. The following figure depicts the same: ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 22 .Requirements Specifications Validate Requirements Acceptance Testing System Testing Architectural Design Verify Design Integration Testing Detail Design Unit Testing Coding iii. Effective Methods of Software Testing proposes the 11 Step Software Testing Process. The 'V' Model as proposed by William E Perry – William E Perry in his book.
Define Software Requirements Assess Development Plan and Status Develop the Test Plan Build Software Test Software Requirements Test Software Design Program Phase Testing Operate and Maintain Software Execute and Record Results Acceptance Testing Report Test Results Operate and Maintain Software Test Software Installation Test Software Changes Evalutate Test Effectiveness ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 23 .
they interact with each other and often cause conflict. (IEEE) standards. 1. There are many quality models and standards. pros and cons. especially when software providers try to incorporate them into the software development process. B. ISO 9000. C. ISO 9126 incorporates six quality goals. and how assessments and baselines can be developed using a quality model. the Malcolm Baldrige National Quality Award. This category will test the CQA candidate’s understanding of model objectives. Three often used models are discussed here as examples. D. McCall's Model of Software Quality (The GE Model. product revision.3. SPICE (ISO 15504). Boehm's Model (1978) is based on a wider range of characteristics and incorporates 19 criteria. structure.Software Life Cycle Processes. ISO 12207 Standard for Information Technology . and product transition. and the Quality Assurance Institute’s Approach to Implementing Quality. The criteria and goals(1) defined in each of these models are listed below: ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 24 . The criteria in these models are not independent. The Institute of Electrical Electronics Engineers. Inc. Quality Models and Quality Assessment A. Purpose of a Quality Model To satisfy business goals and objectives Requirements are imposed by a customer For competitive reasons As a guide (roadmap) to continuous improvement 2. Most notably are the Software Engineering Institute’s Capability Maturity Model (CMM). each goal having a large number of attributes. A. 1977) incorporates 11 criteria encompassing product operation. There are many models of software product quality that define software quality attributes. A.
defining. Malcolm Baldrige was Secretary of Commerce from 1981 until his death in a rodeo accident in July 1987. and heroic efforts required by individuals to successfully complete projects. The Software CMM has become a de facto standard for assessing and improving software processes. and to publicize these organizations’ successful performance strategies.B. attribute. Few processes are defined. strategic planning. goals and attributes interchangeably. Now considered America’s highest honor for performance excellence. Few if any processes in place.S. The Capability Maturity Model for Software (CMM or SW-CMM) is a model for judging the maturity of the software processes of an organization and for identifying the key practices that are required to increase the maturity of these processes. Baldrige was a proponent of quality management as a key to this country’s prosperity and long-term strength. to recognize quality and business achievements of U. B. organizations by the President of the United States. and. The CMM is designed to provide organizations with guidance on how to gain control of their process for developing and maintaining software and how to evolve toward a culture of software excellence. Malcolm Baldrige National Quality Award A.S. The Baldrige Award is given by the President of the United States to businesses—manufacturing and service. successes may not be repeatable. the SEI and community have put in place an effective means for modeling. He took a personal interest in the quality improvement act that was eventually named after him and helped draft one of the early versions. Characterized by chaos. Industry Quality Models 1. and business results.S. human resource focus. we will use the terminology in ISO 9126 . education and health care. Initial (Level 1) . C. Software Engineering Institute’s Capability Maturity Model A. information and analysis. These three models and other references to software quality use the terms criteria. disciplined software processes. initiated by the U. 2. B. C. 1) The Capability Maturity Model for Software describes the principles and practices underlying software process maturity and is intended to help software organizations improve the maturity of their software processes in terms of an evolutionary path from ad hoc. To avoid confusion. Defense Department to help improve software development processes. the National Institute of Standards and Technology designed and manages the award and the Baldrige National Quality Program. and measuring the maturity of the processes used by software professionals. periodic panics. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 25 . and success depends on individual effort and heroics. small business. B.S. Through the SW-CMM. Awards are given in manufacturing. slowly growing U.The software process is characterized as ad hoc. A. organizations. metric. and occasionally even chaotic. the Baldrige Award is presented annually to U. service. Congress established the Malcolm Baldrige National Quality Award to promote quality awareness. The CMM is organized into five maturity levels: i. It does this by serving as a model against which an organization can determine its current process maturity and by identifying the few issues most critical to software quality and process improvement. In 1987. quality movement. In recognition of his contributions. starting in 1999. process management. chaotic processes to mature. customer and market focus. SEI = 'Software Engineering Institute' at Carnegie-Mellon University. Congress named the award in his honor. D. jumpstarting a small. small and large—and to education and health care organizations that apply and are judged to be outstanding in seven areas: leadership. In conjunction with the private sector.goal.
Intergroup Coordination. B. and Peer Reviews. as the organization establishes an infrastructure that institutionalizes effective software engineering and management processes across all projects.Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies. It is geared to large organizations such as large U. Integrated Software Management. B. effectiveness. B. They are Requirements Management. realistic planning. They are Defect Prevention. and control of an organization's software processes are believed to improve as the organization moves up these five levels. However. A. developed by the SEI. Both the software process and products are quantitatively understood and controlled. A. Defense Department contractors. CMM = 'Capability Maturity Model'. tailored version of the organization's standard software process for developing and maintaining software. Organization Process Definition. and functionality. standardized. and Software Configuration Management. Predictability. A. and training programs are used to ensure understanding and compliance. requirements management. and Process Change Management.S. They are Organization Process Focus. The focus is on continouous process improvement. All projects use an approved. Software Project Tracking and Oversight. and configuration management processes are in place. successful practices can be repeated. They are Quantitative Process Management and Software Quality Management. Technology Change Management. Metrics are used to track productivity. and quality is consistently high. Defined (Level 3) . and integrated into a standard software process for the organization. The key process areas at Level 3 address both project and organizational issues. Managed (Level 4) .Basic project management processes are established to track cost. schedule. Software Subcontract Management. the empirical evidence to date supports this belief E. vi. The key process areas at Level 5 cover the issues that both the organization and the projects must address to implement continual. Software Quality Assurance. While not rigorous. a Software Engineering Process Group is is in place to oversee software processes. processes. and if reasonably applied can be ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 26 . Optimizing (Level 5) . Software project tracking. and products. B. measurable software process improvement. Software Project Planning. iv.Detailed measures of the software process and product quality are collected. v. Software Product Engineering. iii. The necessary process discipline is in place to repeat earlier successes on projects with similar applications. many of the QA processes involved are appropriate to any organization.ii. Repeatable (Level 2) . The Key Process Areas (KPA) at Level 2 focus on the software project's concerns related to establishing basic project management controls. The key process areas at Level 4 focus on establishing a quantitative understanding of both the software process and the software work products being built. Training Program. Project performance is predictable. The impact of new processes and technologies can be predicted and effectively implemented when required.The software process for both management and engineering activities is documented. A. It's a model of 5 levels of organizational 'maturity' that determine effectiveness in delivering quality software. Standard software development and maintenance processes are integrated throughout an organization.
Based on the best current practices in the fields such as human resources and organizational development. and establish a culture of software engineering excellence. Except for Level 1. Each maturity level is a well-defined evolutionary plateau that institutionalizes a level of capability for developing the talent within the organization. motivate. They are: Work Environment Communication Staffing Performance Management Training Compensation 3) Defined . The P-CMM helps organizations to characterize the maturity of their work-force practices. The P-CMM can also be used by any kind of organization as a guide for improving their people-related and work-force practices. integrate work-force development with process improvement. and successfully managing the people assets of the organization. developing effective teams. organize. each maturity level is decomposed into several key process areas that indicate the areas an organization should focus on to improve its workforce capability. The motivation for the P-CMM is to radically improve the ability of software organizations to attract. and retain the talent needed to continuously improve software development capability. The key practices describe the infrastructure and activities that contribute most to the effective implementation and institutionalization of the key process area. Organizations can receive CMM ratings by undergoing assessments by qualified auditors. People Capability Maturity Model (P-CMM) The People Capability Maturity Model® (P-CMM®) adapts the maturity framework of the Capability Maturity Model® for Software (CMM®) [Paulk 95].The key process areas at Level 2 focus on instilling basic discipline into workforce activities. and motivation of the work force. It describes an evolutionary improvement path from ad hoc. develop. The P-CMM is designed to allow software organizations to integrate work-force improvement with software process improvement programs guided by the SW-CMM. They are: Knowledge and Skills Analysis Workforce Planning Competency Development Career Development Competency-Based Practices Participatory Culture ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 27 . Each key process area is described in terms of the key practices that contribute to satisfying its goals. disciplined development of the knowledge.The key process areas at Level 3 address issues surrounding the identification of the organization's primary competencies and aligning its people management activities with them. the P-CMM provides organizations with guidance on how to gain control of their processes for managing and developing their work force. set priorities for immediate actions. skills. F. inconsistently performed practices.helpful. The P-CMM consists of five maturity levels that lay successive foundations for continuously improving talent. to a mature. The five maturity levels of the P-CMM are: 1) Initial 2) Repeatable . just as the CMM describes an evolutionary improvement path for the software processes within an organization. guide a program of continuous workforce development. to managing and developing an organization's work force.
ISO 9126 is the software product evaluation standard that serves to eliminate any misunderstanding between purchaser and supplier. It covers documentation.4) Managed . Note that ISO certification does not necessarily indicate quality products . at both the organizational and the individual level. development.The ISO 9001:2000 standard (which replaces the previous standard of 1994) concerns quality systems that are assessed by outside auditors. (c)Q9004-2000 . To be ISO 9001 certified.The key process areas at Level 4 focus on quantitatively managing organizational growth in people management capabilities and in establishing competency-based teams. ISO 9002 and ISO 9003 which were integrated into ISO 9001:2000. They are: Personal Competency Development Coaching Continuous Workforce Innovation 3. not just software. ISO 9126 is the software product evaluation standard that defines six characteristics of software quality: 1) Functionality is the set of attributes that bear on the existence of a set of functions and 2) 3) 4) 5) 6) their specified properties.Quality Management Systems: Guidelines for Performance Improvements. ISO 9000 family includes ISO 9001. servicing. and other processes. They are: Mentoring Team Building Team-Based Practices Organizational Competency Management Organizational Performance Alignment 5) Optimizing . under stated conditions Maintainability is the set of attributes that bear on the effort needed to make specified modifications Portability is the set of attributes that bear on the ability of software to be transferred from one environment. by a stated or implied set of users Efficiency is the set of attributes that bear on the relationship between the level of performance of the software and the amount of resources used. which is an international "quality management system" standard--a standard used to assess an organization's management approach regarding quality. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 28 .Quality Management Systems: Fundamentals and Vocabulary. and on the individual assessment of such use. production. D. and certification is typically good for about 3 years. ISO 9000 / ISO 9004 Quality Management Principles and Guidelines on their Application A. after which a complete reassessment is required. (b)Q9000-2000 . design. and it applies to many kinds of production and manufacturing organizations. B.it indicates only that documented processes are followed. The functions are those that satisfy stated or implied needs Reliability is the set of attributes that bear on the capability of software to maintain its level of performance under stated conditions for a stated period of time Usability is the set of attributes that bear on the effort needed for use. E. The full set of standards consists of: (a)Q9001-2000 . installation. a third-party auditor assesses an organization.Quality Management Systems: Requirements. ISO = 'International Organisation for Standardization' . C.The key process areas at Level 5 cover the issues that address continuous improvement of methods for developing competency. ISO 9000 family of standards presents an overview of the standards and demonstrates how they form a basis for continual improvement and business excellence. testing.
development.Standard for Information Technology . It also specifies eight "supporting processes"-documentation. Assurance of the process by which a product is developed (ISO 9001). launched within the International Standards Organization has been developing a framework standard for software process assessment. and the evaluation of the quality of the end product (ISO 9126 ) are important and both require the presence of a system for managing quality. It is especially suitable for acquisitions because it recognizes the distinct roles of acquirer and supplier. design phase. Responsibility is one of the key principles of total quality management. The derivation of the processes is based upon two basic principles: modularity and responsibility.A process is considered to be the responsibility of a party in the software life cycle.Life Cycle processes A.) This is in contrast to a "text book approach. bringing together the major suppliers and users of assessment methods. maintenance. and training.Standard for Information Technology (Software Process Improvement and Capability dEtermination) A. B. The project has three principal goals: 1) to develop a working draft for a standard for software process assessment 2) to conduct industry trials of the emerging standard 3) to promote the technology transfer of software process assessment into the software industry world-wide ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 29 . as discussed later. An individual process is dedicated to a unique function. (IEEE-STD-610) 5. C." where the life cycle functions could be studied as topics or subjects. and will continue until ISO/IEC 15504 is published as a full International Standard. Since 1993. audit. joint review. Software life cycle architecture . infrastructure. B. D. In fact. implementation phase. etc. supply. validation. maintenance. each party has certain responsibilities. 1) Modularity . ISO 12207 . quality control." and the activities into "tasks. 4. and operation.The standard establishes a top-level architecture of the life cycle of software. SOFTWARE LIFE CYCLE . The software life cycle typically includes a concept phase. test phase. The architecture is built with a set of processes and interrelationships among these processes. quality assurance. design. requirements phase. and sometimes retirement phase.acquisition. It is not applicable to the purchase of commercial-off-the-shelf (COTS) software products. It divides the five processes into "activities. The life cycle begins with an idea or a need that can be satisfied wholly or partly by software and ends with the retirement of the software. SPICE ISO 15504 . or operation of a software system. and problem resolution--as well as four "organizational processes"--management. 2) Responsibility . they are maximally cohesive and minimally coupled to the practical extent feasible. such as management. scheduled by 2002. operation and maintenance phase. the standard is intended for two-party use where an agreement or contract defines the development. In other words. the SPICE (ISO/IEC 15504) (Software Process Improvement and Capability dEtermination) project. Field trials of SPICE-based assessment commenced in January 1995." while placing requirements upon their execution. ISO 12207 describes five "primary processes"-.F. ISO 12207 offers a framework for software life-cycle processes from concept through retirement. verification. that is. improvement. SPICE (ISO/IEC 15504) is a major international initiative to develop a Standard for Software Process Assessment.The processes are modular. The project is carried out under the auspices of the International Committee on Software Engineering Standards through its Working Group on Software Process Assessment (WG10).The period of time that begins when a software product is conceived and ends when the software is no longer available for use. configuration management. measurement. installation and checkout phase.
IEEE = 'Institute of Electrical and Electronics Engineers' . 7. The Approach shows methods for improving all of the activities within IT. 1) Our Approach is a business-oriented approach. and develop a strategic measurement dashboard. processes that ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 30 . inhibitors. constraints. weaknesses. 5) Continuously improve the above process categories. C. information & other technologies. QAI has developed a detailed how-to approach to quality improvement. 4) Establish strategic and tactical dashboards to enable management to effectively use quantitative data in their management processes. and provide proof that IT is operating effectively and efficiently. organizational weaknesses. The Approach recognizes that many organizations must first reestablish credibility because of missed schedules. D. A well-defined mission statement ii. 3) Establish. implement. The Seven Phase Performance Implementation Framework: 1) Improving performance is not a one-time “quick fix”. Manage by process iii. Each category is segmented into specific how-to processes. align and deploy processes to support the defined management results. management’s perception of “where the pain is”. customer/user needs. Manage toward results ii. It recognizes the close working relationship that must exist between IT management. and failure to implement the needed requirements. A clearly defined vision B.6. build an enviroment where products are completed on time and within budget. This phase is to conduct an organization wide data gathering process to assist in: identifying strengths. It is a continuum of activity that requires the understanding and buy-in of all employees. 3) Quality and Performance Analysis i. creates standards such as 'IEEE Standard for Software Test Documentation' (IEEE/ANSI Standard 829). customers/users. specific performance objectives. The IEEE is a global technical professional society serving the public interest and members in electrical. opportunities. This approach is composed of five process categories. QAI'S Strategic Model 1) QAI's Strategic Model contains four processes critical to your success: i. E. organizational strengths. overrun budgets. The Institute of Electrical Electronics Engineers (IEEE) Standards A. Manage by fact iv. electronics. and threats. The Quality Assurance Institute’s Approach to Quality Implementation A. 'IEEE Standard for Software Quality Assurance Plans' (IEEE/ANSI Standard 730). and staff. The objectives of this phase are to: establish and clarify the organization’s vision. 'IEEE Standard of Software Unit Testing (IEEE/ANSI Standard 1008). and a feedback system based upon metrics. QAI has developed a customizable approach that is driven by your management's style. B. QAI's Approach for Managing Quality in a Changing World is designed to enable IT organizations to restore credibility. computer. QAI subscribes to a sevenphase approach: 2) Establishing A Partnership i. and others. superior performers.among other things. 2) Align information services with corporate objectives and define the desired results to support those objectives. These five categories are: 1) Establish a quality environment within the I/S function. Manage continuous improvement 2) QAI believes that the foundation to any quality initiative is: i.
Criteria may include A. Evaluating Solutions i. (SW-CMM (IEEE-STD-610)) A formally approved version of a configuration item. The primary industrial standards body in the U. Model Selection Process 1. regardless of media.0) A configuration identification document or a set of documents formally designated by the Government at a specific time during a configuration item’s life cycle. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 31 . and that can be changed only through formal change control procedures. The initially approved documentation describing a system’s or configuration item’s functional characteristics and the verification tests required to demonstrate the achievement of those specified functional characteristics. and the verification tests required to demonstrate the achievement of those specified functional and interface characteristics. This phase ensures that the solutions are increasing performance as defined by management’s strategic measurement dashboard. to accomplish the agreed upon improvement goals is the goal of this phase. ii. audit existing standards for applicability and status. as follows: i. Management commitment to include needed: a. the agreed upon solutions. Improving Solutions i. ANSI = 'American National Standards Institute' A. Using proven industry work practices. Develop a Consensus Approach i.. b. For configuration management. C. This phase is to prepare and deliver the overall organizational assessment findings and gain the consensus agreement of the project sponsors to the findings. Baselines. Implementing Solutions i. publishes some software-related standards in conjunction with the IEEE and ASQ (American Society for Quality). Applicability of model to the IT organization’s goals and objectives. (IEEE/EIA 12207. Need for Baseline Assessments D. Applicability of model to the IT organization’s goals and objectives. 8. B. plus approved changes from those baselines. constitute the current configuration identification. additional design constraints. interface requirements with interfacing configuration items. Functional Baseline. there are three baselines. This final phase is to work towards building a culture and environment conducive to continuous improvement of performance enhancers. tools (manual or automated) in use. which are established sequentially. A specification or product that has been formally reviewed and agreed upon. b. Management commitment to include needed: C. Allocated Baseline. Product baselines are references points in vital areas of the application that can be used to measure development progress. The initially approved documentation describing a configuration item’s interface characteristics that are allocated from those of the higher level configuration item or those to a lower level. that thereafter serves as the basis for further development. BASELINE a. This phase is to implement within the organization’s culture and constraints. Using Models for Assessment and Baselines: A.S. How an IT organization selects a model. Developing Solutions i. B. Need for measurable goals and objectives E. c. and QAI’s improvement framework. formally designated and fixed at a specific time during the configuration item's life cycle. and establish performance baselines.4) 5) 6) 7) 8) are currently performed.
This Policy aims at: 1. which is used as a benchmark for comparing actual process performance against expected process performance. hashing out release criteria at the time the test plan is written is far superior to doing it at the release checklist meeting. It is our goal to posture our company for market expansion. The most important prerequisite for a successful implementation of any major quality initiative is commitment from executive management. it is impossible to determine if we have met our objectives. If we have not met our objectives. A process performance baseline is typically established at the project level. It's a long term commitment aimed at continuously improving the way we work. (DODD 5010. ii. then we cannot in good conscience say that a product is ready to ship. CMM’s Process capability baseline (PCB) . Is a clear definition of the result you are trying to achieve. the selected physical characteristics designated for production acceptance testing and tests necessary for production and support of the configuration item.C. Without clear quality goals agreed upon up front. providing them with the resources that are necessary to carry out their duty to the best of their abilities. Quality Goals .Our journey is Total Quality Management--fully satisfying our customers requirements through a process of continuous improvement. CMM’s Process performance baseline (PPB) . The initially approved documentation describing all of the necessary physical and functional characteristics of the configuration item. Quality Management/Leadership A. Management’s Quality Directives C. ii. f. Unpleasant as it may be. thereby providing improved job security and quality of life for all b. 4. It is management’s responsibility to establish strategic objectives and build an infrastructure that is strategically aligned to those objectives. Example of a Corporate Quality Vision: A. Product Baseline. commits itself to carry out a Quality Policy. 10/87) A system life cycle documentation standard established to enhance program stability and provide a critical reference point for measuring and reporting the status of program implementation. CM. Phelps Dodge Copper Products & Refining Corporation . B. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 32 . iii. Explain how the vision will be achieved. including manufacturing processes and procedures. Mission Statement a. managing our business processes. providing a safe work environment.defined as “a documented characterization of the range of expected results that would normally be achieved by following a specific process under typical circumstances. Assuring the training and the professional growth of its employees. It's critical to understand that Total Quality Management is not a short term program. and supplier selection/retention.defined as “a documented characterization of the actual results achieved by following a process.19.d. any required joint and combined operations interoperability characteristics of a configuration item (including a complete summary of other service and allied interfacing configuration items or systems and equipment). iii. Quality Vision i. Examples of Quality Goals: A. Progress implies a goal. e. although the initial process performance baseline will usually be derived from the organization’s process capability baselines. materials. This category will test the CSQA candidate’s understanding of the management processes used to establish the foundation of a quality-managed environment. Quality Goals i.To that goal the management of Savaré I.
and Fiber-Optic Communications industries worldwide. Quality Policy i. Argo-Tech Corporation . proposing actions of improvement and verifying their correct applications. including objectives for quality and commitment to quality . Bangalore. Outstanding service to our Customers b." It goes on to say that the policy "shall be relevant to the supplier's organizational goals and the expectations and needs of its customers. Timely and Qualitative software solutions. Newport Corporation – To be the leading supplier of high quality optics. Can be defined as standards ii. Mummert & Partners. Semiconductor Equipment. Quality Charter i. ii. Spectra-Physics Scanning Systems – “We the employees of SpectraPhysics Scanning Systems make the personal commitment to first understand our customers expectations then. Quality Values i. Quality Management Principles provide understanding of and guidance on the application of Quality Management. Section 4. c. Cost-effective. Honesty and integrity in all that we do D.1 of the ISO 9001 standard requires that management "shall define and document its policy for quality. shareholders. Convincing project results e.To meet or exceed all requirements agreed to with our customers. Example of Quality Values: A. quality. Our Quality Values and Beliefs: a. . to meet or exceed our commitment to those expectations by performing the correct tasks defect free. Zenith Software Limited. The statement of the responsibilities & authorities of all Quality function performers. Keeping under control the products of non compliance. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 33 . Top-notch industry know-how and specialized expertise d. Quality Assurance Charter a.Our quality principles comprise the following concepts: a. and value in our products c. iii.” c. Creativity and teamwork in the workplace d. Computer Peripherals. suppliers. b. Quality Principles i. and community e. Maximum quality in performing projects f. instruments. . micropositioning and measurement products and systems to the Scientific and Research. Developing its organization to foster improvements in the company services. Respect for our Customers. Best quality for our clients c. Examples of Quality Principles: A. employees. Promotion of quality in human-resource work d. Quality is a principle for our management and work b. Examples Of Corporate Quality Policies: a. The statement of the enterprise’s commitment to Quality. Can be defined as procedures." A. Inc. Innovation. ii. on time. . every time. 3. – “We practice continual Improvement to achieve customer delight by providing Customer-Centric.” b.1. spotting and eliminating the causes of non compliance.2. Powerful internal organization g.
Quality Assurance is a professional competency whose focus is directed at critical processes used to build products and services. Process Orientation: A Quality Champion leads. We will use technology innovatively and with a human touch. based upon the holiday information provided by the tour operator. B. We will do so with warmth. and positively leads others towards that vision by example. F. Quality Assurance A. grace and with typical Temasek style. Establishing a Function to Promote and Manage Quality: A. Selling Quality i. has a positive attitude. and the like. by listening to our customers and meeting their needs. Common methods are: a. we will create the best environment to work and study. Educating all members in the value of quality & their responsibility for it. 3. c. People Oriented: A Quality Champion is an open minded individual who is receptive to new ideas. Questionnaires c. shares responsibility with IT management to market and deploy quality programs. or work group performance. Examples of Qualit Charters: A. AITO defines ‘quality’ as “providing a level of satisfaction which. The profession is charged with the responsibility for tactical process improvement initiatives that are strategically aligned to the goals of the organization. B. team or work group results. and by continually improving our processes. This category will test the CQA candidate’s ability to understand and apply quality assurance practices in support of the strategic quality direction of the organization. Observation ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 34 . to gather the data that can be used for the improvement of the enterprise’s information processes. team.) E. products and services. Interviewing b. Identifying or developing and using problem reports. We will achieve academic and administrative excellence by encouraging and expecting the creative involvement of all staff. (Customer may be internal and/or external. Build and mature a quality assurance function including staffing. We will by our own example. inspire our students and community to keep improving themselves through continuous learning. 2. aims to meet or exceed a customer’s reasonable expectations. D. The spokesperson for quality within the IT organization. 1. At Temasek Polytechnic. Customer Focus: A Quality Champion actively works to make the customer a priority in their workplace and perseveres regardless of the barriers that may be encountered. Leadership: A Quality Champion has a vision of the quality organization. 5. iii. regardless of the type of holiday sold or the price paid”. so as to achieve our mission and vision for the betterment of the people of Singapore. planning. and encourages the participation of all individuals in the organization. and plan execution.ii. team or work group. shares that vision with others. We at Temasek Polytechnic are committed to exceeding the expectations of our stakeholders and customers in the delivery of all our courses and services. supports and/or participates in the development of processes which will lead to better organization. courtesy. C. Data-Gathering Techniques: A. A statement of corporate standards and quality of service. Co-operation: A Quality Champion embraces the spirit of teamwork by working in co-operation with others in order to achieve the desired organizational. B. Quality Champion: A. AITO is an association of independently-minded companies specialising in particular areas or types of holiday and sharing a common dedication to high standards of quality and personal service.
Quality Plan: A. it is often useful to "take stock" and look back at how the process was managed. in business terms? If not. How you ensure that the product works properly (verification). iv. vii. reviews of test scripts. Quality Tools: A. 7. code reviews. both incrementally during development and for the entire product before delivery to EPRI. Problem Trend Analysis: A. methods. B. Identifies repetitive problems and assesses how often given problems occur. SOFTWARE QUALITY ASSURANCE PLAN . viii. The Quality Plan describes: i. incident reports. iii. Repertory Grids Concept Mapping Joint Application Design 4. It also provides a mechanism to track progress of problem resolution. 5. and criteria you use to determine whether the software has passed each review. e. C.. The main interest in this analysis is locating where key problems are occurring and the frequency of occurrence. Post-Implementation Reviews (PIR): A. but it will also help improve any future implementations. The Quality Plan outlines how you will build quality into the software and documentation. Develop a tactical quality plan. How you track multiple development builds of the software to avoid confusion (configuration management). The dates assigned to key tasks in the Quality Plan are entered into the project plan. B. Technique used to review results of projects after their completion and implementation. Process Analysis and Understanding: A. Examination of problem reports. How you track and resolve defects. walk throughs. using and encouraging the use of quality tools. resource utilization. and better manage error-detection resources. How and when you conduct design reviews. to seek out error-prone products. How you plan for and execute testing. Applying Quality Assurance to IT Technologies and IT Technical Practices ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 35 . or only the most complex parts?). A PIR is an independent. Understanding. Identifying activities that comprise a process so that analysis can be performed. what should be done? What are the lessons learned that will improve future performance? 8. vi. How you control changes. objective review that is a key part of the benefits management process. C. After the implementation of any major new piece of software. and so forth. etc. reviews of test results (for example. ii. f. Definitions. Evaluation may include compliance with approved requirements. 6. Analysis of the gathered data to understand a process and its strengths and weaknesses and ability to watch a process "in motion" so that recommendations can be made to remove flawintroducing actions and build upon successful flaw-avoidance and detection actions. anticipate future error experience. 9. B. The Quality Plan describes how a developer's overall quality process guidelines will be applied to a project. It is used to answer the questions: Did we achieve what we set out to do. Process Identification: A. is 100% of all code checked. B. expected ROI. Not only will this highlight any issues which may need resolving over the forthcoming months. How you ensure that the product meets the requirements (validation).Plan which indicates the means by which the SQA requirements are met during the information system’s life cycle.d. It defines what is meant by the various quality-related tasks in the Project Plan. v.
The initiation of a quality improvement program reduced the number of defectives?) ii. v. Identify the problem area b. iii. by process) d. Management Tools: a. by machine. iv. Although the print quality is also of some concern. Count the named incedences d. Analyzing problems or causes by different groupings of data. (ex. (ex. Steps for preparing a Pareto analysis: a. Pareto charts are extremely useful because they can be used to identify those factors that have the greatest cumulative effect on the system. and in an orderly format. Prioritizing problems or causes to efficiently initiate problem solving. Pareto Charts are used for: a. such defects are substantially less numerous than the edge flaws. Pareto Chart i. this allows the user to focus attention on a few important factors in a process.The phenomenon whereby a small number of concerns is usually responsible for most quality problems. (ex. and thus screen out the less significant factors in an analysis. the most essential factors for the analysis are graphically apparent. Pareto Principle . in decending order. Which problem with Product X is most significant to our customers?) b. Name the events/items/causes that will be analyzed c. an Italian economist who found that a large percentage of wealth was concentrated in a small proportion of the entire population. The principle is named for Vilfredo Pareto. Ideally. (ex. Validate reasonableness of the Pareto analysis b. Analyzing the before and after impact of changes made in a process. When this is done. Solution of what production problem will improve quality most?) c. Focusing on critical issues by ranking them in terms of importance and frequency.A. They are created by plotting the cumulative frequencies of the relative frequency data (event count data). Rank the count by frequency (using bar chart) e. Cause and Effect Diagram (Fishbone) ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 36 . From the Pareto Chart it is possible to see that the initial focus in quality improvement should be on reducing edge flaws.
In any case. c. and brainstorming. Minor branches correspond to more detailed causal factors. Thus.a. Discovery Analysis Improvement Monitoring Implementation Verification d. as it illustrates the relationship between cause and effect in a rational manner. it is essential to establish a system that will continuously promote quality in all aspects of its operation B. given a particular effect. which typically involves either reducing the process variability or identifying specific problems in the process. f. b. attempts to find key problem areas in a process can be a hit or miss proposition. c. In many instances. This type of diagram is useful in any analysis. d. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 37 . a Cause and Effect diagram of the related process is created to help the user see the entire process and all of its components. Each major branch of the diagram corresponds to a major cause (or class of causes) that directly relates to the effect. In this instance. the diagram is constructed to identify and organize possible causes for it. other methodologies may need to be developed to allow for sufficient customization to a certain specific process. The primary branch represents the effect (the quality characteristic that is intended to be improved and controlled) and is typically labelled on the right side of the diagram. the tools should be utilized to ensure that all attempts at process improvement include: a. i. Furthermore. Thus. This diagram. it was decided to collect data on the curetimes of the material c. it is important to note that the mere use of the quality control tools does not necessarily constitute a quality program. The tools listed above are ideally utilized in a particular methodology. e. Having decided on which problem to focus on. Problem Identification Tools: Demonstrate an understanding of tools such as flow charts. to achieve lasting improvements in quality. also called an Ishikawa diagram (or fish bone diagram). b. is used to associate multiple possible causes with a single effect. check sheets. However.
by incorporating a depiction of the system under analysis into the form. ii. control charts. the utility of the checksheet may be significantly enhanced. The data collected will be used in subsequent examples to demonstrate how the individual tools are often interconnected ii. By breaking the process down into its constituent steps. Flowchart i. flowcharts can be useful in identifying where errors are likely to be found in the system. Checksheet i. and force field analysis. scatter diagrams. iii. The function of a checksheet is to present information in an efficient. graphical format. A defect location checksheet is a very simple example of how to incorporate graphical information into data collection. This may be accomplished with a simple listing of items. in some instances. Problem Analysis Tools: Demonstrate an understanding of tools such as histograms. C.a. the flowchart simplifies the analysis and gives some indication as to what event may be adversely impacting the process. However. Flowcharts are pictorial representations of a process. b. By breaking down the process into a series of steps. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 38 . i. Additional data collection checksheet examples demonstrate the utility of this tool.
d. iii. Applying curing time test data to create a scatterplot. some degree of quality improvement is likely.0 minutes. Histogram i. Deviations from a normal distribution in a histogram suggest the involvement of additional influences in the process. e. In addition to the ease with which they can be constructed. the curetime distribution does not appear to be a normal distribution as might be expected. Control Chart ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 39 . ii. but is bimodal instead. histograms provide the easiest way to evaluate the distribution of data. Scatter diagrams are graphical tools that attempt to depict the influence that one variable has on another. Data for a test of curetimes was collected and analyzed using a histogram. Thus. From this chart. ii. Scatter Diagram i. graphical view of accumulated data. it is possible to see that there are very few defects in the range of approximately 29. A common diagram of this type usually displays points representing the observed value of one variable corresponding to the value of another variable. including its dispersion and central tendancy.c.5 to 37. Histograms provide a simple. it is possible to conclude that by establishing a standard curetime within this range.
Run charts evolved from the development of these control charts. ii.i. On the other hand. Applying statistical formulas to the data from the curetime tests of base material. f. The control chart is the fundamental tool of statistical process control. Forcefield Analysis ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 40 . Run charts are useful in discovering patterns that occur over time. Thus. ii. improvements in common cause variation require fundamental changes in the process. as it indicates the range of variability that is built into a system (known as common cause variation). As a result. iii. which can typically be found and eliminated. Run Charts i. The bounds of the control chart are marked by upper and lower control limits that are calculated by applying statistical formulas to data from the process. g. but run charts focus more on time patterns while a control chart focuses more on acceptable limits of the process. it helps determine whether or not a process is operating consistently or if a special cause has occurred to change the process mean or variance. Data points that fall outside these bounds represent variations due to special causes. Run charts are used to analyze processes according to time or order. we can see that the process is in a state of statistical control. it was possible to construct X-bar and R charts to assess its consistency.
D. Client server.Refers to any of a variety of computer systems that use more than one computer. or a selling channel. B. Web based systems i. This includes parallel processing. list the forces that are helping. The objective of this skill is to identify where and how the Quality Assurance professional can control IT technologies and technical practices such as: A. storefront. i. Identifying risks of distributed processing. Web-based systems integration is the art of combining multiple systems (including Legacy systems and proprietary software applications) into a new system that is accessible through a Web browser. On the right side. E. however. A method of analyzing a situation by looking at all the forces and factors affecting the issue. ii. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 41 . Distributed Processing . A technique which helps us to achieve improvement by considering those factors or forces that encourage change or those which work against change. F. ii. ii. Brochure ware. Backup and Recovery i. On the left side. Ensuring customer’s confidential data is not compromised. Security i. B. It should be used whenever a change or improvement is needed. Reducing development cycle time with disciplined processes. or could help. 2. Most distributed processing systems contain sophisticated software that detects idle CPUs on the network and parcels out programs to utilize them. Restart application after problems are encountered. E-Commerce. in which a single computer uses more than one CPU to execute programs. Improvement will happen only if the encouraging factors are strengthened or the inhibiting factors are weakened. drive the group towards the goal. 1. to run an application. C.i. or processor. Privacy i. or could get in the way of reaching the goal. list the forces that are hindering the situation. i. More often. distributed processing refers to localarea networks (LANs) designed so that a single program can run simultaneously at various sites. Protecting access to your organization’s technology assets.
and the subsequent ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 42 . Storefront . Supply Chain Management focuses on globalization and information management tools which integrate procurement. That is why ERP is often referred to as back-office software. maintaining inventories. ii. once institutionalized. transformation of these materials into intermediate and finished products. customer service. iii. i. B.The software you use to build and manage your online store is critical to the overall success of your e-commerce venture. A. Knowledge management involves the identification and analysis of available and required knowledge assets and knowledge asset related processes. and tracking orders. A new business strategy built around demand and trust. The umbrella under which products are ordered. It is the information technology available to "enable" business transactions electronically H. including product planning. A. ii. parts purchasing. or other corporate media. Brochure ware . iii. and logistics from raw materials to customer satisfaction. G. Understanding how to build a partnership with your most valuable customers. interacting with suppliers. flexible payment options and clear confirmation that their order has been received. although the complexity of the chain may vary greatly from industry to industry and firm to firm. marketing. Customer Relationship Management(CRM). C. This is an accurate definition.A website that is little more than a corporate brochure. E-Business. but doesn't give us much insight into the excitement surrounding eBusiness and eCommerce. internal customers and suppliers. Enterprise Resource Planning (ERP). Knowledge Management(KM). iii. eBusiness is an interaction with business partners. D. ERP attempts to integrate all departments and functions across a company onto a single computer system that can serve all those different departments' particular needs. Supply Chain Management (SCM). Your customers will want easy navigation of your product catalog. C. video. E. The process. B. Supply chains exist in both service and manufacturing organizations. process and technology to maximize all of your relationships – with your day-to-day customers. A. all the modern features of a shopping cart system.ii. conveys knowledge embedded in its users to others in the organization. Customer Relationship Management (CRM) is the seamless coordination between sales. operations. B. a simple check-out process. and the distribution of these finished products to customers. ERP (Enterprise resource planning) is an industry term for the broad set of activities supported by multi-module application software that helps a manufacturer or other business manage the important parts of its business. and delivered. ERP's best hope for demonstrating value is as a sort of battering ram for improving the way your company takes a customer order and processes it into an invoice and revenue—otherwise known as the order fulfillment process. The right CRM strategy integrates people. distribution channel partners. i. created. where the interaction is enabled by information technology. field support and other functions that touch your customer. providing customer service. A supply chain is a network of facilities and distribution options that performs the functions of procurement of materials.
however. B. planning and control of actions to develop both the assets and the processes so as to fulfil organisational objectives. The ASP also employs the people needed to maintain the application." D. B. Outsourcing. The terms "ASP" and "Application Service Provider" are applied specifically to companies that provide services via the Internet.A collection of data designed to support management decision making. Data Warehousing (DW). Knowledge Management caters to the critical issues of organizational adaption. In many cases. B. and the creative and innovative capacity of human beings. In most cases. The ASP bills for the application either on a per-use basis or on a monthly/annual fee basis. this interpretation explicitly addresses the strategic distinction between knowledge and information explained earlier. an ASP is a service provider whose specialization is the implementation and ongoing operations management of one or more networked applications on behalf of its customer. One key attribute beginning to rapidly evolve is the emphasis on Webbased e-business application management as an important differentiator from the more traditional outsourced client-server application management services. C. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 43 . survival and competence in face of increasingly discontinuous environmental change. the term ASP has come to denote companies that supply software applications and/or software-related services over the Internet. D. A. Development of a data warehouse includes development of systems to extract data from operating systems plus installation of a warehouse database system that provides managers flexible access to the data. unlike most prevailing definitions. it embodies organizational processes that seek synergistic combination of data and information processing capacity of information technologies. Unlike most conceptions of knowledge management proposed in information systems research and in trade press. the ASP can provide the service for free or will even pay the customer. Here are the most common features of an ASP: A.. C.. either in a browser or through some sort of "thin client. managing. The term data warehousing generally refers to combine many different databases across an entire enterprise H. i. A contractual service offering hosting. The ASP owns. (Information Strategy: The Executive's Journal) F. operates and maintains the servers that run the application. this conception is better related to the new model of business strategy discussed earlier. and Asian Strategy Leadership Institute Review). Essentially." (Journal for Quality & Participation.C. Data warehouses contain a wide variety of data that present a coherent picture of business conditions at a single point in time. Application Service Providers (ASP). The ASP makes the application available to customers everywhere via the Internet. A repository of historical data used to make decisions. A. and access to an application (that is commercially available) from a centrally managed facility.. Furthermore. Hewlett-Packard Executive Intelligence. Simply stated. The ASP owns and operates a software application. Data Warehousing . G. Its primary focus is on How can knowledge management enable business strategy for the new world of business? and What strategic outcomes should knowledge management try to achieve? rather than What goes into the nuts and bolts of the machinery that supports knowledge management? It relates more closely to the dynamic view of business strategy as driver of the corporate information strategy.
____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 44 . The primary purpose of a walkthrough is to identify defects in the product as early in the Systems Life Cycle as possible. and psychology of the technique. Quality control is a component of internal control. An opportunity to monitor adherence to standards. Planned and formal technique used to verify compliance of specific development products against their documented standards and requirements. iii. Verification and Validation 1. Verification Methods A. E. B. This category will test the candidates understanding of quality control principles and methods. Knowledge should cover principles. Knowledge of the methods and procedures used by management to provide direction to their staff. Additional benefits of conducting walkthroughs (actually significant benefits in our situation) are: A. Walkthroughs i. processes and applications for access via a virtual private network or an Internet-based browser. D. The technique.The act of hiring an outside source. A basic understanding of typical manual and automated controls within an information system designed to ensure data integrity. and reporting on the operation of the information system function. System of Internal Control 1. The earlier a defect is identified. Developing a process to solicit service providers (Request For Proposal . B. ii. and processes to manage and control expectations and status. and the easier it is to take corrective action. Knowledge of the subset of management controls focused on assuring a completed project meets the user’s true needs E. 6. ii. ranging from informal peer reviews to structured reviews for the purpose of early error detection for removal. an independent group or person. security. structured and easy to maintain products. rationale. and systems performance. The producer of the product or service. B. Quality control should occur both during the build of a product or service and after completion. i. and/or the customer. Application Controls A. accounting. Inspections i. Quality Control A. Ensure readable.A. staff. Outsourcing . D. and to ensure through governance. Management Controls A. is typically used for design products and untested code. Internal System Controls A. Quality control comprises all methods employed to detect the presence of defects. the less costly it is to correct. the process for selecting a service provider. Knowledge of how software applications are controlled. C. In Process Reviews A.RFP). can perform quality control. process integrity. A method for disseminating new concepts and conventions. usually a consultant or application service provider. Quality Control Practices A. A method for improving group communication. financial integrity. A method for exposing people to new areas of the system. iii. to transfer components or large segments of an organization's internal IT structure. rules.
____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 45 . Savings result from early detection and correction avoiding the increased cost that comes with the detection and correction of defects later in the life cycle. iv. Although peer reviews are part of achieving CMM level 3. Software Inspections were introduced in the 1970s at IBM. 4-5 people participate and expend 1-2 hours of preparation and 1-2 hours of conduct each. This cost of 10 to 20 hours of total effort per session results in the early detection of 5-10 defects in 250-500 lines of new development code or 1000-1500 lines of legacy code. the population of Software Inspections adopters ranges from level 1 to 5. What is the impact of changing a requirement? C. Is this requirement necessary? D. and roles of participants. Formal code inspections are one of the most powerful techniques available for improving code quality. and preventing their leakage into field operations. and maintainability. Are we done? L. What acceptance test will be used to verify a requirement? K. and traceability of the requirements of a system. The cost of performing Software Inspections includes the individual preparation effort of each participant before the session and the conduct effort of participants in the inspections session.ii. Is this design element necessary? I. Code inspections are even more useful when inspectors hunt for specific errors rather than casually browse for bugs. availability. Code inspections -. vii. Are all requirements allocated? G. Software Inspections provide value in improving software reliability. a Key Process Area (kpa) of the CMM. Where is a requirement implemented? C. ii. Requirements Tracing i. viii. Typically. Software Inspections are a rigorous form of peer reviews. How do I interpret this requirement? E. Why is the design implemented this way and what were the other alternatives? H. consistency. which pioneered their early adoption and later evolution. Software Inspections are a disciplined engineering practice for detecting and correcting defects in software artifacts. The development and use of Requirements Tracing techniques originated in the early 1970s to influence the completeness. The Return on Investment for Software Inspections is defined as net savings divided by detection cost. structure (including rules). and many organizations limit their software process improvement agenda to the kpas for the maturity level they are seeking to achieve. v. What mission need is addressed by a requirement? B. The detection cost is the cost of preparation effort and the cost of conduct effort. Is the implementation compliant with the requirements? J. They provide an answer to the following questions: A. Methods to ensure that requirements are implemented correctly during each software development life cycle phase. Knowledge should cover purpose. vi. iii. The net savings then are up to nine times for major defects and up to three times for minor defects. An undetected major defect that escapes detection and leaks to the next phase may cost two to ten times to detect and correct. A minor defect may cost two to four times to detect and correct.peers reviewing code for bugs -complement testing because they tend to find different mistakes than testing does. What design decisions affect the implementation of a requirement? F. The goal is to identify and remove bugs before testing the code.
or kill points. assumption-based truth maintenance networks. the completion of each phase of development. These phase-end reviews are often called phase exits. or indexing of requirements. Cross referencing . design. and the phases typically take their names from these items: requirements. and others as appropriate. are usually a part of these reviews. progress. structural. and regression.iii. Knowledge of the various techniques used in testing. Incremental (top-down and bottom-up). The majority of these items are related to the primary phase deliverable. A. WHITE BOX . Testing techniques i. text. design. A software testing technique whereby explicit knowledge of the internal workings of the item being tested are used to select the test data. clear box and open box testing. to its subsequent deployment and use.g.e.. in both a forward and backward direction (i. white box testing uses specific knowledge of programming code to examine outputs.Also known as glass box. and all visible code must also be readable. There is no required preparation on the part of the audience and limited participation. and propagation). A walkthrough gives assurance that no major oversight lies concealed in the material. The test is accurate only if the tester knows what the program is supposed to do. B. constraint networks.Black Box testing (Functional testing) attempts to find discrepancies between the program and the user’s ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 46 . Requirements traceability is defined as the ability to describe and follow the life of a requirement. Phase-End Reviews A.. B. White box testing does not account for errors caused by omission. Unlike black box testing. Review of products and the processes used to develop or maintain systems occurring at. through its development and specification. WALKTHROUGH . BLACK BOX . turnover. a... He or she can then see if the program diverges from its intended goal.g. such as human (walkthroughs/inspections). or near. C. start-up. e. black box (data driven). and through periods of ongoing refinement and iteration in any of these phases).. stage gates. Test Concepts A. from its origins. and methods for designing and conducting.A presentation of developed material to an audience with a broad cross-section of knowledge about material being presented. Restructuring . B. Decisions to proceed with development. c. At the end of a project they are commonly called “Post Mortems Review” 2. Should cover topics such as purpose. programming.The documentation is restructured in terms of an underlying network or graph to keep track of requirements changes (e. tagging. and specialized tables or matrices that track the cross references). chaining mechanisms. risk. numbering. Specialized templates and integration or transformation documents These are used to store links between documents created in different phases of development. white box (logic driven).This involves embedding phrases like "see section x" throughout the project documentation (e. Each project phase normally includes a set of defined work products designed to establish the desired level of management control. A formal written report of the findings and recommendations is normally provided. based on cost. etc. C. b. build.g. schedule. Validation Methods A.
Testing every possible input stream is unrealistic because it would take a inordinate amount of time. Also known as functional testing. D. Unit tests tell a developer that the code is doing things right. The tester does not need knowledge of any specific programming languages. and its outputs are verified for conformance to specified behavior. functional tests tell a developer that the code is doing the right things. string. are termed regression tests. The test cases are difficult to design. B. They ensure that a particular method of a class successfully performs a set of specific tasks. It is ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 47 . Unit Testing may be defined as the verification and validation of an individual module or 'unit' of software. Each test confirms that a method produces the expected output when given a known input. many program paths will go untested. Unit tests are written from a programmer's perspective. A. iv. and controls) and recovery. either because the initial execution did not proceed successfully to its conclusion or because a flaw was discovered in the system or subsystem being tested. C. B. The tester does not ever examine the programming code and does not need any further knowledge of the program other than its specifications. The test is unbiased because the designer and the tester are independent of each other. system. c. whether or not successful. a. A test of an application software unit. iii. if any. Testing is done from the user's point-of-view. b. The advantages of this type of testing include: i. For example. Unit Testing .An essential aspect of unit testing is to test one feature at time. performance (subsets include volume and stress. A software testing technique whereby the internal workings of the item being tested are not known by the tester. Testing methods i. The test can be redundant if the software designer has already run a test case. The test is done from the point of view of the user. The disadvantages of this type of testing include: i. The first execution of a planned test. not the designer.d. security. therefore. description of what the program should do.It may be necessary to execute a planned acceptance. REGRESSION ACCEPTANCE TEST . It subjects the program or system to inputs. Test cases can be designed as soon as the specifications are complete. or unit test more than once. Software users are concerned with functionality and features of the system. Subsequent executions. iii. is termed an initial test. Methods of types such as unit/program. integration. ii. in a black box test on a software design the tester only knows the inputs and what the expected outcomes should be and not how the program arrives at those outputs. ii. A.
Determines the behavior of a system with expanded workloads simulating future production states such as added data and an increased amount of users. These include scroll bars. Unit testing may require developing test driver modules or test harnesses. server requests. Scalability Testing . thus providing additional insight and advantage. etc. A. A. B. Stress Testing . Security Testing . F. allows an organization to view its network the same way an attacker would. push buttons. upper and lower boundaries. Also. Penetration Testing D. C. Log Review G. Functional Tests . unit testing often requires detailed knowledge of the internal program design. FunctionalTests test the entire system from end to end. Network Mapping B. Virus Detection ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 48 . Volume Testing . A. etc B. which test the behavior of a single class. E. These tests confirm that the system does what users are expecting it to.written from a user's perspective. a company can identify problematic parts of a Web application before it is accessed by hundreds or thousands of users. The purpose of Volume Testing is to find weaknesses in the system with respect to its handling of large amounts of data.Determines the breaking point or unacceptable performance point of a system to reveal the maximum service level it can achieve. Password Cracking F. In addition. using the techniques recommended in this report. “Routine” Unit Testing includes identifying all fields and testing for input.seeks to verify the physical and logical limits to a system's capacity and ascertain whether such limits are acceptable to meet the projected capacity of the application's required processing. links. A load test simulates user activity and analyzes the effect of the real-world user environment on an application.the most "micro" scale of testing for testing particular functions or code modules. E. testing. D. Integrity Checkers H.Determines the response time of a system with various workloads within the anticipated normal production range. output. G. Vulnerability Scanning C. All standard GUI elements should be identified and validated. By load testing a Web application throughout development. The following types of security testing: A.The primary reason for testing a system is to identify potential vulnerabilities and subsequently repair them A. Security Test & Evaluation E. Unlike UnitTests. Testing allows an organization to accurately assess their system’s security posture. as well as calculations when appropriate. Load Testing .
iii. or component to verify thatmodifications have not caused unintended effects and that the item. iii. performance. of course. ii. delivered by the development team.I. The approach of using personnel not involved in the development of the product or system in its testing. The functionality. C. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 49 . ii.discovered. System testing specifically goes after behaviors and bugs that are properties of the entire system as distinct from properties attributable to components (unless. regression testing is initiated after a programmer has attempted to fix a recognized problem or has added source code to a program that may have inadvertently introduced errors. D. ii. All will provide a report on the . the component in question is the entire system). system. This process is called war dialing. and systems have been properly designed and implemented. recovery. War Dialing A. v. Regression testing i. This is important because after all the units are tested individually we need to ensure that they are tested progressively. Selective testing of an item. The software is of high quality. transaction synchronization bugs (often misnamed "timing bugs"). System Test i. The selective retesting of a software system that has been modified to ensure that any bugs have been fixed and that no other previously-working functions have failed as a result of the reparations and that newly added features have not created problems with previous versions of the software. Testing that is focused on an entire end-to-end business process. There are several software packages available (see Appendix C) that allow hackers and network administrators to dial large blocks of phone numbers in search of available modems. The simplest definition of Integration Testing that I could find states that "An integration test verifies that all the parts of an application "Integrate" together or work as expected together". subsystems. iii. It is a quality control measure to ensure that the newly-modified code still complies with its specified requirements and that unmodified code has not been affected by the maintenance activity. iv.000 numbers in a matter of days. the software will replace/support the intended business functions and achieves the standards required by the company for the development of new systems. Examples of system testing issues: resource loss bugs. Independent i. Verification that current changes have not adversely affected previous functionality. modules. Certain war dialers will even attempt some limited automatic hacking when a modem is discovered. Integration Test i. E. or componentcomplies with its specified requirements. security. system. F. Test which verifies that interfaces and interdependencies of products. A test of an entire application software system conducted to ensure that the system meets all applicable user and design requirements. numbers with modems. Also referred to as verification testing. is as specified by the business in the Business Design Specification Document and the Requirements Documentation. The software delivered interfaces correctly with existing systems. A computer with four modems can dial 10. throughput bugs.
operability tests. and size and users demand new functionality and features the need to perform Integration Test becomes more obvious. the user is involved in validating the acceptability of the system against acceptance criteria using the operational test environment. and. In cases where an acceptance test is conducted. at the sponsor’s option. v. Upon completion of all units a complete "System Test" is performed to ensure that data 'flows' from the beginning to the end of the Application. This test phase verifies compliance with the system design objectives and tests each module/program/system against the functional specifications using the system test environment. An Integration Test will thus allow "flaws" in the application between different Objects. Components. environmental impact on the operating systems. ACCEPTANCE TESTING . disaster recovery tests. if applicable. the training plan. an acceptance test may or may not be conducted. ii. Establishing the test in the operational environment requires coordination between the System Developer and the Information Processing Centers and is used to validate any additional impacts to the operating environment. stress testing. (SW-CMM (IEEE-STD-610)) SOFTWARE ACCEPTANCE TEST (SAT) . The completion of the SAT should result in the formal signing of a document accepting the software and establishes a new baseline. If the problem were to discovered in a system test at the end of the Development cycle it would probably require more resources to correct than during the cycle. and Modules etc to be uncovered while the Application is still being developed and the developers are still conceivably working in the same portion of the application. security and control tests. Testing of the system to demonstrate system compliance with user requirements. Often there is a deadline that drives businesses to develop new applications. a volume test. G. One of the ways that the QA team contributes to the project is to perform Integration Tests on the various units as they are developed.iv. Planning (Test Plan) ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 50 . and security. iv. and in an effort to preempt the market the time for Development and of course testing is generally shortened as the project matures. it is not conducted in lieu of a system test but in addition to a system test. SOFTWARE QUALIFICATION TEST i. But as an application increases in complexity. Test Program Development A.The Software Acceptance Test is used to test effectiveness of the documentation. The QA Team does not have to wait for the entire system to be completed before Testing is implemented but can take the various units after they have been developed and ensure that they function correctly together. Many individuals use the terms System Testing and Integration Testing interchangeably and for simple applications that do not have many components the criteria and test scripts required to perform testing are similar. The SQT should include a performance test. iii. System Acceptance i. B. In this test phase. For a given release of an application software system.A test of an application software system that is performed for the purpose of enabling the system sponsor to decide whether or not to accept the system. This is especially important in today's market where the drive is to be the first to market a product. Formal testing conducted to determine whether a system satisfies its acceptance criteria and to enable the customer to determine whether to accept the system. a data conversion test H.
Performance/Load Testing D. Analysis techniques used to evaluate results of testing. execution conditions. A distinction must also be made between manual and automated test scripts. Cases i. customer. Test cases describe what you want to test. includes planning for regression testing.i. and validation of cases. they are software testing software. iii. What is the difference between Test Cases and Test Scripts? A. A test case typically describes detailed test conditions that are designed to produce an expected result.Defines the procedures to be followed when applying a test suite to a product for the purposes of conformance testing. System Testing C. iii. (SW-CMM (IEEE-STD-610)) “Acceptance Criteria" means the written technical and operational performance and functional criteria and documentation standards set out in the project or test plan. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 51 . ii. Development of test data. Test Procedure . User Acceptance Testing The criteria that a system or component must satisfy in order to be accepted by a user. and expected results that determine whether the software being tested meets functional requirements. In test automation. In addition to the master test plan. Development of test objective (cases). execution. ii. E. as well as level-specific test plans which establish protocol for each required level of testing. D. Procedures i. lists items to be tested and serves as a communication device between all members of the project team. You can also carry this concept to manual test scripts by keeping the test script free of specific test data Development. For many programs. Acceptance Criteria i. These automated test scripts or procedures closely resemble source code. including techniques. Automated test scripts are sometimes referred to as test procedures. A specific set of test data and associated inputs. ii. iii. In the chaining approach. Test scripts also contain expected results. ii. ii. Many large projects require a Master Test Plan which establishes the test management process for the overall project. or other authorized entity. a test script may be placed in a loop and read many different test cases from test data files. A QA Team typically creates a test plan and uses it to guide the QA team's efforts. A test plan provides an overview of the project. Test scripts describe how to perform the test. C. Data i. Rule-based software test data generation is proposed as an alternative to either path/predicate analysis or random data generation. The chaining approach for automated software test data generation which builds on the current theory of execution-oriented test data generation. Selection of techniques and methods to be used to validate the product against its approved requirements. and evaluation of procedures used for testing. Unit Testing B. test data are derived based on the actual execution of the program under test. Tools related to generation of test data. these projects may include test plans for: A. The plan also identifies sufficient and proper tests to assure that previously tested related functions will execute properly. In fact. but usually in more general terms. the execution of the selected B. approaches for verification.
This is the classic tradeoff between test scripts and test cases.. i. H. Creation of test specifications. This can also be its greatest curse. E. A good software specification document can: A. Scripts i. The experiments have shown that the chaining approach may significantly improve the chances of finding test data as compared to the existing methods of automated test data generation. By dividing the full feature set into useful subsets. Knowledge should cover purpose. and what subsets of features comprise a useful solution. The decision to use test cases versus test scripts depends on: A. F. The importance of documenting a test of a specified sequence. G. The chaining approach uses these statements to form a sequence of statements that is to be executed prior to the execution of the selected statement. iv. Focus should be on the purpose and preparation. It describes the features the new product should have. so the design can allow for them. preparation. Test scripts describe how to perform the test. The test case specifications should be developed from the test plan and are the second phase of the test development life cycle. C. C. and usage. B.e. iii. Specifications i. ii. B.statement may require prior execution of some other statements. Reduce the time needed to complete the project by determining the usability of the system and providing the customers with a realistic expectation of what the system will do — before it is built. you can better plan a staged delivery that will test your assumptions and validate your design. Documentation of the steps to be performed in testing. The existing methods of test data generation may not efficiently generate test data for these types of programs because they only use control flow information of a program during the search process. Tools and methods to access test results. The intensity of the test. The test specification should explain "how" to implement the test cases described in the test plan. A software specification document is crucial for a successful project. and confirming these subsets. data dependence analysis automatically identifies statements that affect the execution of the selected statement. D. you want to know in advance the conditions to be tested and how they should behave. At the same time. iii. Sometimes you want the randomness of user actions. The main benefit of a test script is that it predefines a procedure to follow in performing a test. ii. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 52 . Improve customer satisfaction since their expectations are met or exceeded. The level of predictability of how a user will interact with a web interface. The degree of freedom a user is intended to have in interacting with a web interface. D. The chaining approach uses data dependence analysis to guide the search process. A well-understood specification reduces unplanned features and informs developers where future features will be needed. The importance of sequence in the user’s correct performance of a task. Determine what features are most important. Analysis Techniques i.
describe: 1) Its impact on system performance. The entire boolean expression is considered one true-or-false predicate regardless of whether it contains logical-and or logical-or operators. including identification of requirements not met 2) The impact on system design to correct it 3) A recommended solution/approach for correcting it. Likewise. vi. Here is a description of some fundamental measures: A. Functional testing examines what the program accomplishes. or testing of the system tested. or constraint. Structural testing compares test program behavior against the apparent intention of the source code. methods.ii. operation. Data flowanalysis provides interesting information about the structure of the code that can be used for deducing static properties of the code and for deriving coverage information. limitations. B. branch or path level. which compares test program behavior against a requirements specification. taking into account possible pitfalls in the structure and logic. iii. c. Structural testing is also called path testing since you choose test cases that cause paths to be taken through the structure of the program. separated by logical-and ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 53 . A large variety of coverage measures exist. C. or constraints that were detected by the testing performed. and test coverage tools used for monitoring the execution of software and reporting on the degree of coverage at the statement. Structural testing examines how the program works. Knowledge of purpose. Additionally. this measure includes coverage of switch-statement cases. The academic world more often uses the term "test coverage" while practitioners more often use "code coverage". v. iii. Management Summary B. Code coverage analysis is a structural testing technique (AKA glass box testing and white box testing). Code coverage analysis is sometimes called test coverage analysis. The Test Results Analysis Report is an analysis of the results of running tests. iv. a coverage analyzer is sometimes called a coverage monitor. and interrupt handlers. Test Logs/Traces Identify any remaining (open) deficiencies. If no recommendation is provided state. ii. v.Condition coverage reports the true or false outcome of each boolean sub-expression. Test Results Analysis C. This contrasts with functional testing (AKA black-box testing). Provide any recommended improvements in the design. The results analysis provide management and the development team with a readout of the product quality. Decision Coverage . The two terms are synonymous. iv.This measure reports whether each executable statement is encountered. For each remaining (open) deficiency. limitation. C. exception handlers.The following sections should be included in the results analysis: A. Describe the impact on the system. without regard to how it works internally. for each recommendation. Problem/change reports may be used to provide deficiency information. Statement Coverage . Condition Coverage . Code coverage i.This measure reports whether boolean expressions tested in control structures (such as the if-statement and while-statement) evaluated to both true and false. Provide an assessment of the manner in which the test environment may be different from the operational environment and the effect of this difference on the test results. Test Completion Criteria A. indicate: None. Do not confuse path testing with the path coverage measure.
Also known as MC/DC and MCDC.This measure reports whether you invoked each function or procedure. G. The hypothesis is that faults commonly occur in interfaces between modules. Condition coverage measures the subexpressions independently of each other. Multiple Condition Coverage . Here is a description of some variations of the fundamental measures and some less commonly use measures: A. as well as most other measures. This "linear" sequence can contain decisions as long as the control flow actually continues from one line to the next at run-time. Also known as predicate coverage.This measure reports whether you executed each function call. Predicate coverage views paths as possible combinations of logical conditions. BullseyeCoverage measures function coverage.Condition/Decision Coverage is a hybrid measure composed by the union of condition coverage and decision coverage. without requiring a flow graph. Sub-paths are constructed by concatenating LCSAJs. when present. Path Coverage . C. E.This measure reports whether each of the possible paths in each function have been followed. As with condition coverage.Multiple condition coverage reports whether every possible combination of boolean sub-expressions occurs. F. It is useful during preliminary testing to assure at least some coverage in all areas of the software. The test cases required for full multiple condition coverage of a condition are given by the logical operator truth table for the condition. Predicate coverage includes path coverage and multiple condition coverage.This variation of path coverage considers only the sub-paths from variable assignments to subsequent references of the variables. Condition/Decision Coverage . The disadvantage is that it does not avoid infeasible paths.vi. The advantage of this measure is the paths reported have direct relevance to the way the program handles data. B. This measure was created at Boeing and is required for aviation software by RCTA/DO-178B. D. H. One disadvantage is that this measure does not include decision coverage. This measure requires enough test cases to verify every condition can affect the result of its encompassing decision. Path coverage includes decision coverage. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 54 . A. and logical-or if they occur. Also known as call pair coverage C. An LCSAJ is a sequence of source code lines executed in sequence. Linear Code Sequence and Jump (LCSAJ) Coverage . Call Coverage . B.This variation of path coverage considers only sub-paths that can easily be represented in the program source code. Condition/decision coverage includes decision coverage and condition coverage (by definition). Decision coverage includes statement coverage since exercising every branch must lead to exercising every statement. Data Flow Coverage . You can compare relative strengths when a stronger measure includes a weaker measure. the sub-expressions are separated by logical-and and logical-or. Researchers refer to the coverage ratio of paths of length n LCSAJs as the test effectiveness ratio (TER) n+2. D. Function Coverage . Broad. The advantage of this measure is that it is more thorough than decision coverage yet avoids the exponential difficulty of path coverage. shallow testing finds gross deficiencies in a test suite quickly. D. A path is a unique sequence of branches from the function entry to the exit. Modified Condition/Decision Coverage .
This measure reports whether each machine language conditional branch instruction both took the branch and fell through. Knowledge of risk assessment and risk abatement techniques used in the testing process. such as "-" substituted for "+". and communication of risks in software-intensive programs.This measure reports whether boundary situations occur with relational operators (<. Race Coverage . B. Loop Coverage . narrowing the focus on particular aspects of risk. For example. and more than once. >. exactly once. and with alternate variables substituted. The Software Risk Evaluation (SRE) is a service that helps projects establish an initial baseline set of risks and mitigation plans¾ one of the key first steps for putting risk management in place. loop coverage reports whether you executed the body exactly once. process. The program's own personnel participate in the identification. Object Code Branch Coverage . For do-while loops. The hypothesis is that boundary test cases find off-by-one errors and mistaken uses of wrong relational operators such as < instead of <=. analysis. The Taxonomy-Based Questionnaire is structured into three main areas of software risk: Product Engineering. For example. information not reported by others measure.This measure reports whether you executed each loop body zero times. Risk i.This measure reports whether multiple threads execute the same code at the same time. It helps detect failure to synchronize access to resources. It is useful for testing multi-threaded programs such as in an operating system. Each of these categories is subdivided further. G. E. The valuable aspect of this measure is determining whether while-loops and for-loops execute more than once. pointers also present problems. Researchers have proposed numerous variations. It reports whether test cases occur which would expose the use of wrong operators and also wrong operands. and Program Constraints. Requirements. iii. Scale) asks: "Is the system size and complexity a concern?". Weak Mutation Coverage . and mitigation of risks facing their own development effort. analysis. Development Environment. and more than once (consecutively). H. An SRE is used to identify and categorize specific program risks emanating from product. and constraints. all of which increase the complexity of this measure. I. It works by reporting coverage of conditions derived by substituting (mutating) the program's expressions with alternate operators. As with data flow analysis for code optimization. The SEI Software Risk Evaluation (SRE) Service is a diagnostic and decision-making tool that enables the identification. mitigation. variations distinguish between the use of a variable in a computation versus a use in a decision. the thirteenth question on the Questionnaire (Product Engineering. Risks to a software project must first be identified.This measure is similar to relational operator coverage but much more general. One way of identifying software project risks is using a questionnaire such as the SEI TaxonomyBased Risk Identification Questionnaire. This measure gives results that depend on the compiler rather than on the program structure since compiler code generation and optimization techniques can create object code that bears little similarity to the original source code structure. ii. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 55 .Another disadvantage is complexity. resources. and between local and global variables. >=). tracking. <=. management. Relational Operator Coverage . F.
the process must continue. Schedule risk Risk Identification What key technical area or process is at risk? Risk Analysis Determine the root cause of the risk. Error rate i. A detailed plan of how the risk will be prevented and/or mitigated is created. Risk Communication Provide status of the risks on a regular basis. conformance. Process improvement focuses on building the product right. Next. the contingency plan is put in effect. v. a plan should be formulated to address each risk. Who is responsible for the Risk Management activity? D. and tools for managing risks in a project. and if the trigger is reached. A trigger should be set up. iv. ix. managing change. vi. Specify why the risk is important? B. The risk must be analyzed After analyzing software risks. Cost risk E. The five risk areas are: A. determine which risks are important to deal with C. and schedule can be further segmented into five risk areas. What information is needed to track the status of the risk? C. Programmatic risk (environment related) D. F. assess continuously what could go wrong (risks) B. x. Risk management focuses on building the right product. C. Understanding of mean time between errors as a criterion for test completion. methods. any given risk may have an impact in more than one area. Risk management and process improvement are complementary. It is a Process Area (PA) at Maturity Level 3 in the CMM IntegrationSM (CMMISM) staged model. Planning stages should cover the following: A. Action planning can be used to mitigate the risk via an immediate response. The cumulative average time that a manufacturer estimates between failures or occurrences in a component. Technical risk (performance related) B. iii.iv. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 56 . xi. The reliability of a system is measured by the probability of errors. managing variability. Risk Abatement What can you do about a risk? Identify possible solutions. innovation. However. The probability of the risk occurring. develop a mitigation/contingency plan or accept the risk. viii. Once risks have been identified in some manner. cost. and/or the potential impact of the risk may be mitigated by dealing with the problem early in the project. vii. Quantify your risks by determining the likelihood of an event and the potential consequence to the ISS. Risk Management is a practice with processes. project performance. Contingency planning can be used to monitor the risk and invoke a predetermined response. activity improvement. Reliability. and uncertainty. It provides a disciplined environment for proactive decision making to A. and control Software Risk factors that impact a product’s performance. Supportability risk (performance related) C. implement strategies to deal with those risks Risk management is currently a key process area (KPA) in the Systems Engineering CMM® and the Software Acquisition CMM. expressed in MTBE (Mean Time Between Errors) and availability expressed as MTBF (mean time between failures). What resources are needed to perform the activity? E. G. ii.
Change control over the requirements. vi. track and communicate issues and standardize the software build process. and PVCS Configuration Builder for automated software builds. closed. and guidelines). implementation and operations i. Also.v. hardware and operating system (support) software change control. PVCS Professional combines PVCS Version Manager. Software Change Control 1. etc.e. Project. evaluated. Example Tool: a. F. As PVCS Professional facilitates communication.. and manage workflow tasks involved in team collaboration. A. Automates software builds for standardized. standards. and tracked. PVCS Professional enables organizations to protect development assets. Applications should be specified. Where possible. plans. PVCS Tracker and PVCS Configuration Builder in an integrated suite for software configuration management. A complete package for software configuration management. New software and changes to existing software should be prepared in the Development Environment by appropriately authorized development or applications support staff. ii. Mean Time Between Failures . design. Change Control Environment a. Once assessed as satisfactory. coordinates tasks and manages changes. production b. Change control procedures cover the establishment of methods for identifying. Changes to software are not permitted in the testing environment. c. Professional combines PVCS Version Manager. Tracks and communicates issues across the enterprise iii. repeatable development success.The "down-time" during which the managed application is unavailable due to failure. of a particular software project. PVCS Tracker. development ii. Merant PVCS Professional gives teams the power to organize and manage software assets. Migration of software between environments should only be undertaken after obtaining the appropriate sign-offs as specified in the Software Change Control Procedures. 2.. code. designed and coded according to systems development methodology. 1. Software change control is both a managerial as well as technical activityand is essential for proper software quality control. scheduled. the new or modified software should be transferred to the Testing Environment for systems and acceptance testing by an appropriate testing group. testing iii. The process by which a software change is proposed. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 57 . i. automate software configuration management tasks. 2. At the project level these activities should be included as part of the project plan or in a software change control plan. teams can speed time to market with efficient parallel development. procedures. 1. documentation. more code re-use and fewer errors. storing and changing system items that pass through development. Organizes. Environment. according to an agreed test procedure. Change control over the project management environment that projects function within (i. approved or rejected. manages and protects software assets ii. implemented. integration. Reliability is a measure that indicates the probability of faults or the mean time between errors. three separate environments should be maintained for each strategic system : i. d.
Defect Recording. Maybe version management is more appropriate. In general. 1. software developers have a generally agreed code of practice with regard to software versioning. Difference reporting provides quick access to changes across separate versions of the same file.NET. d. 3. Visual SourceSafe 6. such as labels. Historically. (version) 1. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 58 . the costs and overhead associated with file-based version control often outweigh the benefits. enabling a deeper degree of software reliability. including extensive deployment support. Team members can also reconcile conflicts between different versions of the same file by using a visual merge capability. the version number will be identified by two or three digits e. providing an audit trail for every file and every project. Visual SourceSafe can be used to create site maps and check hyperlinks. Knowledge of controlling multiple releases of configuration items. its second point release and its first mini release or patch. iii. Although not a global standard per se. c. By providing project-oriented software management. G. Java class/package or even complete projects has which functionality.e. Following successful completion of testing and approval by the appropriate systems custodian. 2. 1. the new or modified software should be transferred to the Production Environment for implementation under the control of IT Operations staff. b. Share and linking capabilities promote the reuse of code and components across projects and simplify code maintenance by propagating changes across all shared and linked files whenever a file is updated. Visual SourceSafe also provides many advanced features for Web site management. Parallel development features. Visual SourceSafe enables teams to develop with the confidence that their projects and files will be protected. Defect Management 1. which provides a point-and-click interface for uniting files and avoids potential loss of valuable changes. and the software life cycle grows. Version Control. A contingency plan to enable the software to be restored to its previous version in the event that the implementation is unsuccessful should be prepared where appropriate. creating identical copies that inherit all versioning documentation but may be tracked as new. In SVC you can specifacally tell which version of a program. Additionally. problems within the team development environment stem from the inability to work comfortably in a setting sensitive to their projects and source code. frequency.g. Corrective Action on Defects. a.1 This example indicates that the software is in its first major release. enable teams to fork the development process into parallel projects and files. While every project requires an adequate level of software management. module. enabling developers to know immediately what lines of code have changed. a. Defect Reporting and Tracking. As revisions are made. all changes and documentation are secured by Visual SourceSafe. easily accessible to even the novice user. Identification of the most common sources of information and the different methods. files are added and modified.0c is the ideal version control system for any development team using Microsoft Visual Studio® . such as branching. and types of reporting. individual projects.2. provide snapshots of a project for the quick retrieval of any previous version in the software life cycle. Tracking And Correction i. Versioning features.
Helps in improving processes IT management must accept the responsibility for nonconformance i. No longer the exclusive domain of the Information Systems department or even Information Systems Auditors. high tech has created new roles and responsibilities for everyone in the audit function. Customer is always right ii. standards and procedures must be developed. Quality is a binary state ii. Auditors must know what the new technologies are. recorded and measured i. Produce products and services on-time at the lowest possible cost i. delivered at any time. Quality must be defined quantitatively i. Meet Customer’s True Quality Needs i. IT Auditing Coordination A. IT policies. 2. g. rather than reactive. continually updated and followed. Cannot survive without customers b. Analyzing problem data and using problem-solving principles such as identification of problems. h. worldwide demand for knowledge. Requirements documents are defect prone iii. Quality at any cost. If Quality is not measured. 80% of all defects are directly attributable to ineffective processes The customer’s view of Quality is the correct view of Quality i. will not satisfy customers Create enthusiasm and cooperation between management and staff for quality i. political and social forces are creating an urgent. the risks and exposures involved and how they affect audit plans and the audit.i. it can not be controlled The goal of IT management and staff must be to produce defect-free products & services Non-conformance must be detected as early as possible. Identify customer’s true needs and update the requirements document. response to implementation of IT audit recommendations. B. Dynamic economic. and joint projects with the IT audit section of the organization’s internal auditing department. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 59 . i. if needed. Economic issue ii. In their broadened role they participate with management in strengthening the overall control framework. j. In an atmosphere like this it is critical that all auditors understand the impact of Information Systems on control and auditing. Today’s auditors must be fully integrated auditors. and defining/documenting and applying the appropriate solution. F. c. d. Everybody’s responsibility ii. Knowledge of IT audit function and how to effectively coordinate with work schedules. Today’s auditors are becoming more proactive and coactive. well documented. QAI Recommended Quality practices a. Everyone must ‘buy in’ into the quality principles & methods Reduce product inspections and testing by building processes that produce defect-free products. e. Uniqueness of information technology ii. establishment of applicable objectives. understanding information systems and able to function effectively within a technical environment. f.
Process development group i. b. c. Nonconforming and unpredictable -. b. a process is a system of causes: the people. and install process. and others.the ideal state B. and Improve Work Processes A. Define. The group method improves working practice and outcomes and is likely to contribute to far greater job satisfaction. Implementing a newly defined process is as complex and risk laden as defining the process and includes training. Conforming yet unpredictable -. The world is constantly changing. A process is defined as any set of conditions or set of causes that work together to produce a given result.the state of chaos iv. We measure the results and the ways in which they are delivered to determine quality. Tailoring processes i. Compliance and Enforcement a. Process development process i. d. Modifying existing processes to better match the needs of a project or environment. and procedures working together in a specified manner to produce an intended result. The identified group within the enterprise that has the responsibility and authority to identify. On-line standards i. Developing/Building Processes b. help to define process performance. across some or all of the enterprise. Maintenance of approved processes in an automated environment. or generate. Standards needs assessment i. a. equipment. Defining and continuously improving work processes enables you to maintain the pace of change without negatively impacting the quality of your products and services. e. materials. These characteristics. In other words. The purpose of a process is to produce results such as products or services. Conforming and predictable -. using on-line terminals rather than paper manuals. The methods for process mapping. Use statistical tools to indicate the degree to which a process is “in control. selection of processes to build. and timeliness of the products and services. Implementing a process i. cost. Measurements of process performance are used to evaluate the ability of a process to produce products or services with the characteristics we desire. acquire. Administering Processes a. The four possibilities for any Process: A. ii.g.the threshold state C.” a.the brink of chaos D. d. We must constantly improve our ability to produce quality products that add value to your customer base. energy. Build.. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 60 .7. quantity. The Process Development Group has bee developed by Richard Reynolds at Indigo Rose as a highly effective way of enabling people to be more productive working in a team. customers are more knowledgeable and demanding. Nonconforming and predictable -. and how to define a process and how to continuously improve process capability. Implement. Waiver b. c. and quality and speed of delivery are now critical needs. or for the elimination of obsolete or non-beneficial standards. b. but come from the other standardized portions of the enterprise to perform the standards function. Determination of the needs of the enterprise for new or modified standards. Process committee i. c. This category will test the candidate’s understanding of process components. The group provides a safe and supportive framework for employees at all levels to learn to communicate openly and to work constructively in any environment where the achievement of common goals is important. and the procedures to build processes. An implementation of the standards group in which the members are not part of a fixed group.
Statistical methods used to monitor process performance.e. on a near real-time basis so that the operator can take action on the process. The World Wide Web Consortium (W3C) develops interoperable technologies (specifications. 8. statutory regulations. project performance. Measurement . and visibility into. Standards developed outside the enterprise that may affect the operation. guidelines. products. process capability. B. and objectives. A.g. and collective understanding.. and tools) to lead the Web to its full potential.c. Probability and Statistics 1. This is done in order to ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 61 . W3C is a forum for information. In some cases. f. iii. SPC is a method of monitoring a process during its operation in order to control the quality of the products while they are being produced -. Sources i. within acceptable variance from standards) and to help identify the root cause of process problems that are causing defects. A properly established measurement system is used to help achieve missions. or opportunities of the enterprise (e. Acquiring externally developed standards and adapting them for beneficial use within the enterprise. and other public domain standards). customer satisfaction. Automated process enforcement i. product quality. i. Collecting measurement data on process performance in use of process improvement. Use of measures and other information allow organizations to learn from the past in order to improve performance and achieve better predictability over time. Acquisition and customization i. Locating potential process improvements. It involves gathering information about the product. c. effectiveness and efficiency of processes. and quality management at all levels. communication. Technical standards i. B. Statistics are used for both determining whether or not the processes under control (i. process. which may be useful to the enterprise. Measurement can be used to measure the status of processes. Knowledge of where to find standards developed outside the enterprise (e. FAA.rather than relying on inspection to find problems after the fact. and as a tool for management to use in their decision-making processes. industry standards and specifications. i. The method by which release from the requirements of a specific process may be obtained for a specific situation... What gets measured gets done. IEEE). Quantitative Methods A. and ISO/IEC 15504 b.provides objective information about. software. Establish process measures 1. e. 2. goals. Process Improvement Methods a. This category will test the candidate’s understanding of measures and how to build an effective measurement program.g. or the process itself. process performance. The Capability Maturity Model (CMM) certainly affirms this viewpoint and represents measurement practices as critical components of project. commerce. Measurement data is most reliable when it is generated as a by-product of producing a product or service. Externally Developed Standards a. The use of precompilers and other tools to detect noncompliance. ii. The quality assurance professional must ensure quantitative data is valid and reliable and presented to management in a timely and easy-to-use manner. evaluating them. Software development standards include: IEEE. and product and service quality. the tools can correct the noncompliance. and providing management with the information and techniques to introduce beneficial modifications to the process. DOD. Statistical process control (SPC).
Top managers make important decisions for companies and have the most influence on the future of the business. Assignable causes also can include factors too minor to bother about. as its name implies. D. Statistical Process Control. Assignable causes in converting operations could include a change in raw materials. It is hard to detect and reduce. the process may be subject to large and unusual changes in the cause system from time to time which result in non-random variation in the process performance. Top management. B. chemicals. Random variation is inherent in a system. Fluctuations in the process performance come from two sources." are occurring—to be tracked down and eliminated. physical and mental reactions of people. Today US businesses are in the process of implementing SPC to build quality into products and services. out-of-round rolls. by its methods of operation. Assignable variation is easy to detect and easier to reduce than random variation because its causes are known. equipment. On the other hand. then the process is in statistical control. Universally. or machine to machine. Dr. operator. has built defects into the process. Define Problem Proposal Pareto chart Implemen t Solution Analyse Data Collect Data Define Process List Possible Flow Chart Analyse Charts Brainstorm Ranking Fishbone Sampling D. W. Dr. Deming has created 14 points to serve as a guideline. Edwards Deming. Notice that improvement is a never-ending cycle. the presence of an impurity. To assist management. According to Dr. person to person. etc. procedure. bad bearings. etc. 80 percent of all quality problems are due to management. Once all assignable causes are removed from a process. due to random causes or chance. who had worked with Walter Shewhart.identify special causes of variation and other non-normal processing conditions. Deming. taught SPC to the Japanese after World War II. assignable causes can almost always be eliminated. businesses and industries use mathematics and statistical measurements to solve problems. b. Statistical methods used to differentiate normal variance in the operation of processes (random). Variation . Even a superior product or service can be improved on. These differences among products. a. Random variation occurs. C. C. environment. the so-called "assignable causes. or whether non-random deviations. Random causes rarely can be eliminated. Such variation is referred to as assignable or special cause variation since the variation is generally due to causes that could have been ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 62 . are called variation. from variances that are associated with the root cause (assignable). Assignable variation in the product performance occurs due to a change in machine setup. thus bringing the process under statistical control and reducing variation. a set point change. or other specific causes. is a tool that businesses and industries use to achieve quality in their products and/or services. Managers need to determine whether a production system is undergoing only random fluctuations in its operation. or the process output over time. SPC for short.) are responsible for random variation in the process performance and is referred to by Shewhart as common cause variation. The problem solving cycle shown below illustrates the process of continual improvement. Random and assignable causes A. poor calibration of the instruments. 2. This is not to say that one day management decided to make inferior goods. Fluctuations over time in the inherent process cause systems (differences in material.Differences exist from product to product. There is an increasing demand for managers and workers who understand and are able to apply Statistical Process Control methods.
the process is no longer predictable. Common cause variation of process performance is characterized by fluctuations that are random. coding. and overall product development c. perfective. a distribution function of some sort. Average values are plotted to monitor the process output using tools such as trend charts. design. The standard deviation is a measure of inconsistency in a process. C. When process performance is limited to common cause variation. process variation will decrease with future execution of the process and the process will become stable and predictable. the standard deviation. Characteristics of measures and methods i. Process mean. it will be within a distribution function and is therefore predictable. B. Measures and Metrics a. Standard deviation is the square root of the mean sum of the squares of the deviations from the mean. and status (open or closed) B. Shewhart's control charts are the primary vehicle used to analyze process performance variation. requirements specification. Standard deviation is widely used to quantify the variability of a process. A. as a statistical limit. or perhaps in terms of a design document).prevented. and vary within predictable bounds. Defects: count the number found by testing and by customers and their type. When assignable causes are removed. testing) and maintenance activities (adaptive. meaning that they can be derived from examination of the software itself (usually in the form of source or object code. severity. meaning that they can only be derived from observation of the execution of the softwareComputer scientists and software engineers have done a lot of research ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 63 . Standard deviation. Work effort distribution: record the time spent in development activities (project management. Some software measures can be characterizes as static. Normal distribution is characterized by two parameters. The definitions and concepts. number of requirements. The total variation that may be observed in process performance is expressed by the equation: Total variation = Common cause variation + assignable cause variation E. The control charts employ upper and lower control limits (UCL and LCL) to delineate or filter assignable cause variation from common cause variation. When variation in process performance includes assignable cause variation. The limits are empirically derived from measurements of the variation in the process performance over time 3. Type of software measurements: a. the observed distribution of the process performance variation tends to approach. or GUI elements b. object classes. Product size: count lines of code. project milestones. In the industrial environment. G.e. Other measures can be characterized as dynamic. corrective) d. F. B. mean is the value where a process is expected to operate or the target value. Statistical methods used to accumulate and analyze problems incurred as a result of operating processes. A. Estimated and actual duration (calendar time) and effort (labor hours): track for individual tasks. control charts. i. Problem characteristic analysis A.. Calculating probabilities using the normal distribution requires the estimate of the process mean. or pre-control charts. Assignable cause variation arises from events that are not part of the normal process. in statistical control or stable. the normal distribution is used to predict the probability of producing defective product. mean and standard deviation. and are due to sudden or persistent anomalies within one or more components of the cause system. function points. In manufacturing operations. When the cause system is constant. A process capability is defined as six times standard deviation.
a performance measure is a dynamic software measure. i. one measure alone is insufficient to measure the features of the design paradigm or to accomplish the objectives of the software project. The measure should be robust. and monitoring of projects • Better understanding of both the software development process and the development environment • Identification of areas of potential process improvement as well as an objective measure of the improvement efforts • Improved communication E. The commitment cannot be just a policy statement. assisted by more than 60 specialists from industry. Clearly. If a smaller value is better for one type of measure in the suite. which can be defined in several ways. staff-hour). or observer. identified four direct measures and several indirect measures that software engineering organizations can use to improve their software development processes. This includes allocating staff as well as tools. c. The measure relates to the product. The policy must be followed with the allocation of resources to the measurement program. There is a scale upon which we can make a comparison of two measures of the same type. staff-week. However. usability. that are reasonable. academia. and bounds. D. Often. and the process of collecting the data for the measure is objective. b. d. The calculation of the measure is repeatable and the result is insensitive to minor changes in environment. Software metrics are measurements made on a software artifact. Before examining the details of software measures (often called metrics). and government. There are a few other software properties that are generally believed to be important but which we don’t yet know how to measure very well. and so conclude which of the two measures is more desirable. a federally funded research and development center at Carnegie Mellon University. For example. Organizations with successful measurement programs report the following benefits: • Insight into product development • Capability to quantify tradeoff decisions • Better planning. man-month. Among these are reliability and complexity. The measure should be meaningful. Finally.trying to define the important measures of software engineering. then smaller is better for all other types of measures in the suite. control. This suggests that a collection or suite of measures is needed to provide the range and diversity necessary to achieve the software project's objectives. stability for use). and portibility. schedule. effort (labor-month. let's consider which properties of a measure. there is a realistic lower bound. A measure should have the following characteristics to be of value to us: a. in general. A measure is a numerical value computed from a collection of data. The properties or attributes of software that are directly measurable are size (source lines of code (SLOC)). it must be total commitment. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 64 . many of the potential benefits that an organization can derive from a sound measurement program is often not achieved due to a half-hearted commitment by managers to a measurement program. A suite of measures should be consistent. The measure is precise. These include maintainability. tool. Researchers at the SEI. C. there are other attributes of software that seem important but that we don’t know how to measure at all. The measure should suggest a norm. One of the most significant efforts was undertaken over the last four years at the Software Engineering Institute (SEI). and quality(freedom from defects. A suite of measures adds an additional consideration. e. such as zero for number of errors. scale. Another property whose measure is widely regarded as fundamentally important is performance. and there should be a rationale for collecting data for the measure.
F. Measure - n. A standard or unit of measurement; the extent, dimensions, capacity, etc.
of anything, especially as determined by a standard; an act or process of measuring; a result of measurement. v. To ascertain the quantity, mass, extent, or degree of something in terms of a standard unit or fixed amount, usually by means of an instrument or process; to compute the size of something from dimensional measurements; to estimate the extent, strength, worth, or character of something; to take measurements. Measurement - The act or process of measuring something. Also a result, such as a figure expressing the extent or value that is obtained by measuring. Techniques or methods that apply software measures to software engineering objects to achieve predefined goals. A measure is a mapping from a set of software engineering objects to a set of mathematical objects. Measurement goals vary with the software engineering object being measured, the purpose of measurement, who is interested in these measurements, which properties are being measured, and the environment in which measurement is being performed. Examples of measures include software size, Halstead's software science measures, and McCabe's cyclomatic complexity. Associated models include sizing models, cost models, and software reliability models. Data Definition Frameworks (DDF) are primarily used to define measurements as well to communicate more effectively what a set of measurements represent. Secondary DDF uses include assistance for: identifying issues that can be used to focus data analysis designing databases for storing measurement data developing data collection forms A DDF can be used to define a set of measurements. For example, a single DDF can be used to identify a line of code measurement, i.e., identify what is to be counted. A DDF can also be used to help communicate what has been counted. A DDF does this by allowing a user to identify specifically what was included and excluded in a measurement. For example, if I have a count of lines of code, say 317,300 lines of code. The DDF helps me communicate what that number represents by identifying what types of code were counted and included in that number and what types of code were specifically not counted, i.e., excluded.
b. Complexity measurements
i. Quantitative values accumulated by a predetermined method that measures the complexity of a software product, such as code and documentation. ii. Software complexity is one branch of software metrics that is focused on direct measurement of software attributes, as opposed to indirect software measures such as project milestone status and reported system failures. There are hundreds of software complexity measures, ranging from the simple, such as source lines of code, to the esoteric, such as the number of variable definition/usage associations. iii. An important criterion for metrics selection is uniformity of application, also known as "open reengineering." The reason "open systems" are so popular for commercial software applications is that the user is guaranteed a certain level of interoperability-the applications work together in a common framework, and applications can be ported across hardware platforms with minimal impact. The open reengineering concept is similar in that the abstract models used to represent software systems should be as independent as possible of implementation characteristics such as source code formatting and programming language. The objective is to be able to set complexity standards and interpret the resultant numbers uniformly across projects and languages. A particular complexity value should mean the same thing whether it was calculated from source code written in Ada, C, FORTRAN, or some other language. The most basic complexity measure, the number of lines of code, does not meet the open reengineering criterion, since it is extremely sensitive to programming language, coding style, and textual formatting of the source code. The cyclomatic complexity measure, which measures the amount of decision logic in a source code function, does meet the open reengineering criterion. It is completely independent of text formatting and is nearly independent of programming language since the same fundamental decision structures are available and uniformly used in all procedural programming languages. iv. Ideally, complexity measures should have both descriptive and prescriptive components. Descriptive measures identify software that is error-prone, hard to understand, hard to modify,
____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 65
hard to test, and so on. Prescriptive measures identify operational steps to help control software, for example splitting complex modules into several simpler ones, or indicating the amount of testing that should be performed on given modules. v. There is a strong connection between complexity and testing, and the structured testing methodology makes this connection explicit. A. First, complexity is a common source of error in software. This is true in both an abstract and a concrete sense. In the abstract sense, complexity beyond a certain point defeats the human mind's ability to perform accurate symbolic manipulations, and errors result. The same psychological factors that limit people's ability to do mental manipulations of more than the infamous "7 +/- 2" objects simultaneously apply to software. Structured programming techniques can push this barrier further away, but not eliminate it entirely. In the concrete sense, numerous studies and general industry experience have shown that the cyclomatic complexity measure correlates with errors in software modules. Other factors being equal, the more complex a module is, the more likely it is to contain errors. Also, beyond a certain threshold of complexity, the likelihood that a module contains errors increases sharply. Given this information, many organizations limit the cyclomatic complexity of their software modules in an attempt to increase overall reliability. B. Second, complexity can be used directly to allocate testing effort by leveraging the connection between complexity and error to concentrate testing effort on the most error-prone software. In the structured testing methodology, this allocation is precisethe number of test paths required for each software module is exactly the cyclomatic complexity. Other common white box testing criteria have the inherent anomaly that they can be satisfied with a small number of tests for arbitrarily complex (by any reasonable sense of "complexity") software.
vi. Cyclomatic Complexity Metric (McCabe & Associates, Inc.) A. Cyclomatic Complexity is a measure of the complexity of a module's
decision structure. It is the number of linearly independent paths and therefore, the minimum number of paths that should be tested. B. Cyclomatic complexity measures the amount of decision logic in a single software module. It is used for two related purposes in the structured testing methodology. First, it gives the number of recommended tests for software. Second, it is used during all phases of the software lifecycle, beginning with design, to keep software reliable, testable, and manageable. Cyclomatic complexity is based entirely on the structure of software's control flow graph. C. Cyclomatic complexity measures branches in the control flow of a program. In the simplest possible code, there are 0 branches and cyclomatic complexity equals 1. For every branch, a value of 1 is added to the complexity total.
vii. Halstead Software Metrics (Dr. Maurice Halstead) A. Program Length
The total number of operator occurrences and the total number of operand occurrences.
B. Program Volume C. D. E. F. G.
The minimum number of bits required for coding the program. Program Level and Program Difficulty Measure the program's ability to be comprehended. Intelligent Content Shows the complexity of a given algorithm independent of the language used to express the algorithm. Programming Effort The estimated mental effort required to develop the program. Error Estimate Calculates the number of errors in a program. Programming Time The estimated amount of time to implement an algorithm.
____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 66
H. I. J. K. L.
Line Count Software Metrics Lines of Code Lines of Comment Lines of Mixed Code and Comments Lines Left Blank
c. Size measurements
i. Methods developed for measuring the (primarily software) size of information systems, such as lines of code, function points, etc. Also effective in measuring software development productivity. ii. The most widely used size measure is a count of source lines of code (SLOC). Unfortunately, there are as many definitions of what to count as there are people doing the counting. Some people count executable statements but not comments; some include declarations while others exclude them; some count physical statements and others count logical statements. Published information on software measures that depend on this measure is therefore difficult to interpret and compare. One SEI report says this about measurement of source code size: “Historically, the primary problem with measures of source code size has not been in coming up with numbers —anyone can do that. Rather, it has been in identifying and communicating the attributes that describe exactly what those numbers represent.” The precision of a measurement of source lines of code does not depend on the numbers used in counting (everyone agrees to use the nonnegative integers), so it must depend on what we choose to count. A comprehensive definition of what kinds of statements or constructs in a program to count is necessary before precise measurement is possible.
iii. Function Points
A. Allan Albrecht (Reference 1), in collaboration with John Gaffney, Jr. (Reference 2), designed FPs as a direct measure of functionality. FPs are a weighted sum of the number of inputs, outputs, user inquiries, files, and interfaces to a system. The latest counting rules are defined in Release 3.0 (1990) of "Function Point Counting Practices Manual," by the International Function Points Users Group (IFPUG). B. Function Points and the Function Point Model are measurement tools to manage software. Function Points, with other business measures, become Software Metrics. C. Basic function points quantify the size and complexity of an application based on that application's inputs, outputs, inquiries, internal files, and interfaces. The resulting count is then adjusted based on the complexity of the system defined by a set of general system characteristics. Since function points are independent of language, operating system, platform or development process, it avoids the problems that arise from the use of source lines of code (SLOC) to measure the size of an application. Function points have been gaining in popularity and usage in recent times. At the 1993 International Conference on Applications of Software Measurement, it was announced that function points had become the most widely used metric in the world. D. Function Points measure Software size. Function Points measure functionality by objectively measuring functional requirements. Function Points quantify and document assumptions in Estimating software development. Function Points and Function Point Analysis are objective; Function Points are consistent, and Function Points are auditable. Function Points are independent of technology. Function Points even apply regardless of design. But Function Points do not measure people directly. Function Points is a macro tool, not a micro tool. Function Points are the foundation of a Software Metrics program. E. Software Metrics include Function Points as a normalizing factor for comparison. Function Points in conjunction with time yield Productivity Software Metrics. Function Points in conjunction with defects yield Quality Software Metrics. Function Points with costs provide Unit Cost, Return on Investment, and Efficiency Software Metrics, never before available.
____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 67
Supplemental Software Measures. (The other two are time and money. many attributes like marble bathrooms and tile floors might actually make the smaller house more expensive. Function points are a measure of the size of computer applications and the projects that build them. might also make the smaller house more valuable as a residence. but until now unmeasured. technology or capability of the project team used to develop the application. Function Points. The fact that Albrecht originally used it to predict effort is simply a consequence of the fact that size is usually the primary driver of development effort. they are also in the software business. I. the Function Point Model. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 68 . and the Software Metrics they generate. Function Points and Usage or Volume measures create Software Metrics that demonstrate an organization's ability to Leverage software's business impact.) Without them your organization is only two thirds whole. Calculating several variations on the function points produced per month theme tells them how well they are doing in this regard. Other factors. Function Point analysis can be used for: a. The Leverage of E Commerce is obvious. Function Points and Skill measures provide Software Metrics for Employee Service Level Agreements to meet current and future company skill needs. Function points are not a perfect measure of effort to develop an application or of its business value. It is important to stress what function points do NOT measure. All of the above Software Metrics can prove your organization is Doing Things Right! But the real and biggest value of Function Points and Software Metrics is proving you are Doing The Right Things! G. development methodology. H. However. Function Points can even measure the Corporate Vision and generate Software Metrics to report progress toward meeting it.F.Many executives have come to the conclusion that regardless of their core business. This is often illustrated with an analogy to the building trades. although the size in function points is typically an important factor in measuring each. are only the third measure that transcend every part of every organization. Measure productivity -. It is independent of the computer language. Function Point Analysis. Function Points support Customer Satisfaction measures to create Value Software Metrics. point of view. like location and number of bedrooms. The size is measured from a functional. The function points measured size. A three thousand square foot house is usually less expensive to build one that is six thousand square feet. J. or user.
c. b. in effect.To put them in perspective. M. At best. The size in function points is an attribute that needs to be tracked for each application and project.Since the beginning. A + B = C 2. In embedded software engineering. The evolution of feature points included algorithms. or any variable name. and logical operators. counting lines of code measures software from the developers' point of view. For example. we observed that all algorithms contain four basic characteristics: elements. They. this will allow decisions regarding the retaining. K. Maurice Halstead. unique and total operators and unique and total operands. function points have been used as an estimating technique. take longer to develop. Normalize other measures -. c. less than. we see the results shown in Figure 1. some are more complex. Outsourcers. d. arithmetic operators. reports and other external objects. d. The same 100 delivered defects on a 10. some are simplistic. retiring and redesign of applications to be made. frequently use function points to demonstrate compliance in these areas.B. originated by Allan Albrecht in 1979 while he was with IBM. If we count the number of elements and operators in any given algorithm. Just as he observed that all software contains four basic characteristics. Estimating is obviously necessary for the cost benefit analysis that justifies application development. N. IF (D lt A/B*((C/G-E*F)) AND A/B gt 0 THEN D=0 5. subtract. NOT. divide.Companies outsourcing significant parts of their IS requirements are concerned that the outsourcing entity deliver the level of support and productivity gains that they promise. applications algorithms abound. but only counted the number of algorithms used. OR. L. accurate estimation is required for proper staffing. Arithmetic operators: add. Relational operators: equal. other measures frequently require the size in function points. Monitor outsourcing agreements -.b. treating them all equally. A+B+((C/G-E)*F)=D 4. and correctly executing those algorithms are critical to the applications and add to the complexity of the development effort. Elements: A. Algorithm 1. IF (A AND B) OR Arithmetic Relational Logical Elements Operators Operators Operators Total 3 1 1 0 5 5 3 1 0 9 7 5 1 0 13 10 9 6 0 3 1 1 7 20 17 ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 69 . Logical operators: AND. like CSC and IBM Global Services. multiply.Companies must analyze their portfolios of applications and projects.D. A*(B-D)/C = E 3. relational operators. 100 delivered defects on a 100 function point system is not good news. Estimate development and support -. Designing.C. PC screens of data.000 function point system are much easier to take. and printed reports. While at Purdue University. coding. Since function points are based on screens. this measure takes the users' view Basic function points. exponents. he identified the commonality of software characteristics in his software science methods . How then can we consider any sizing metric without considering the algorithm complexity characteristic? We approached the algorithm problem in a manner not unlike that of the late Dr. were designed primarily for business applications using disk files. Along with other data. debugging. They may be defined as follows: a. Even for strategic projects that need no quantitative justification. Drive IS related business decisions -. greater than. e.
In the first example. When found in executable code. O. easy to use. The EFPs are then added to the Engineering Function Point Summary Sheet and included in the overall calculation of unadjusted engineering function points. over time. however. so do the total EFPs. Q. d.((C AND (D OR (E OR F))) AND (G OR H)) THEN X=1 Figure 1: Algorithm Complexity. It can then be tracked to ensure completion of the project on time. consistent. It is not the "silver bullet" of metrics. collection. but it is a positive step in the right direction for measuring software in an early. efficient execution timing. the greater the effort needed to deliver a quality product. One of the fundamental tenets of the statistical approach to software test is that it is possible to create fault surrogates. It provides management with a simple. Defect measurements i. or data definition in a computer program. Only then can it be said that all the relevant faults have been isolated and removed from the software system. process. ii. While we cannot know the numbers and locations in faults. and no logical operators for a total of five engineering function points (EFPs). flexible tool to track development progress against the projected plan utilizing deliverables instead of tracking staff hours used or number of dollars left in the budget for the project. we can. The system can be used to provide a productivity rate and used as a planning tool for new projects to provide more accurate and achievable schedules and resource allocation. usually related to system size. Engineering function points and its tracking system provide a sizing metric very early in the software development cycle when it can be the most useful and least expensive and can be used throughout the development cycle to check progress as it is made. one arithmetic operator (+). such as defects/1000 function points. Faults are defects that have persisted in software until the software is executable. and reproducible manner. Software faults and other measures of software quality can be known only at the point the software has finally been retired from service. This provides us with a consistent and reproducible method of counting and differentiating algorithms when counting function points in an engineering environment. It is independent of software language used. iii. hardware platform. nor are all defects the result of human mistakes. a defect is frequently referred to as a fault or a bug. This system provides a consistent. we can see that there are three elements (A. P. It is as effective with a one-person team with a small project as it is with a 100-person team with a very large project. and posting time. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 70 . As the algorithms increase in complexity.B. not just from the standpoint of writing code. one relational operator (=). The tracking mechanism reflects any requested changes and immediately shows the impact and the effort needed to keep the current time schedule intact. not all human mistakes are defects. but for correct element definition and usage. or development process. Values associated with numbers or types of defects. and consistent functional results in accordance with required action. A software defect is a manifestation of a human (software producer) mistake. It is a more definitive sizing metric than using SLOC that is difficult to define and is not consistent across multiple software languages.C). It avoids extensive intrusion into the developer's time allowing more opportunity for the creative function and requires little configuration. A fault is an incorrect program step. build models based on observed relationships between faults and some other measurable software attributes. reproducible method to predict how large a software project will be before it has ever been designed or coded. We can consider that the larger the point total. We will define a software defect to be any flaw or imperfection in a software work product or software process.
and check for error prone places in your project. Style. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 71 . Customer satisfaction i. A. Automated Complexity Tools (example): i. Defect Distribution. Achieving 100% efficiency in defect removal has only been observed twice out or more than 10. a fact that inevitably places increased emphasis on defect related software measurements. and government agencies. A. Enforce programming standards. Project Analyzer is a complete code review and quality control tool for Visual Basic. including requirements themselves. CFI Group. the American Customer Satisfaction Index (ACSI) is a uniform and independent measure of household consumption experience. Customer Quality Evaluation Measurement Methods a. A powerful economic indicator.iv. Metrics. Somewhat surprisingly. Code coverage and testing effort measurements complement the defect metrics and provide additional software product as well as process quality indicators. v. D. The overall average for software defects in the United States for 2001 hovers around 5 defects per function point from initial requirements through one year of production. Are you sure all the forms resize? How about error handling? B. When otherwise similar projects are compared. The best organizations in terms of overall quality use synergistic combinations of formal inspections. Errors can enter software applications from a variety of sources. Some of the most advanced thinking in the business world recognizes that customer relationships are best treated as assets. most forms of testing are less than 50% efficient in finding defects. The ACSI is produced through a partnership of the University of Michigan Business School. average for removing defects prior to delivery is about 85%. length of names. formal testing. and that methodical analysis of these relationships can provide a road map for improving them. depth of conditional nesting. the American Society for Quality (ASQ). and can remove more than 96% of them before delivery to clients. the ACSI tracks trends in customer satisfaction and provides valuable benchmarking insights of the consumer economy for companies. Estimate the quality of your code with metrics such as logical lines of code. cyclomatic complexity. designs. those removing more than 95% of defects before release have shorter schedules and lower costs than those removing less than 85%. The identification and removal of software defects constitutes the basis of the software testing process. while Defect Age.exe by up to 100s of kB. you remove unnecessary code. Fix that spaghetti. ii. The American Customer Satisfaction Index (ACSI) was developed to provide business with this analytical tool. Best-in-class organizations create only about half as many defects. source code. industry trade associations. Determination of the level of service perceived by the customer including the ability to meet requirements and overall expectation. The U. These activities average around 65% in removal efficiency. Established in 1994. Find inefficient code such as unnecessary Variants. Detect dead code and decrease your . Defect Density and Defect Type metrics allow the quantification of the quality on software modules. often referred to as "the voice of the nation’s consumer. and the international consulting firm." is published quarterly in the Wall Street Journal and provides a benchmark for success in the private sector. comment to code ratio. Optimization. get recommendations for better coding style. and very complete defect measurements. e. vi. An optimal suite of formal inspections and test stages can top 99% in cumulative defect removal efficiency. It is important to note that excellence in software quality has a very positive return on investment. With Project Analyzer's problem detection feature. Defect Detection Rates and Defect Response Time metrics allow for pinpointing software inspection and testing process shortcomings. Functionality.000 projects examined. in that at least half of latent defects remain after the testing is finished. and “bad fixes” or secondary defects introduced during defect repairs. and have topped 85%.S. so the volume of delivered defects averages about 0.75 defects per function point. The ACSI. vii. The most efficient defect removal activities yet measured are formal design and code inspections.
b. and statistics to be provided. while ensuring that end users have a factual understanding of network realities. At a minimum. A service-level agreement (SLA) is an informal contract between a carrier and a customer that defines the terms of the carrier's responsibility to the customer and the type and extent of remuneration if those responsibilities are not met. (International Engineering Consortium) iv. The establishment of a contract with the customer to maintain an agreed upon service level for the customer’s application(s). ii. For the first time. inappropriate data collection methods. Performance/service measures (i.e.. SERVICE LEVEL AGREEMENTS . Customer Satisfaction Measurement for ISO 9000 explains in a clear and simple manner how to conduct a professional customer satisfaction survey that will produce a reliable result--as well as be consistent with the requirements of ISO 9001:2000. availability or downtime. payment of "failure credits. Customers and service providers are encouraged to discuss a number of different scenarios that may lead to compensation by the service provider if expected outcomes are not met. when they are delivered and where. The SLA is an insurance policy of sorts. The SLA is the key to setting the users’ service expectation. and reference documents B. SLAs generally include minimum standards related to the following: A. the responsibilities of the service provider and the customer. major players. typically to include response time. Price or charge to the customer for use of the service vii.iii. It ensures that the organization understands and works in sync with business goals. Scope: includes the purpose. and invalid statistic analysis. A contract between a network service provider and a customer that specifies the services the network service provider will furnish. conformance requirements. Service-level agreements i. workload. the following points should be addressed in the SLA: A. number of users that can be served simultaneously. Customers should ensure they have an accurate understanding of the terms of the agreement vi. iii. Information systems are becoming increasingly more complex. and the criteria and metrics for measuring performance. and callback/repair-dispatch response time. A SLA contains a set of definitions that identifies what the service deliverables are. the Service-Level Agreement (SLA) serves as a valuable tool in meeting this challenge by documenting the success of the system in meeting needs and expectations. Chargeback systems compare costs to expected benefits and provide benefits (i. SLAs should be short yet precisely define the services and the level of services to be provided. A SLA defines the acceptable levels of information systems (IS) performance. Customers should understand what the service provider considers an acceptable level of performance. ISPs often provide SLAs for its customers. Services SLAs may specify often include the percentage of time services will be available." monetary or otherwise) to clients if the service provider fails to achieve the agreed upon service levels. network infrastructure ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 72 . SLAs allow organizations to measure stated objectives by comparing actual performance to the performance levels or standards specified by the SLA. help-desk response time.. Constraints (i. the ISO 9000 quality management standard requires that registered companies measure customer satisfaction. A SLA is a set of broadly defined. repeatable functions and processes––the output of which is delivered to users in accordance with pre-agreed performance levels. v. Many customer surveys produce misleading results due to poor questionnaire design.Documents service objectives. background information.e. the customer’s definition of downtime may be quite different than the definition used by the service provider.e. serviceability. and user satisfaction) B. Defines help desk support. and dependencies) C. reliability. network availability. Technical Support: provides an overview of the support services provided during business hours and after hours. The SLA should not describe how the service is delivered. procedures.. network response time. software and network C. objectives. Supported Environment: defines the hardware. While information systems are evolving. the state of systems management remains constant. For example. rules and regulations. The service can originate from another part of the user’s enterprise or from a third party. Customers can set requirements for IS and network services and weigh them against the service cost.
for example. iii. Localization: problems with supported languages. It’s important for test professionals to remember that many kinds of quality risks exist. hangs. F.) ii. we can group risks into four categories. Data quality: a database becomes corrupted or accepts improper data. The most obvious is functionality: Does the software provide all the intended capabilities? Other classes of quality risks: i. responsiveness. and so forth. either due to usage profiles or the technical risk of the problem? c. In other words. which allows the project team to manage and balance quality risks against the other three areas. Risk Analysis a. etc. The priority of a risk to system quality arises from the extent to which that risk can and might affect the customers’ and users’ experiences of quality. data back-up and recovery. understanding of the customer's problem. The findings of company performance should be analyzed both with all customers and by key segments of the customer population E. In any software development project. currencies. archive-data-special file recovery and server support Support Staffing: (within IS operations) should include security administration and customer services. Use cases: working features fail when used in realistic sequences. Reliability: too often—especially at peak loads—the system crashes. v. Performance: the system functions properly. Priority: How much does a failure of the system in this area compromise the value of the product to customers and users? iii. An understanding and application of the basic techniques needed to provide consistent reliable results. Using quantitative data to manage i. the more testing that problem area deserves. etc. vii. f. Quality risks: How might the product lack customer-satisfying behaviors or possess customerdissatisfying behaviors? Testing allows us to assess the system against the various risks to system quality. Robustness: common errors are handled improperly. Usability: the software’s interface is cumbersome or inexplicable. e. One approach I like is to use a descending scale from one (most risky) to five (least risky) along three dimensions. viii. Likelihood: What are the odds that a user will encounter a failure in this area. Service-level Partnerships: outline the relationships established with providers c. ii. i. network connections. Severity: How dangerous is a failure of the system in this area? ii. Methods for using quantitative data as a management tool.D. (A common method is a questionnaire. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 73 . support and server maintenance. Evaluation methods i. service promptness. i. Schedule risks: How might the project exceed the allotted time? iii. courtesy of staff. A basic and effective baseline customer satisfaction survey program should focus on measuring customer perceptions of how well the company delivers on the critical success factors and dimensions of the business as defined by the customer. the system fails. iv. permitting objective evaluation. support and other operational problems Response-time Goals: includes hardware and software orders. You can prioritize in a number of ways. the more likely a problem or the more serious the impact of a problem. Financial risks: How might the project overrun the budget? ii. but too slowly. Volume/capacity: at peak or sustained loads. Customer service departments should be staffed with resource managers and technology coordinators. time zones. Feature risks: How might we build the wrong product? iv. E. b. kills sessions. vi. d.
errors can be detected earlier. (3) defines resources required to accomplish each task. Extra Information A. D. DEVELOPMENT COSTS .An assertion concerning the functioning of an application software entity. C. A formal or informal plan for carrying out a particular test that: (1) defines tasks to be performed. Samples of live data may be used for test data if they are analyzed.A tool for directing the software testing which contains the orderly schedule of events and list of materials necessary to effect a comprehensive test of a complete application. H. PROTOTYPE . G. you cannot substitute any prototype for a paper specification. computer usage. interface. TECHNICAL REQUIREMENTS . (2) specifies sequential dependencies among the tasks.9. derived from the Operational Requirements Document. and equipment costs.A system life cycle documentation standard that identifies high-level requirements and defines the objectives and overall structure of the test and evaluation for a system.g. Prototyping is a complement to other methodologies.An active model for end-users to see touch feel and experience. and the cost of any new computer equipment and software. test management strategy and structure. supply. costs associated with the installation and start-up of the new system must be calculated. performance.Development costs include personnel costs. and data elements created by users.A series of dependent tasks for a project that must be completed as planned to keep the entire project on schedule. TEST PLAN . CRITICAL PATH . (4) schedules task starts and completions. and critical technical parameters. TEST CASE . test tasks to pertinent user/design requirement. development. key performance parameters and operational performance parameters (threshold and objective criteria). and (3) a definition of the overall processing sequence for the test. It is the working equivalent to a paper design specification with one exception .. (2) a definition of the test time dimension (the time span covered by the test and the division of that time span into discrete periods.The individual or group who will use the system for its intended operational use when it is deployed in its environment. training. There is no standard format for test data. the truth of which must be demonstrated through testing in order to conclude that the entity meets established user/design requirements. Examples of technical requirements include functional.The high-level design of a planned application software test. or maintenance work (e. However. END USER . TEST DATA . Those requirements that describe what the software must do and its operational constraints. I. and required resources to critical operational issues. analysts. and supplemented as necessary.e. In addition. a hypothetical user environment intentionally constructed to be sufficiently diverse and complex to support execution of all relevant test cases. and resource requirements for test and evaluation. It relates program schedule. evaluation criteria. TEST and EVALUATION PLAN . E. J.. i. It details the test strategy. response time). and major decisions. schedule. and those parts of the document directed toward other personnel should be presented in suitable terminology. records. B. TEST ARCHITECTURE . design specifications. Those parts of the document directed toward the user staff personnel should be presented in noncomputer-oriented language. to determine completeness in terms of all conditions which can occur.Any requirements related to software. and developers to test requirements. A test architecture includes: (1) a structural blueprint. and quality requirements F. via an initial traceability matrix.Files. ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 74 . and (5) links. and software code. a. It provides the framework within which detailed test and evaluation plans are generated.
The notion is specific to a test selection criterian: i.a set of activities. Can anyone tell me the procedure to arrive at the metrics baselines report? I believe that it depends upon the organisation to organisation when they will update the MBR. i.. 4. P. . testing is "complete" when the tests specified by the criterion have been passed.erroneously used to mean 100% branch coverage. and whether software installed is running without hiccups. required set up procedures.A description of the test which must be executed to verify a system/software requirement. Postings from CQAfolks (Yahoo) Testing Standards: Testing Standards mainly are those set by ISO. Absolutely complete testing is impossible. methods. by Boris Beizer. For revision we can take the existing & closed project in that defined duration let say in my orgainsation we do it after 3 months. 5. costs for a CMMI assessments would be around 45000 USD while that for CMM would be around 28000 USD. a CMM Level 4 initiative can reach you to a tangible milestone at an early date (say 9 to 12 months). “advaitslele@indiatimes. required execution procedures. For exsiting project we will take only those phases which are complete. L. 3. TEST SCRIPT . A project closure report can be prepared either by formal project closure report or it can be in the form of a checklist. and transformations that people use to develop and maintain software work products [CMU/SEI 91]. You also have standards set by British for their own companies to follow. A CMMI Level 4 initiative could run for much longer (say 18 months and above). TEST REQUIREMENTS .com”) How to prepare a Project Closure Report: (courtsey of Advait.K. Software process .Quoted from the Glossary/Index of Software Testing Techniques .The process of evaluating software to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase (IEEE-STD-610). practices. COMPLETE TESTING . otherwsie to make the new MBR we should have atleast 4-8 datapoints it means that project for same nature or technolgy. and required evaluation procedures. to demonstrate which type of testing is to be executed in parallel with the phases of the SDLC.. It is not correct that CMM is being phased out by end of 2003. you can fill the report and get it counter signed by the client.A system life cycle documentation standard that is the design specification for a test run. So. A CMM Level 4 initiative would produce tangible results for you and you can reach a milestone in a comparative lesser duration and can act as a strong springboard to launch your further initiatives. depending upon the service provider. N.* * . M.e. in the report you can have columns to check whether all the reuqirements are met. plans procedures. (ISBN 0-442-20672-0) O. NIST. Currently. Test requirements should generally exist at levels corresponding to the requirements. 2. organizations who have been assessed for CMM Level 5 are finding that they need about 15 months to acheive a proper CMMI Level 5 implementation. and all the installation activities are over at the client's site. This is part of the traceability matrix.e. if not later than that. when you are in the installation phase.2nd ed. after a careful examination ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 75 . Software Work Product .any artifact created as part of the software process includingcomputer programs. It defines the test cases to be executed. Given a profile with ISO 9001:2000 certificate and Process focus in the organization. IEEE. We have confirmed this with SEI and the same is indicated in SEI's web site as well. SEI-CMM or DoD.Ankur Handa CMM & CMMI certain facts: CMM or CMMI? how long will it take? can transition between from ISO to CMMI can happen? 1. CMM is definitely on till end of 2005. and associated documentation and data [CMU/SEI 91]. This is misinformation prevailing in the market. 6. V-Model is only an approach for testing. V-Model is not a standard. VERIFICATION .
3) Tools used : Tools used in the coding phase . Listen to customer 2. Design 3. A prototype is a scled down version of the full system and serves for further clarification of the requirements. Testing tools. Software Engineering Paradigms Also known Software Process Models . Testing The Prototyping Model This model skips the rigorous steps involving requirements analysis and specifications since the customer is unsure about what is required and the developer would like to make sure the implementation is not going to be a big problem. Have customer test-drive the new prototype. Metrics like : Productivity achieved v/s Planned . Its a cyclic process as shown below: 1. checklist . Quality of product (Include Acceptance Defects ) Customer Complaint received and resolution 2) Process Details : Methodology used. They mainly involve the question as to what software development process is all about and how it can best be controlled. guidelines in any. Closure of Project report needs to address following topics 1) General Information : Project Description . Through the years. Deviations in the process. The following is a quick list of these models: Linear Sequential Model Also known as waterfall model as well as SDLC = Software Development Life Cycle. Duration . this model involves welldefined steps or phases in the software development process. Project management tool . Build/revise mock-up 3. After a cursory requirements gathering and a quick design. Technology used. Contingency plan etc 5) Metrics for all the phases a) Effort Estimation b) Schedule Phase wise c) Test Defects d) Review defects e) Paretto analysis f) Cause and effect analysis g) Defect Leakage across phase h) Defect density I) Any other metrics as required for the customer 6) SWOT analysis of the project 7) Overall conclusion of the project .of the system at client's site after installation (u can get the client to sit besides u when this check is being performed) you can fill up the report / checklist. Coding 4. Team size etc. these are various strategies for successful solution of software engineering problems. the developer build a prototype. various models emphasizing the product or process views to one degree or another have been proposed and discussed in the literature. then start all over The RAD Model ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 76 . Analysis 2. CM tools etc 4) Risk Management : Risk assessment details . These steps are: 1. Tailoring the templates.
This approach involves the following phases: 1. using the ptotyping philosophy. 3. 3. In later increments. increasingly complete versions are engineered. as with the earlier models.2) ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 77 . (ISO 9126: 1991. State transition diagrams are use d for keeping track of and controlling various phases. The early iterations may be a paper or simple prototypes. Rather than defining as simple sequence of activities. increasingly more complex versions of the software is developed. and requires careful assesment and analysis of risks at each step. It relies heavily on software reuse which is a big issue in and of itself. 3. The Component Assembly Model This is essentially a spiral model with technical framework based on object technologies.21) Features: Features are identified properties of a software product which can be related to the quality characteristics. Process Modeling 4. The Spiral Model originally proposed by Boehm. Data Modeling 3. beneficiary or second party. this model highly realistic. The Concurrent Development Model Also known as concurrent engineering. this model is based on keeping track of many phases of the project simultaneously. Business and product requirements change and the details and extensions of the core software needs to adapt to these changes. When applied to client-server models. (ISO 8402: 1986. Since it emphasizes the evolutionary nature of the software development proces where customer and developer both understand and react better at each step. this is an evolutionary model which combines the iterative nature of prototyping with the systematic aspects of life cycle model. The activities associated with various phases are grouped together and defined as a state. this model is based on using appropriate models --as above-. client. this model is not a panacea. this model defines a network of clusters of activities. In an iterative manner. 3. Business Modeling 2.4) Defect: The nonfulfilment of intended usage requirements. The following list defines terms that could be relevant for evaluation purposes: Assessment: An action of applying specific documented assessment criteria to a specific software module. But. this model defines activities in two dimensions: The system dimension and the component dimension. The Incremental Model The first increment is normally referred to as core product and thereafter. limited versions are introduced to meet the competitive requirements. It is an attempt to produce the iterative versions of the product from prepackaged sotware components corresponding to object classes. more sophisticated versions are devloped with new features. Therefore. package or product. user.This Rapid Application Development model is essentially an SDLC with extremely short life cycle. Software is developed in a series of incremental releases. (ISO 9004: 1987. package or product for the purpose of determining acceptance or release of the software module. In essence. (ISO 9126: 1991. Testing and turnover The Evolutionary Models These models are based on evolutionary nature of software development process. This model is fairly new and has not yet been assessed for efficacy.for different components of a large system at the same time.1) Customer: Ultimate consumer. Application generation 5. It is a highspeed adaptation of SDLC using component based construction approach.
(ISO 8402: 1986.7) Quality surveillance: The continuing monitoring and verification of the status of procedures. Users.19) Measurement: The action of applying a software quality metric to a specific software product. (ISO 9126: 1991.4) Liability (product/service): A generic term used to describe the onus on a producer or others to make restitution for loss related to personal injury.2) Security: Attributes of software that bear on its ability to prevent unauthorized access. processes.1) Quality assurance: All those planned and systematic actions necessary to provide adequate confidence that a product or service will satisfy given requirements for quality. 3. (ISO 8402: 1986. to programs and data.2.5) Nonconformity: The nonfulfilment of specified requirements. (ISO 8402: 1986. 3. The term `reliability' is also used as a reliability characteristic denoting a probability of success or a success ratio.7) Rating level: A range of values on a scale to allow software to be classified (rated) in accordance with the stated or implied needs.8) Recoverability: Attributes of software that bear on the capability to re-establish its level of performance and recover the data directly affected in case of a failure and on the time and effort needed for it.2.3) Inspection: Activities such as measuring. 3. The computer program and data contained in firmware are classified as software. procedures. A. Used to determine the rating level associated with the software for a specific quality characteristic. These levels are called rating levels. (ISO 8402: 1986. 3. products and services.6.1.5) Software: Intellectual creation comprising the programs. methods.4. (ISO 9126: 1991. the circuitry containing the computer program and data is classified as hardware. (ISO 9126: 1991.2. (ISO 9126: 1991.Firmware: Hardware that contains a computer program and data that cannot be changed in its user environment. 3. 3. testing. (ISO 9126: 1991. (ISO 9000-3: 1991.e. property damage or other harm caused by a product or service. conditions. (ISO 9126: 1991. rules and any associated documentation pertaining to the operation of a data processing system. (ISO 9126: 1991.20) NOTE -. and analysis of records in relation to stated references to ensure that specified requirements for quality are being met.4) Resource behaviour: Attributes of software that bear on the amount of resources used and the duration of such use in performing its function. whether accidental or deliberate.The basic difference between `nonconformity' and `defect' is that specified requirements may differ from the requirements for the intended use. A. (ISO 8402: 1986. (ISO 9126: 1991. (ISO 8402: 1986. 3. examining. 3. (ISO 8402: 1986. 3. 3. (ISO 9126: 1991. 3.2.18) Replaceability: Attributes of software that bear on the opportunity and effort of using it in the place of specified other software in the environment of that software.20) Quality: The totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs.2.11) Rating: The action of mapping the measured value to the appropriate rating level. represented by a specific set of values for the quality characteristics. gauging one or more characteristics of a product or service and comparing these with specified requirements to determine conformity. 3. (ISO 8402: 1986. Appropriate rating levels may be associated with the different views of quality i.14) Level of performance: The degree to which the needs are satisfied. Managers or Developers.1) ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 78 . 3.3) Reliability: The ability of an item to perform a required function under stated conditions for a stated period of time. (ISO 8402: 1986. 3.6) Quality control: The operational techniques and activities that are used to fulfill requirements for quality. A. 3. A.
(ISO 9126: 1991. 3. (ISO 9126: 1991. A software quality characteristic may be refined into multiple levels of sub-characteristics. The quality is represented by the set of rated levels associated with the software product. A. 3.3) Suitability: Attribute of software that bears on the presence and appropriateness of a set of functions for specified tasks. 3. (ISO 9126: 1991.11) Software quality assessment criteria: The set of defined and documented rules and conditions which are used to decide whether the total quality of a specific software product is acceptable or not. (ISO 9000-3: 1991. A. (ISO 8402: 1986. (ISO 9000-3: 1991. (ISO 9126: 1991.2) Software item: Any identifiable part of a software product at an intermediate step or at the final step of development. by a stated or implied set of users. procedures and associated documentation and data designated for delivery to a user.4.2.4) Time behaviour: Attributes of software that bear on response and processing times and on throughput rates in performing its function. (ISO 9126: 1991. 3. and on the individual assessment of such use. (ISO 9126: 1991.3) Validation (for software): The process of evaluating software to ensure compliance with specified requirements.14) Specification: The document that prescribes the requirements with which the product or service has to conform.1) Testability: Attributes of software that bear on the effort needed for validating the modified software. (ISO 9126: 1991. A.2.7) Verification (for software): The process of evaluating the products of a given phase to ensure correctness and consistency with respect to the products and standards provided as input to that phase.22) Stability: Attributes of software that bear on the risk of unexpected effect of modifications.12) Software quality characteristics: A set of attributes of a software product by which its quality is described and evaluated. (ISO 9000-3: 1991. (ISO 9000-3: 1991.5.3. 3. A. A. (ISO 9126: 1991.1) Understandability: Attributes of software that bear on the users' effort for recognizing the logical concept and its applicability. 3. 3. 3. (ISO 9126: 1922.214.171.124 ____________________________________________________________________________________________________________________________ CSQA Exam Notes Revised: 08/19/2002 Page: 79 .13) Software quality metric: A quantitative scale and method which can be used to determine the value a feature takes for a specific software product. 4.Software product: Complete set of computer programs.1) Usability: A set of attributes that bear on the effort needed for use. (ISO 9000-3: 1991.2.3) Software quality: The totality of features and characteristics of a software product that bear on its ability to satisfy stated or implied needs. 3.5.2.
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue listening from where you left off, or restart the preview.