You are on page 1of 298

Guide to the


Version 6.2 © 2008

Copyright © Quality Assurance Institute 2008 All Rights Reserved No part of this publication, or translations of it, may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or any other media embodiments now known or hereafter to become known, without the prior written permission of the Quality Assurance Institute. Visit for additional courseware and training seminars.

Table of Contents
Skill Category 1 Quality Principles . . . . . . . . . . . . . . . . . . . . . . . . . . 1-1
1.1. Vocabulary of Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-1 1.2. The Different Views of Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-3 1.2.1. The Two Quality Gaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-4 1.2.2. Quality Attributes for an Information System . . . . . . . . . . . . . . . . .1-5 1.3. Quality Concepts and Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-6 1.3.1. PDCA Cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-8 1.3.2. Cost of Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-9 The Three Key Principles of Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-10 Best Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-11

1.3.3. Six Sigma Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-11 1.3.4. Baselining and Benchmarking . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-12 1.3.5. Earned Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-12 1.4. Quality Control and Quality Assurance . . . . . . . . . . . . . . . . . . . . . . .1-12 1.4.1. Quality Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-13 1.4.2. Quality Assurance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-13 1.4.3. Differentiating Between Quality Control and Quality Assurance .1-14 1.5. Quality Pioneers Approach to Quality . . . . . . . . . . . . . . . . . . . . . . . .1-15 1.5.1. Dr. W. Edwards Deming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-15 1.5.2. Philip Crosby . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-18 1.5.3. Dr. Joseph Juran . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-21

Version 6.2


Guide to the CASQ CBOK

Skill Category 2 Quality Leadership . . . . . . . . . . . . . . . . . . . . . . . . . .2-1
2.1. Leadership Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1 2.1.1. Executive and Middle Management Commitment . . . . . . . . . . . . . 2-2 Executive Management Commitment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2 Middle Management Commitment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2

2.1.2. Quality Champion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2 Leadership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2

2.2. Quality Management Infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . 2-3 2.2.1. Quality Council . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-3 2.2.2. Management Committees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4 2.2.3. Teams and Work Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4 Personal Persuasion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5 Resolving Customer Complaints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-6 Written Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-7

2.3. Quality Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-10 2.3.1. The Six Attributes of an Effective Quality Environment . . . . . . . . 2-10 2.3.2. Code of Ethics and Conduct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-11 2.3.3. Open Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-12 Guidelines for Effective Communications . . . . . . . . . . . . . . . . . . . . . . . . 2-12

2.3.4. Mission, Vision, Goals, Values, and Quality Policy . . . . . . . . . . . 2-18 Mission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Quality Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-18 2-18 2-19 2-21 2-22


Version 6.2

Table of Contents

Skill Category 3 Quality Baselines . . . . . . . . . . . . . . . . . . . . . . . . . . 3-1
3.1. Quality Baseline Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-1 3.1.1. Baselines Defined . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-1 3.1.2. Types of Baselines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-2 3.1.3. Conducting Baseline Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-2 Conducting Objective Baseline Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4 Conducting Subjective Baseline Studies . . . . . . . . . . . . . . . . . . . . . . . . . . 3-5

3.2. Methods Used for Establishing Baselines . . . . . . . . . . . . . . . . . . . . . . .3-6 3.2.1. Customer Surveys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-7 3.2.2. Benchmarking to Establish a Baseline Goal . . . . . . . . . . . . . . . . . . .3-7 3.2.3. Assessments against Management Established Criteria . . . . . . . . . .3-9 3.2.4. Assessments against Industry Models . . . . . . . . . . . . . . . . . . . . . . .3-12 3.3. Model and Assessment Fundamentals . . . . . . . . . . . . . . . . . . . . . . . .3-12 3.3.1. Purpose of a Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-12 3.3.2. Types of Models (Staged and Continuous) . . . . . . . . . . . . . . . . . . .3-13 Staged models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-13 Continuous models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-13

3.3.3. Model Selection Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-13 3.3.4. Using Models for Assessment and Baselines . . . . . . . . . . . . . . . . .3-14 Assessments versus Audits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-15

3.4. Industry Quality Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-16 3.4.1. Software Engineering Institute Capability Maturity Model Integration (CMMI®) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-16 Maturity Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-16

3.4.2. Malcolm Baldrige National Quality Award (MBNQA) . . . . . . . . .3-17 Other National and Regional Awards . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-18

3.4.3. ISO 9001:2000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-18 Model Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-18

3.4.4. ISO/IEC 12207: Information Technology – Software Life Cycle Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-19 Model Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-19

3.4.5. ISO/IEC 15504: Process Assessment (Formerly Known as Software Improvement and Capability Determination (SPICE)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-19 Model Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-19

Version 6.2


Guide to the CASQ CBOK 3. . . . . . . . . 3-20 4 Version 6. . .2 . Post-Implementation Audits . . . . . . . . . . . . . . . . .4. . .6. .

. . . . . . . 4. . . . . . . . . . . . . . . . . . . . . . . . . . . Histogram . . . . . . .2. . . . . . . . . . . . . . . . . . . . . . . . . . .2.2. . . . . . . . . . . . . Affinity Diagram . . . . . . . . . . . . . .4. . . . . . . . .1. . . . . . . Deployment Phase 1: Assessment . . . . . . . . . . . . . .3. . . . . . . . . . . . . . . . . . . . . . . . . 4. . . .4. . . . . . . .1. . . . . . . . . . . . . .3. . . Line Chart . Quality Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3. . . . . . . . . . . . . . . . . . . . .3. . . . . . . . . . . . . . . . . . . . . . . . . . . .1. . . . . . . . . . . . . . . . .4-6 4. . . . . .3. . . . . . .2. . . . . . . . . . . . . . . . . .7. . . Flowchart and Process Map . . . . . . . . . . .2 5 . . . . . . . . . . . . . . . . . . . . Long-Term Actions . . . . How the Quality Function Matures Over Time . . . . . . . . . . . . .3. 4-18 4-20 4-21 4-22 4-23 4-25 4-27 4-27 4-28 4-29 4. . . .2. . . . . .4. . . . . . . . . .1. . . . . .1. . .5. . . . . . . . . . . . . . . . . . . The Deployment Process . . . . . .3. . . . . . . . . .8. . . . . . . . . . Benchmarking . . . . . . . . . . .2. . .6. . . . .2. . . . . . . . . . . . . . . . . . 4-31 4. . . . . . . 4-30 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pie Chart . . . . . 4-31 4. . . . . 4-12 4. . . . Check Sheet . . . .Table of Contents Skill Category 4 Quality Assurance . . . .3. .3. .3. . . Brainstorming . . . Establishing a Function to Promote and Manage Quality . . . . . .4-1 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . Presentation Tools . . . . . . . . . . .2. . . . . . .3. . . . . . 4-17 4. . .1. . . . . . .2. . 4. . . . .1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-5 4. . . . . . . . . . . . . . . . . . . . . . . .2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.1. . . . . . . . . .1. . . . . .2. . . . . . . . . . . . . .1. . . . . . . . . .4. . . . . . . . . . 4-8 4.2. . . . . . . . . . . . . . . .1. . 4-12 4. . 4-9 4. . .2. .2. . Bar Chart . . . . . . . . .6.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2. . Deployment Phase 2: Strategic . . . . . . . . . . . .4. . Differences in Responsibilities .2. . . . . .2. . . . . . . . . . . . . . . . . 4.1. . . . . . . . . . Cause-and-Effect Diagram . . . . . . . Short-Term Actions . . . . .2. . .1. . . . . . . . . . . . . . .3. . . . . . .2. . . . . . . . . . . . . . . . . . . Matrix . . . .4-33 Version 6. . . . . . . . . . . . . . . . 4-8 4. . . . . . . 4. . . . . . . . . . . Pareto Chart . . . . . . . . . . . .4-26 4.4-3 4. . . . Types of Internal Audits . . . . . . . . . Critical Success Factors for Deployment . . .1. . . .1. . . . . . . . . . . . . . . . . . . . .1. . . . . . . 4. . . . .1. . .2. . . . . . . . 4. .2. . . . . Nominal Group Technique . . . . . . . . Three Phases of Quality Function Maturation . .4-32 4. . . . . .4-30 4. . . . . . . . . . . . .4-7 4. . . Run Chart . . . 4-4 4. . . . . . . . . . . . . . . . . . . . . .2. . . . . .2. . . . . . . . . Process Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . .2.3. .2. . . . . .2. . . .1. . . . . . . . . . Table . . . . . . . . . .1. . . . . . .4-18 4. . . .4-29 4. . . . . . IT Quality Plan . . . . . . .1.1. . . .4. .2.2.4-31 4. . . . . . . . . . . . .3.3. .2. . . . . . . . . . . . . 4-14 4. . Management Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2. . .1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.2. . . .2. . . . . . . . .1. . . . . . .2. . . . . .1. . . . . . . . . . . . . . . Internal Auditing and Quality Assurance . . . . 4-1 4. . . . . . . . . . . . . . Statistical Tools . . . . . . Force Field Analysis . . . Control Chart . . . . . . 4. .2. . . . . 4-3 4. . 4-10 4. . .1. . . . . Scatter Plot . . . . . . . . . . . . . .2. . . . . . . . . . . .2. . . . . . . . . . . . . .4-32 4. . . . .2. 4. . . .4-4 4. . . . . . . . . . . . . . . . . . . . . . . . . . .5. . . . . . . . . . . . . . . . . . . . . . . . .1. . . . . . . . . . . . . . . . . . . . . . . . .1. . . . . . . . Deployment Phase 3: Tactical . .

. . . . . . .11. . . 5-4 5. . . . . . . . . . . . . 5-15 5. .3. . . . . . . . . . . .1. . . . . . . . . . . The Six Basic Planning Questions . . . . . . . . . . .5. . . . . . . . . . . . .1. . . .3. .4.1. . . . . . . . . . . . .5. . . . . . The Planning Process . . . .7. . 5. . . . . . . Organization/Delegation Planning . . . Relationship of Maturity Level to Cost to Do Work . . . . . . Business or Activity Planning . . . . . 5-15 5.5. . . . . . . . . .5. . . . . . . . .5. . . 5. . . . Relationship between People Skills and Process Definitions . .2. . . . . .3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4. . . . . . Budget/Resources Planning . . . . . . .1. . . . .3. . . . . . .9. . . . Assumptions/Potential Planning .8. . 5. . . 5. . . .8. . . . . . . 5-15 5. . . . . . . . . . . . .4. . . . . . . . . . . . .4. . . . . . . . . . . . . . . . . . . . .9. . . . . . . . .1. . . . . . . .4. . .3. . . . . . . . . . . . . . . . . Relationship of Process Maturity and End User Satisfaction . . . . . . 5-16 5. 5-2 5. . . . . . . . . . . . . . . . 5-6 5. 5-5 5. . . .2. . . . . . . .4. . . . . Relationship of Process Maturity to Defect Rates .5. . 5-15 5. . 5-15 5. . . . . 5-16 6 Version 6. . . . . . . . . . . . . . .1. . .4.1. . Relationship of Individuals' Assessment of How They are Evaluated to Work Performed . . . .1. . . . . . . . . . .3. . . . . . . . . . . . . . . . . . . .5. . . 5-10 5. . . . .4. . . . . . 5-4 5. . .3. . . . . . . . . 5-1 5. . . . The Fallacy of Having Two Separate Planning Processes . . .6. . . . . . . .3. . . . . . . . . . . . . . . . Environment Planning . . . .5. . . .7. . . Integrating Business and Quality Planning . . . . . . . . .5. . . .5. . . . . .4. Planning Concepts . . . . . . . . . . Planning Process Overview . . . . . .2.2. . . . . . Objectives/Goals Planning . . . . . . . . 5. . . 5-14 5. . . . . . . . . The Planning Cycle . . . . . Relationship of What Management Relies on for Success . . . . . . . . . . . . . . .3. . . . . . . . . . . . Relationship of Process Maturity to an Organization's Willingness to Embrace Change . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5. . . . . . .2. . . . . . . . . . Planning Should be a Single IT Activity . . . Policies/Procedures Planning . . . . Capabilities and Opportunities Planning . . . . . . . . . . . . .1. . . . . . . . 5-16 5. . . . . . . . . . .4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3. . . .4. . . . . . .1.5-1 5. . . .2. . . . . . . . . . . . .Guide to the CASQ CBOK Skill Category 5 Quality Planning . 5-13 5. . . .11. . . . . . . Strategy/Tactics Planning . Planning Activities for Outsourced Work . . . 5. . . . .4. . . 5. . . . . . . . . . . .10. . . . .3. .4. . . Relationship of Process Maturity and Cycle Time . . . . . . .2 .3. . . . 5-6 5. . . . . . . . . . . .5. .3. . . . . . . . . . . . . . 5. . . . . . . . . . .1. 5-14 5.3. . . . . . . . . .1.3. . .1. . . Planning to Mature IT Work Processes . . .1. . . . . . Relationship of Do and Check Procedures . . Relationship of Tools to Process Maturity . . . . 5. . . . . . . . . . . . . . . . . . . .1. .2. . . . . . . . . . . . .1. 5-10 5-10 5-10 5-11 5-11 5-11 5-12 5-12 5-12 5-12 5-13 5. .6.4. . . . . . . Relationship of Process Maturity and Staff Job Satisfaction . . . . . . . . . . . . . 5-14 5. . . The Common Activities in the Planning Process . .4. . . . . .4. . 5-8 5. . . .1. . . . 5-15 5. . . . .10. . . . . . How to Plan the Sequence for Implementing Process Maturity . . . . . . .5. . . . . . . . .1. . . . . . . . . . . Priorities/Schedules Planning . . 5-4 5. . . . .1. . . . . . . . . . . . . . .5. . . . . . . . . . . Prerequisites to Quality Planning . . . . . . . .4. .5. .5.

. . . . . . . . . . . . . . . . . . . . Act Processes . . . . . . . . . . . . . . . Process Categories . . . . . . . . . . . . . .3. . . . . . . . . . . . . .4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-1 6. . . . . . . .5. . . . . . . . . . . . . . .6-1 6. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-9 6. . . . . . .5. . . . . .2. . . . . . . . . . . . . . . . . . . .1. . . . . . . . . .2 7 . . . .3. . . . .2. . . . . . .1. . . . . . . . . . . . .5. . . . . . .4. . Testing . . . . . . . . . . . . . Process Workbench and Components .1. . . . . . . . .6-14 6.1. . . . . . . . . Planning Processes . . 6-7 6-8 6-8 6-8 6. . . . . . . . .2. . . . . . . .1. . . . . .2. . . . . . . . .1. . . . . . . . . . . . . . . Do Processes . . . . . . . . Process Improvement Process . .1. . . . . . . . . . . . . . . . .4. . . . . . . .3. . . . . . . . . . . . . . . .1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.1. . . . 6-20 6. . . . .6-1 6. .Table of Contents Skill Category 6 Define. . . . . . . . . . . . . .3.6-9 6. . . . . Process Template . . . . . . . . . . . 6. . . .6-10 6. . . . . . . . . . . . . . . . Product and Services Continuum . Process Management Processes . . . . . . . . . . Work Process Continuum . .2. . Build. Process Mapping . . . . . . . 6-14 6. . . . . . . .4. . .2. . . . . . . . . .2. . . . . 6-21 6. . . . . 6-11 6.2. . . . . 6-24 Version 6. Definition of a Process . . . . . . . .6. . . . .1. . . . . . Implement. . . . . . . . . . . . . . . .5. .1. .1. . . . .2.5. . . . . . .2. . . . . . . . . .2.1. . . .2. . Why Processes Are Needed . . . . . . . .6-2 6. .2. . . . . .3. . . .6-11 6. . . . . . . . Process Planning . . . . .6-7 6. . . . . . . .3. . . . . . . . . . . . . . . . . . . . . . . . .7. . . . . . . . . . . . . . . . Process Inventory . . Process Improvement Teams . . Process Management Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How Processes Are Managed .6-21 6. 6. . . . . . . . . . . . .1.6-5 6. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2. . . . . . . . . . . . . . . . . . . . . . 6-22 6. . . .2. . . . .2. . . . . . .2. . . . . . . . . . . . .4. . . . . . . . . . . . . . . 6-18 6. . . . . . . . . . . . . . . . Check Processes Continuum . . . . .2. . .1. . . . . . . . . . . . . . Check Processes . . . . . . . . . .1. . .6-17 6. . . . . . . . . . . . . . . . . . . . 6. . . . . .3. . . .2. . . . . . . . .1. . .1. . .1. . . . . . Process Definition . . 6-13 6. . . . . . . . . . . .3. . . . . . . . and Improve Work Processes . . . . . . . . . . . . .1. . . . . . . . . . . 6-12 6. . . . The Process Maturity Continuum . . . . . .2. . . . . . .1. Identify Control Points . . Customer Involvement Continuum . . . . . Process Measurement . . . . . . . . .6-3 6. . . . .

. . . . . . . . . . . .2 . . . . . . . . . . . . . . . .2. . . . . 7. . . . . . . . . . . . . . . Thread .1. . . . . . . . . . . . 7-10 7. . . . . . .4. . .4. . . . . . . . . .3.5. . . . Testing Concepts . . . . . . . . . . . . . . . . . . .2. . . . . . . . . .5. . 7-10 7. . . . . . . 7-3 7. . . Test Objectives . Independent Testing . . . . Regression . . . . . . . . . . . . . . .3. . . . . . . . . . . . . . . Feasibility Reviews . . . . . . . . . . . . .2. . . . . . . . . . Stress versus Volume versus Performance . .1. .1. .1. . . . . . . . . . . . . . . . . White-Box .7-1 7. .1. . . . . . . . . . . . .1. . . . . 7.1. . . . . . . . . . . . . . . . . . 7. . 7-9 7. . . . . . Verification versus Validation . . . .9. The Testers’ Workbench . . . . . Phase-End Reviews . . .6. . . . . . . . . . . . . . . 7-11 7. . . . . . . . . 7-5 7. . . . . . .1. . . . . . .1. . . . . . . . . . . . . . Requirements Reviews . . . 7-11 7. . . . . . . . . . . . . . . .6. . . . . . . . . . 7. . . . . .1.2. . 7-12 7. . . . . 7. . . . . 7-10 7. . . . .4. . . . . . . . . . . . .3. . . . .3. . . . . . . . . . . . . . . . . . . . . Verification Techniques . . . . . . . . . . . . . . . . . . . . . .1. . 7-12 7. . . . . . . . . .2. .1. . . . . . . . . . . . . . Test Stages . . . . . . . . . . . . . . . . . .1.3. . . . . . . . . . . . . 7-2 7. . . . . System Testing . . . . . . . . . . . . . . . . . . . . . Requirements Tracing . . Integration Testing . . . . . . . . . . . . . . . . .3. . .9. . . .2.1. . .2.9. . . . . . . . . . . . . . . . . Verification and Validation Methods . . . . . . . . .1.Guide to the CASQ CBOK Skill Category 7 Quality Control Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Management of Verification and Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7. . .2. . . . . . .2. . Code Walkthroughs . . . . . . . . . . . 7-13 7-13 7-13 7-13 7-14 7-14 7-14 7-15 7-16 7-16 7-16 7. . . . . . . . . . . Code Inspections or Structured Walkthroughs . . .1. . . . . . . . . . . . . . .1. . . . . . . . . . . . . . .5. . . . . . . . . .4. . . . .2. . . . . . Static versus Dynamic Testing . . . . . . . . . . . . . . . . . . . . .1. 7-13 7. . . . .2. . Black-Box . .2. . .8. .2. . . . . .6. . . . . . . . .2. . . Reviews and Inspections .5. . . . . . . . . . . . . . .1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2. . . . . . . . . . . . . . . . . . . . . . . 7-3 7. . . . . . .3.2. . .3. . . . . . . . . . . . . . . . . . .1. . . . . . . . . . . . . . .1. . . . . . . . . . . Checkpoint Reviews . .3. . . .2. . . . . . . . . . . 7-9 7. . . . .9. . . . .2. . . . . . . . . . . . . . . . . . . . . . . . Design Reviews . . . . . . . . . . .1. . . Post-Implementation Reviews . . . . . 7-8 7. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7. . . . . .9. . . . . . . . . . . . . . . . . . Inspections . . . . . . . . . . . . . . The Life Cycle Testing Concept Example . . . . . . . . . . . . . . . . . . . . . . . . . . 7-14 7. . . . . . . . . . . . . . . . . . . . . . .1. . . . . . . Structural and Functional Testing . . . 7-8 7. . . . . . . . . . . . . . 7. . . . . . . . . . .2. . . . .9. . .2.1. . . .1. . . . . . . . . . . . . . . . . . . . . . . 7-6 7. . . . . 7-4 7. . . . . . . . . . . . . . . User Acceptance Testing . . . . . . . . . . . . . . . . . . . . .2. . . . . . . . . 7. . . . .2. . . . . . . . . . . . .1. . .2. . . . . . . . . . . . . . . . . . . . .4. . . . . . .2. . . . . .2. . . . 7-1 7. . . . . . .2. . . . . . . . . . .3. . . . . . . . . . . . . . . . 7-4 7. Computer System Verification and Validation Examples . 7-17 8 Version 6. . . Incremental . . . . . . Validation Techniques . . . . . . . .9. . . . . . . . . . . . . . . . . . . . . 7. . . . . . . 7-1 7. . . . . . . . . . . . . . . . .5.7. . . .2.2. . 7-2 7. . . . . . . . . . . Review Formats . . . . . . .3. . . .2. . . . . . . . . In-Process Reviews . . . . . . . . . . . . . . . . . . 7-2 7. . . . . .2. . . . . . .1. . . . . . . . .2. . . . .

. . . . . .4. . . .2. . . . . . . .4. . . . . . . . . Structural Testing .2. . . . . . . .7-20 7.4. . . . . .2. . 7-18 7. . . . .2. . . . . . . . . . . . . .3. . . . . .1. . . . Defect Management Process . .2 9 . . .1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Software Configuration Management . . . . . . Defect Management . . . . . . . . . . . . .4. . . . Severity versus Priority . . . . . . . . . . . . . . . . . . . .4. . . . . . . . . Software Change Control . . . . . . Defect Reporting . . .3. . . . . . . . . . . . . . . .7-18 7. . . .4. . . . . . . . . . . . .7-20 7. . . . . . . . . . . . . . . . . . . . . . . . .7-21 Version 6. . . . . . . . . . . . . . . . . . . . . .2.7-19 7. . . . . . . .4. .3. . . . . .7-20 7. Change Control Procedures . . . .7-18 7. . . .4. . . . . . Using Defects for Process Improvement . . . . . . . . . . . .1. . 7-18 7. . . . . . . . . . . . . . .Table of Contents 7. . . . . . . . . .3. . . . . . . . . . . .7-21 7. . . . . . . Functional Testing . . . . . . . . . . . . .

. . . . .1. . . . . . . . . . .1. . . . . . 8. . . . . . . . . . . .6. . . . . . . . . . . . . . . .2. . . . . . . . . .7. . . . . . . . . . . . . . . . . Process Measurement . . . Variation and Process Capability . . . . . . . . . . . . . . . . .6. . . .2. . . Ease of Use and Simplicity . . . . . . . . . . . . . . . . . . . . . . .7. . .3. . .2.8-1 8. Special Causes of Variation . . . . . . . . . . . . . . .1. . 8-14 8. .1. . . . . Interval Data . . . . . . . . . . . . . . . . . . . . . . . . . . .1. . . . .3.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-7 8. . . . . .4. . . . . . . . . . . . . Variation and Process Improvement . . . . .1. . . . . . . . .1. . . . . .1. . . . . . . . . . . . . . Calibration . . . . . . . . . . .4. . . . . . . . . . Types of Measurement Data . . . . . . . .1. . . . .3. . . . . . .4. . Validity . . .2. . . . . . . .7.1. . . . . . . . . . . Ratio Data . .1. .5. . . . . . . . . . . . . . . . . . . . . . . . . 8. . . Metrics . . . . . Attributes of Good Measurement . . . . . . . . . . . . .2. . . . . . . . . . . . Complexity . . 8-8 8. . . . . . . . . . . . . . . . . 8-2 8. . . . . . .4. . . . . . . . . . . . . . . . . . . 8-9 8. . .2. . Customer Perception of Product Quality . . . . . . . . . . . . . . . . . . Measurement Concepts . . . . . . . . .1. . . . . 8. . . . . . . . . . .3. 8-6 8. Quality .Guide to the CASQ CBOK Skill Category 8 Metrics and Measurement . . . . . . . . . . . . . . . . . . . . . . .2.8. . . . . . . . .1. 8-2 8. . . . .6. . . . . . . . . . . . . . . . . .2. . . . . . . . .2. . .1. . . . . . . . . .1. . .3. . . . . . . . . . . 8-13 8. . . . . . . . . . . . . . 8-3 8. . . . . . . . . . . . . 8-10 8. . . . . . . . Common and Special Causes of Variation . . . . . . . . .2. .2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 . . . Measures of Central Tendency . 8-5 8. . . . . . . . . . . . . . . .1. . . . . . . . .3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3. . . . . . . .1. . . . . . . .1. . . 8-15 8. . . . . . .2. . . . . . . . . . Using Quantitative Data to Manage an IT Function . . .6. . . . . Common Causes of Variation . . . . . . . .3. . . . .4. . . . . . . . . . . . . . . . . . . . . . . . .1. . Product Measurement . . . . . .1. . . . . . . . . . . . . . . . . . . 8. .1. . . . . . . Measurement Dashboards . . . . . . . .3. . . Process Capability .1. . . . . . . . . . . . . . . . .1. . . . .4. . .4. . . . . . . . . . . . . . . . . . . . . . . . . .4. .1. . . . . . . 8-9 8. . . . . 8-3 8-3 8-4 8-4 8. . . . . . . . . . . . . . . . . . . . . 8. . . . . . . . . . . . . . . Size . . . Objective and Subjective Measurement . . . . . . . 8-5 8-5 8-5 8-5 8-5 8. . . . . . . . . . . 8-2 8. . . . . . . . . . . . . . . . . . . . . . Timeliness . . . . . . Standard Units of Measure . . . . . Measurement in Software . . . . 8-11 8. . . . . Statistical Process Control . . . . . . . . . . . . . . . . . . . . . . . . 8-11 8. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Measurement Program . . .1. Reliability . . . . . . . . . . . . . . .1.3. . .1. . 8-13 8. . . . . .1. . . . . . . . . Ordinal Data . . . . . . . . . . . . . 8-1 8. . . . . . . . . . . . . . . . . . . .1. . .5. . . . . .2. 8-4 8. . . . . . . . . . . .6. . .3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nominal Data .2. . . . 8-6 8. .3. . . . 8-16 10 Version 6. . . . . . .1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.4. . . . . . . . . . . . . . . . .2. . . . 8-5 8. . . . . . . . 8-4 8. . . . . . . . . . . . . . . .2. . . . . . . Key Indicators . . . . . . . . . . . . . . . . . . . . . . . . 8. . . . . 8-10 8. 8-8 8. . . . 8. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6. . .1. .

. . . . .3.4. .5. . . . . . . . . . . . . . . . . . . . . . .4. . .4. . . . . . . . .4. . . . . . . . . Magnitude Dependent . . . . .8-22 Version 6. . . . . . . .4. . .4. . . . .3. .2. . . . . . . .8-17 8. . . . . Defining Risk . . . . . . . . . Risk Management . . . . . . . . 8. . . . . .2. . . . . . . . . . . . . . . . . . .4. . . .4. . . . . . . . . . . . . . . . . 8. . . . . . . 8-17 8-17 8-18 8-18 8-18 8. . . . . . . . . . . . . . . . . . . . . . . .8-18 8. . . .8-17 8.8-21 8. . . . . . . . . . . . . . Prerequisites . . . . . . . . . . . . . . . . .1. . . . . . .2. . . . .8-21 8. . . . . . .4. . . . . . . . . . .1. . . . . . . . . . . . . .2. . . . . . . . . . . . . . . . . . Managing Risk . The Need for Measurement . . . . . . . . . Situational . . . .2. . . . . . . . . . . . Interdependent . . . . . . . . . . . . .2. . . . . . . . . . . .5. . . . . . . . . . . . . . . . . . . . . . . .8-20 8. . . . . . . . . . . . . . . . . . . . . . . . 8. . . . . . . . .5. . . . . . . . . . . . . . . . . . . . . . . . . .2 11 . . . Characterizing Risk . . . . . . .4.1. . . . . . . . . . . . . .2. . . . . . . . .8-19 8. . . . . . . . . . . . Software Risk Management . .2. . . . . . . . . . .4. . . . . . . . . . Time-Based . . . . .4. . . . .8-17 8.5. . Implementing a Measurement Program . . . . . . .Table of Contents 8. . . . . . . . . . Value-Based . . . . . . . . .5. . . . . . . . . . . . . . . . . . . . . . 8. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Risks of Integrating New Technology . . . . .4. . . .

. . . . . . . . .9-1 9. . Cost versus Benefit of Controls . . . . . . . . . IT Areas Where Security is Penetrated . . . . Establishing a Security Baseline . . . .Guide to the CASQ CBOK Skill Category 9 Internal Control and Security . . . . . . . . . . . . . . . . . . . . . . . 9-13 9. . . . . . . . . . . . . . . . . . Security Awareness Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-5 9.3. . . . . . . . . 9-5 9. .5. . . . . . . . . . . . . . 9-14 9. . . . . . .2. . . . . . . . . Building Internal Controls . Database Storage and Retrieval . . . . . . . . . . . . . . . . . . . Model for Building Transaction Processing Controls . . . Accidental versus Intentional Losses . . . . . . .1. . . . . . . . . . . . . . . . 9. .3. . . . . . . . . . . . . . . . . . 9-3 9. . . . . . .5. . .3. . . Risk versus Control . . . . . . . . . Building Adequate Security . The Internal Auditor’s Internal Control Responsibilities .5. . . . . . . . . . . . . . . 9-3 9. . . . . . . . .4. Using Baselines . . . . . . . . . . . . . . . . . . . .1. . . . . . . Principles and Concepts of Internal Control . . . . . .5. . . . .1. . Creating Baselines . . . . . . . . . . . . . .6. . . .5. . 9. . . . . . Corrective Controls . . . .1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Functional Vulnerabilities . . . . . . .1. .6. . . . . . . . . . . . . . . . . . . . . . Internal Control and Security Vocabulary and Concepts . . .5. . . . . Environmental or General Controls . . . . . . . .1. . . . . . . . . . . . . . . . . .6. .1.5. . . . . 9-13 9. . 9-8 9. . . . Transaction Entry . .1. . . 9. . . . . . . . . . .2. . . . . . . . . . . . Security Practices . . . 9-11 9-11 9-11 9-12 9-12 9-12 9. . . . . . . . . . . . . . . 9-10 9. . . . .1. . . . . . .2. . . . . . . . . .3. . . . . . . . . . .1. . . . 9-15 9. . . . . . . . . . . . . . . . . . . . .1. .1. . . . 9-6 9. . . . 9-9 9. . Transaction Origination . . . . . . . . . .6. . . . . . .2. .3. . Environmental versus Transaction Processing Controls . . . . . . . . . . . . . . . . . . .1. . . . . . . . . . . 9-8 9. . . . . . . . .6. . . . .6. . . . . . . . . . . .6. . . . . . . . . . . . . . . . . . . . .1. . . . . . .1. . .3. . Detective Controls . . . . . . . . . . . . . . . .2. . . . . . . . . . . . . . . . . . . . . . . .2. . . . . . . . . 9-14 9. . . . . . .2 . . . . . . . . .5. . . . .3.2. . . . . . . . .1. . . . . . . . . .1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-8 9. . . . . . . . . . Preventive Controls . . . . . . . . . . . . . . . . . . Internal Control Responsibilities .6. . . . . . . . . . . . .1. 9-6 9.2. . . . . . . . . . . . . . 9. . 9-16 12 Version 6. . . Transaction Processing Controls . . . . . . . . . . . . . . .1. . . . . . .4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6. . . . . . . . .5. . . . . . . . . . . Transaction Processing . Perform Risk Assessment . . . . . Detective and Corrective Controls . . . . . . . .3. 9-2 9. . . . . . . . . . . . 9. Where Vulnerabilities in Security Occur . . .3. . . . . . . . .2. . . .6. 9-5 9. . . . . . .2. . . . . . . . . . . . . 9-2 9.2. . . The Quality Professionals Responsibility for Internal Control and Security . .6. . . . . . . . . 9-9 9. . . . . . . . . . . Preventive. . . . .1. .5. . . . . .3. . . . . . . .2. 9-12 9. . . . . . . . . . . . .1. . . . 9-15 9. . . . . . . . . . . . .5. . . . . . .2. 9-12 9. . . . . . . . . . .1. . . 9-7 9. . . .2. . . . . . . . . . . 9-16 9. . .3. . . . . . . . . . . . 9-9 9. .2. . . . . . . . . . . Transaction Output . . . . .2. . . . . . . Transaction Communications . . .1. .4. . . . . . .

. . . . . . . . . . . .5. Contracting Life Cycle .2. . . .5. . . . . . . . . .10-5 10. .2 13 . .1. . . . . . Selecting Software Developed by Outside Organizations . . . . . . . . .2. .1. . . . Quality and Outside Software . . . . . . . .1. . Contractual Relations .1. . . . . . . . . .0.1. . . . . . . . . . .1. Operation and Maintenance of the Software . . . . . 10-9 10. . . . . . . . . . . .4. . . . . . . . . . . Contracting for Software Developed by Outside Organizations .2. . . . . . . . . .2. . . . . . Acceptance Testing Concerns . . . . . . .1. . . . . 10-2 10. . . . . . . . . .1. . A-1 Version 6. . . . . .3. . . . . Operating for Software Developed by Outside Organizations . . . . . . . . . . . . . . . Acceptance Testing . . . .10-8 10. . . .5. . . . . . . . . Selecting COTS Software . . . . . . . . . . 10-10 Vocabulary . .1.1. . .3. Additional differences if the contract is with an offshore organization . . . . . . . . . . . Evaluation versus Assessment . . . . . .10-3 10. . . Purchased COTS software . . . .2. . . . . . . . . .5. . . . .1. . . . .1. . . . . . . .5.10-6 10. . . .10-2 10.1. . . . . . 10-4 10. . . . . .2. . . . . . . . . Outsourced Software . . . . .10-8 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10-7 10. . . . .10-1 10. . . . 10-10 10. . . . . . . . . . . . . What Contracts Should Contain . . . 10-1 10. . . . . . . . . . . . . . . . . . .1. . . . . . . 10-3 10. . . . . . . . . . . . . . .4. . . . 10-7 10. . . . . . Quality Professionals Responsibility for Outside Software . . . . . . . . . . .3.10-6 10. . .1. . . . . . . COTS and Contracting Quality . . . . . . . . . .1. .1. . .Table of Contents Skill Category 10 Outsourcing. . . . . . . . . . . . . .

14 Version 6.2 .Guide to the CASQ CBOK This page intentionally left blank.

there must be a common understanding of what is meant by those terms. and identify opportunity for improvement. quality becomes a difficult program to achieve. concepts. For example.Skill Category 1 Quality Principles B efore an organization can begin to assess the quality of its products and services. Some of the more widely used terms are: Version 6. when the words “process” or “defect” are used. Until the vocabulary is learned and its use encouraged in the organization. and approaches used for improving quality.1 1-1 . which include the following: Vocabulary of Quality The Different Views of Quality Quality Concepts and Practices Quality Control and Quality Assurance Quality Pioneers Approach to Quality page 1-1 page 1-3 page 1-6 page 1-12 page 1-15 1. it first must have a working knowledge of quality principles and basic concepts. This terminology is also referred to as the vocabulary of quality. This category tests the CSQA candidate’s ability to understand and apply these principles. Appendix A provides a glossary of definitions for terminology used in the quality language.2.1 Vocabulary of Quality The quality language is the way quality professionals describe the principles.

a defect is a product requirement that has not been met. Productivity The ratio of the output of a process to the input. a product is a quality product if it meets or conforms to the statement of requirements that defines the product. or a product attribute possessed by a product or a function performed by a product that is not in the statement of requirements that define the product. Policy Managerial desires and intents concerning either processes (intended objectives) or products (desired attributes). The producer’s view of quality has these four characteristics: Doing the right thing. whether in the statement of requirements or not. • 1-2 Version 6. and Doing it on time without exceeding cost. quality is further defined as follows: Quality – Producer View To the producer. A product is a quality product if it is defect free. a defect is anything that causes customer dissatisfaction. This is referred to as fit for use. Quality – Customer View To the customer. and improve its software related activities. This includes efforts of people and equipment guided by policies.1 . and procedures. It is frequently useful to compare the value added to a product by a process. Quality Operationally. manage. This statement is usually shortened to: quality means meets requirements. From the customer's viewpoint.2. Process The work effort that produces a product. usually measured in the same units. a product is a quality product if it meets the customer’s needs. Doing it the right way. execute. A process or set of processes used by an organization or project to plan. standards. Doing it right the first time. the word quality refers to products. • A statement of purpose and an essential set of practices (activities) that address that purpose.Guide to the CASQ CBOK Defect From the producer's viewpoint. monitor. Procedure The step-by-step method followed to ensure that standards are met. control. For a CSQA professional. to the value of the input resources required (using fair market values for both input and output). regardless of whether the requirements were met.

courtesy and respect In addition to the producer and customer views of quality. This is the view of the organization responsible for the project and processes. The customer’s view of quality has these characteristics: • Receiving the right product for their use • Being satisfied that their needs have been met • Meeting their expectations • Being treated with integrity. such as an independent vendor) that provides either the producer and/or the provider with products and services needed to meet the requirements of the customer.1 1-3 . W. Meeting requirements means that the person building the product does so in accordance with the requirements. The producer’s view of quality has these four characteristics: • Doing the right thing • Doing it the right way • Doing it right the first time • Doing it on time without exceeding cost Being fit for use is the customer’s definition.2. Joseph Juran and Dr. These views are as follows: Provider view – This is the perspective of the organization that delivers the products and services to the customer. Of the two definitions of quality. • Supplier view – This is the perspective of the organization (that may be external to the producer’s company. who uses them to create the • Version 6. The customer is the end user of the products or services.Quality Principles Standard A requirement of a product or process. fit for use. Requirements can be very complex or they can be simple. developed. so it can be determined whether they have been met. The infrastructure for quality products and services is illustrated in Figure 1-1. and the products and services acquired. These two definitions are not inconsistent. is the more important. Meeting requirements is a producer’s view of quality.2 The Different Views of Quality Industry accepted definitions of quality are “conformance to requirements” (from Philip Crosby) and “fit for use” (from Dr. Fit for use means that the product or service meets the customer’s needs regardless of the product requirements. Edwards Deming). For example: 100 percent of the functionality must be tested. 1. the organizational infrastructure also includes a provider and a supplier view. but they must be defined in a measurable format. and maintained by those processes. The figure shows the requirements coming from the customer to the producer/provider.

and there may be a long chain of producers/providers and their customers.1 . Closing the producer's gap enables the IT function to provide its customers consistency in what it can produce. In reality. Figure 1-1 Infrastructure for Software Quality Products and Services This infrastructure has been presented simplistically. The quality function must first improve the processes to the point where the producer can develop the products according to requirements received and its own internal standards. However. the producer is the customer for the supplier. the quality characteristics by which an interim producer evaluates supplier products are really producer quality characteristics and not end user/customer quality characteristics.2.1 The Two Quality Gaps Most Information Technology (IT) groups have two quality gaps: the producer gap and the customer gap as shown in Figure 1-2. The customer gap is the difference between what the producers actually delivered versus what the customer wanted.Guide to the CASQ CBOK products and services needed by the customer. The producer gap is the difference between what is specified (the documented requirements and internal standards) versus what is delivered (what is actually built). that McDonald's has now produced consistency in its delivered product. This has been referred to as the "McDonald's effect" . 1. a Big Mac should taste the same. It doesn't mean that every customer likes the Big Mac or that it meets everyone's any McDonald's in the world. making the supplier the producer for the intermediate producer. but rather. Closing these two gaps is the responsibility of the quality function (see Skill Category 4). 1-4 Version 6. This process works because of the two-way measurement process established between the involved parties.2.

and as a basis for measuring quality. keeping consistency while producing products and services needed by the customer. Version 6.2 Quality Attributes for an Information System Quality is a multifaceted concept driven by customer requirements. Management needs to develop quantitative. The level of quality can vary significantly from project to project and between organizations. In IT. and more user involvement through the process of building information products.2. management must decide the degree of maintenance effort that is acceptable. Joint Application Development (JAD) sessions.2.Quality Principles Closing the second gap requires the quality function to understand the true needs of the customer. measurable "standards" for each of these quality criteria for their development projects.1 1-5 . Some of the commonly accepted quality attributes for an information system are described in Figure 1-3. For example. This can be done by customer surveys. the amount of time that it should take for a user to learn how to use the system. the attributes of quality are examined in order to understand the components of quality. etc. Figure 1-2 Two Quality Gaps 1. Skill Category 8 covers this topic in more detail. The processes can then be changed to close the customer gap.

Past Executive Vice President Paul Revere Insurance Group 1. Effort required modifying an operational program. (This is confusion between quality of design and quality of • 1-6 Version 6. Effort required locating and fixing an error in an operational program.Guide to the CASQ CBOK Attributes Correctness Reliability Efficiency Integrity Usability Maintainability Testability Flexibility Reusability Definition Extent to which a program satisfies its specifications and fulfills the user’s mission objectives. however. Soule. The amount of computing resources and code required by a program to perform a function. Extent to which a program can be used in other applications – related to the packaging and scope of the functions that programs perform." Charles E. their actions may not support this view for the following reasons: • Many think that defect-free products and services are not practical or economical. Interoperability Figure 1-3 Commonly Accepted Quality Attributes (Critical Success Factor) for Information Systems "The Paul Revere Insurance Group believes that if a customer does not perceive quality. operating. (This is called acceptable quality level. Effort required to couple one system with another.) Quality experts agree that AQL is not a suitable definition of quality. preparing input.3 Quality Concepts and Practices People say they want quality. or AQL. meaning that high quality is synonymous with high cost.1 . Extent to which access to software or data by unauthorized persons can be controlled. Effort required learning. and interpreting output of a program. Quality is frequently associated with cost.2. Extent to which a program can be expected to perform its intended function with required precision. the entire quality program will be in jeopardy. Effort required testing a program to ensure that it performs its intended function. the program is not accomplished. As long as management is willing to "accept" defective products. and thus believe some level of defects is normal and acceptable.

or lack of knowledge about quality. for quality to happen there must be welldefined standards and procedures that are followed. • Quality by definition calls for requirements/specifications in enough detail so that the products produced can be quantitatively measured against those specifications.e.e. naming of defects by type) • No information on the occurrence of defects by type.. Skill Category 2 focuses on management commitment and a quality management environment. and by location • Unknown defect expectation rates for new products • Defect-prone processes unknown or unidentified • Defect-prone products unknown or unidentified • An economical means for identifying defects unknown • Proven quality solutions are unknown and unused If achieving quality (i.e.. Following are some of the specific contributors for these two categories: Lack of involvement by management • Management's unwillingness to accept full responsibility for all defects • Failure to determine the cost associated with defects (i. by frequency.” Thought revolutions do not come easy. Ishikawa. The bottom line is that making quality happen is a monumental challenge.. as they do not see an immediate payback.. and thus do not strive for compliance to standards. Quality is very difficult to accomplish – it requires the close cooperation of management and staff. poor quality) • Failure to initiate a program to "manage defects" • Lack of emphasis on processes and measurement • Failure to enforce standards • Failure to reward people for following processes Lack of knowledge about quality Lack of a quality vocabulary.1 1-7 . Dr. what is necessary to make it happen) • No categorization scheme for defects (i. defect-free products and services) were easy.Quality Principles conformance. • Version 6. However. best expressed this when he stated that accomplishing quality requires “a thought revolution by management.) Organizations may be reluctant to spend on quality assurance.e. The contributors to poor quality in many organizations can be categorized as either lack of involvement by management. it would have been accomplished years ago. • Many technical personnel believe that standards inhibit their creativity. Few organizations are willing to expend the effort to produce requirements/specifications at the level of detail required for quantitative measurement. Japan’s leading quality expert. which makes it difficult to communicate quality problems and objectives • Lack of knowledge of the principles of quality (i. Achieving quality requires a commitment and the establishment of an environment in which quality can flourish.2.

Check. which must be continually nurtured by the quality function and management.Define the objective. Boehm's estimates are conservative. Recent studies show that with today's more complex systems. Studies by Dr.Create the conditions and perform the necessary teaching and training to ensure everyone understands the objectives and the plan. TRW. W. • 1-8 Version 6. which was developed in the 1930s by Dr. if possible. Figure 1-4 PDCA Concept • Plan (P): Devise a plan . Barry W.1 PDCA Cycle A major premise of a quality management environment is an emphasis on continuous improvement." What this means is that preventing and/or detecting defects early results in huge savings. Dr. and is one of the key concepts of quality. "Can we afford quality?" is: "You cannot afford to ignore it. Clearly describe the goals and policies needed to attain the objective at this stage. Twenty years might be excessive. Boehm concluded that errors are typically 100 times more expensive to correct in the maintenance phase on large projects than in the requirements phase. The approach to continuous improvement is best illustrated using the PDCA cycle.Guide to the CASQ CBOK As a result of his experiences in turning around the Japanese economy. The answer to the question. and Act as shown in Figure 1-4. and IBM in the late 1980s showed geometric escalation in the cost to fix a problem as the software life cycle progressed. expressing it numerically. Determine the procedures and conditions for the means and methods that will be used to achieve the objective. but management must be prepared to invest 2-5 years before the really large pay-backs occur." Harold S.3. 1. The cycle comprises the four steps of Plan. Geneen. Boehm also stated that the total economic impact is actually much larger in operational systems because of the user costs incurred. Boehm at GTE. Shewhart of the Bell System. stated that quality "is the most profitable product line we have. Do. past CEO at ITT. Quality is a long-term strategy. It is also called the Deming Wheel.2. Edwards Deming found that it takes 20 years to change a culture from an emphasis on productivity to an emphasis on quality. Do (D): Execute the plan .1 .

changes in conditions. Look for the cause of the abnormality to prevent its recurrence.If the check reveals that the work is not being performed according to plan. 1. if testing and rework components of a system development process were eliminated or reduced. Act (A): Take the necessary action . is to lower or not meet quality standards. Productivity is an attribute of a process. or abnormalities that may appear.3. This concept can be seen in the ascending spiral of Figure 1-5. For example.2 Cost of Quality Quality is an attribute of a product or service.As often as possible. check to determine whether work is progressing according to the plan and whether the expected results are obtained. and that the anticipated budget and delivery date are fulfilled.2. devise measures for appropriate action. The first. Then perform the work according to these procedures. This is often done under the guise of completing projects on time. Check for performance of the procedures. Check (C): Check the results . or if results are not what were anticipated. • • Figure 1-5 Ascending Spiral The PDCA procedures ensure that the quality of the products and services meets expectations. Repeatedly going around the PDCA circle can improve the quality of the work and work methods. They have frequently been called two sides of the same coin because one significantly impacts the other. The next plan should reflect these changes and define them in more detail.Quality Principles Teach workers the procedures and skills they need to fulfill the plan and thoroughly understand the job. The second and more desirable method to improve productivity through quality is to improve Version 6. Sometimes preoccupation with current concerns limits the ability to achieve optimal results. There are two ways that quality can drive productivity. productivity as measured in lines of code per hours worked would increase. Sometimes workers may need to be retrained and procedures revised. which is an undesirable method.1 1-9 . and obtain the desired results.

it cannot dictate quality for the organization. not management. • • "Quality is free. such as the cost of operating faulty products." Philip B.2. Since this situation does not occur.Money required preventing errors and to do the job right the first time is considered prevention cost.Guide to the CASQ CBOK processes so that defects do not occur. Prevention money is all spent before the product is actually built. but it is not a gift. Failure – Failure costs are all costs associated with defective products. Quality cannot be delegated effectively. thus minimizing the need for testing and rework. damage incurred by using them and the costs incurred because the product is not available. Only management can make quality happen. there are costs associated with getting a defect-free product produced. testing and reviews. The quality function assists management in building quality information systems by monitoring quality and making recommendations to management about areas where quality can be improved. As the quality function is a staff function. Others are costs generated by failures. Management must accept the responsibility for the quality of the products produced in their organization. quality will not happen. This money is spent after the product or subcomponents are built but before it is shipped to the user. Appraisal includes the cost of inspections. The user or customer of the organization may also experience failure costs. but senior management must emphasize and initiate quality improvement. and then move it down through the organization to the individual employees. If every worker could produce defect-free products the first time. There are three COQ categories: • Prevention . The following three quality principles must be in place for quality to happen: 1. A quality function is only a catalyst in making quality happen. Management is responsible for quality. Some failure costs involve repairing products to make them meet requirements. training workers and planning for quality.1 The Three Key Principles of Quality Everyone is responsible for quality.3. 1-10 Version 6. Appraisal – Appraisal costs cover money spent to review completed products against requirements.1 . otherwise.2. Quality improvement should be used to drive productivity. Crosby in Quality is Free 1. The cost of quality (COQ) is the money spent beyond what it would cost to build a product right the first time. This category includes money spent on establishing methods and procedures. the COQ would be zero.

there will likely be barriers to success. Producers must use effective quality control.4 will be defective.3 Six Sigma Quality Most people spend twelve or more years in an educational system in which grades of 90% or higher. The end objective of the quality process must be satisfied customers. “Sigma” is a statistical term meaning one standard deviation.3. Quality is a journey. Version 6. to do a task correctly.6 will be correct. you have a 90% quality rating. but that is totally unacceptable to tire customers. the following characteristics are desirable. only 3. Experience has shown that in most systems. The key focus of companies implementing a Six Sigma program is to develop a good business strategy that balances the cost. At the Four Sigma level there are 6. as opposed to percent performed correctly. • The project should clearly connect to business priorities. in industry. 90% is not a good quality record.2. For example.4 items per million are outside of the acceptable level.3. and to consider measuring output of a process over time (not just a snapshot). or by an independent assessment.2 Best Practices A practice is a specific implementation of a work process.2. features and availability considerations for products.996. 1.1 1-11 . A valuable lesson learned is that decisions made must be tied to the bottom line for the company. etc. Companies should take care to use correct measurements for each situation. Motorola developed a concept called “Six Sigma Quality” that focuses on defect rates. At the Six Sigma statistical level. 1. not a destination. if one out of every ten tires fails.Quality Principles 2.120 defects per million parts. The objective of the quality program must be continuous improvement. Thus. However. When considering a project to improve using Six Sigma. quality. Deming Prize. and no more than 3. or about 6 defects per 1. If one or more of these characteristics is missing. a Four Sigma quality level is the norm. A Best Practice is one of the most effective practices for performing a specific process. the Six Sigma quality level means that out of every one million items counted 999. are considered excellent. “Six Sigma” means six standard deviations. All of the parties and activities involved in producing a product must be involved in controlling the quality of those products. Best Practices are normally identified by benchmarking. 3. This means that the workers will be actively involved in the establishment of their own standards and procedures. For example. a practice would be one organization’s process for estimating the amount of resources required for building a system.000 opportunities. Best Practices are also identified through winners of quality competitions such as the Malcolm Baldrige National Quality Award.

Management should support and approve the project to ensure resources are available. However. The project should have clear. This section differentiates between the two.4 Quality Control and Quality Assurance Very few individuals can differentiate between quality control and quality assurance. The project should have a limited scope that can be completed in less than six months. Most quality assurance groups. including types of benchmarking and a four-step process for conducting benchmarking. process for performing a work task to another organization’s process.1 . the number of defects currently contained in a thousand lines of code is usually calculated quantitatively and can be used to measure improvement. 1-12 Version 6. service or support process. 1.4 Baselining and Benchmarking Baselining is defining the current level of performance. or one part of an organization’s. 1. Benchmarking involves a comparison of one organization’s.5 Earned Value It is important that quality professionals be able to demonstrate that their work provides value to their organization. and that the project continues over time. ROI is not designed to measure subjective values such as customer loyalty. practice quality control.3. barriers are removed. It is recommended that quality professionals use the method(s) recommended by their accounting function for calculating earned value.2. in fact. such as a 50% process improvement.Guide to the CASQ CBOK • • • • • The problem being solved should be very important. Skill Category 3 provides additional details on benchmarking. For example.3. Return-on-investment (ROI) that demonstrates the dollars returned for the dollars invested is one of the more popular means to demonstrate value returned. There is no generally accepted best way to measure the “value earned” from quality initiatives. 1. and describes how to recognize a control practice from an assurance practice. The importance of the project should be clear to the organization. for the purpose of finding best practices or competitive practices that will help define superior performance of a product. quantitative measures that define success.

also contain attributes.2. Impediments to QC include the following: • Quality Control is often viewed as a police action • IT is often considered an art • Unclear or ineffective standards and processes • Lack of process training Quality Control is the focus of Skill Category 7.Quality Principles Quality means meeting requirements and meeting customer needs. such as meetings with customers. such as a requirement document. they can "assure" that the same level of quality will be incorporated into each product produced by that process. 1. Quality is an attribute of a product. source code. Ideally the same group that builds the product performs the control function. and therefore. an agenda might be a quality attribute of a meeting. Once processes are consistent. Processes have the advantage of being able to replicate a product time and time again.4. QC uses reviews and testing to focus on the detection and correction of defects before shipment of products. Services are a form of products. Both quality control and quality assurance are used to make quality happen. Version 6. however. A product is something produced. Of the two. 1. the process is able to replicate similar products with the same quality characteristics. measurement and analysis) needed to provide adequate confidence that processes are established and continuously improved in order to produce products or services that conform to requirements and are fit for use. Quality Control should be the responsibility of the organizational unit producing the product and should be integrated into the work activities. which means a defect-free product from both the producer’s and the customer’s viewpoint. test data. help desk activities and training sessions. For example. load module or terminal screen. Quality Assurance (QA) is associated with a process. A process is the set of activities that is performed to produce a product. Quality is achieved through processes. some organizations establish a separate group or department to check the product.2 Quality Assurance Quality Assurance (QA) is the set of activities (including facilitation.1 1-13 . quality assurance is the more important.4. and the action taken when a nonconformance is detected. training.1 Quality Control Quality Control (QC) is defined as the processes and methods used to compare product quality to requirements and applicable standards. Another type of product is a service that is performed. Even in data processing.

1-14 Version 6.1 . The following statements help differentiate QC from QA: • • • • • QC relates to a specific product or service. and so forth. Thus. in a specific product or service. QA would measure them to find weaknesses in the process and then correct those weaknesses to continually improve the processes. If there is no process.3 Differentiating Between Quality Control and Quality Assurance QC is an activity that verifies whether or not the product produced meets standards. QC verifies whether particular attributes exist. or do not exist. The major impediments to QA come from management. It is possible to have quality control without quality assurance. and acquire or help install system development methodologies. and sees little need for a function that emphasizes managing and controlling processes. many of the impediments to QA are associated with processes. there is no role for QA. Regardless of whether a program is produced using a system development process or done by an individual without a process. Assurance would determine the need for. and encourage quality attitudes and discipline on the part of management and workers. there might be a standard that “ALTER GO TO” statements in COBOL should not be used. Successful QA managers know how to make people quality conscious and to make them recognize the personal and organizational benefits of quality. which is typically results oriented. it could still be checked to determine whether or not “ALTER GO TOs” are in the program. estimation processes. QA is an activity that establishes and evaluates the processes that produce the products. QC identifies defects for the primary purpose of correcting defects.2. QC is the responsibility of the worker. system maintenance processes. QA helps establish processes. It is also a catalytic function that should promote quality concepts.4.Guide to the CASQ CBOK QA is a staff function that prevents problems by heading them off. Once installed. For example. and include the following: • • • • • • • • Management does not insist on compliance to processes Workers are not convinced of the value of processes Processes become obsolete Processes are difficult to use Workers lack training in processes Processes are not measurable Measurement can threaten employees Processes do not focus on critical aspects of products 1. and by advising restraint and redirection at the proper time.

2. quality should be the cornerstone of the corporation. QA is concerned with all of the products that will ever be produced by a process. Juran is well known for his trilogy and distinction between “little-Q” quality and “big-Q” quality. satisfaction of the user.Quality Principles • • • • • • • QA sets up measurement programs to evaluate processes. Deming defined 14 principles for quality. Additional information can be found in his book Out of the Crisis (see the References in Appendix B). frequently performed by a staff function. Within IT this can be translated to mean that the training department teaches the standards. Create consistency of purpose for improvment of product and service In quality-oriented companies. education and maintenance. W. QA personnel should not ever perform quality control unless doing it to validate quality control is working. Deming and Mr. 1. These three are Dr. which formed the basis for the turnaround of the Japanese manufacturing industry. All units within the organization should work toward common goals and purposes. The 14 principles are discussed briefly below. Both Dr. or have a following. W.5 Quality Pioneers Approach to Quality Many individuals have contributed to the quality movement. and systems programming is dedicated to improving application performance.5. training and retraining of personnel. operations is working for the same goals as systems programming. QA is a management responsibility. Crosby have developed a set of quality principles.1 1-15 . Dr. He believed that all 14 principles must be used concurrently to make quality happen. Three individuals are highlighted here because they have either organized a business to promote their concepts. 1. QA evaluates whether or not quality control is working for the primary purpose of determining whether or not there is a weakness in the process. Joseph Juran. 1. Edwards Deming Dr. Assuring quality involves: • Innovating new approaches and allocating resources for long-term planning.1 Dr. new skills required. • Version 6. Edwards Deming. QA identifies weaknesses in processes and improves them. QA is sometimes called quality control over quality control because it evaluates whether quality control is working. Philip Crosby and Dr. Putting resources into research. such as possible new service.

1 . whether it be satisfactory or not. This obligation never ceases. Institute training on the job Training must be totally reconstructed. Improve constantly and forever the system of production and service Constantly improving the system will improve quality and productivity. The aim is to minimize total cost. End the practice of awarding business on the basis of price alone Require meaningful measures of quality along with price. 6. Acceptance of defective systems and poor workmanship as a way of life is one of the most effective roadblocks to better quality and productivity. and provide better training there. 4. but from improvement of the processes used to develop a product or service. by which a project leader calls the worker's attention to every defect or to half of them. Statistical methods are vital aids to the project leader to indicate whether the fault lies locally or in the system. Cease dependence on mass inspection Quality does not come from inspection. Further training cannot help. defects. and inadequate and ineffective supervision. drastic reduction in the number of vendors with whom they deal. but leadership. Requiring statistical evidence of quality control in the purchase of hardware and software will mean.Guide to the CASQ CBOK • Constantly improve the design of products and services. • • • Leaders must know the work that they supervise. is aleady in the product. and thus constantly decrease costs. Most people in management do not understand that the system (their responsibility) is everything not under the governance of a user. Adopt and instuitute leadership The job of management is not supervision.2. A person once hired and trained and in statistical control of his/her own work. in most companies. antiquated methods of training on the job. 3. Eliminate the need for inspection on a mass basis by building quality into the product in the first place. Statistical methods must be used to learn when training is finished. Adopt the new philosophy We are in a new economic age. good or bad. and when further training would be beneficial. material not suited to the job. may be wrong – is certainly wrong in most organizations – and defeats the purpose of supervision. If their work is not satisfactory. failure of management to understand the problems of the product in use. 5. 8. people on the job that do not know what the job is and are afraid to ask. We can no longer live with commonly accepted levels of mistakes. not merely initial cost. 7. 2. can do no better. move them to another job. Drive out fear 1-16 Version 6. Inspection does not improve quality as the quality. The usual procedure.

One common result of fear is seen in inspection. These devices are management's lazy way out. for better quality and productivity. and posters for the work force that urges them to increase productivity. and even in management positions. 9.2. it is not clear to them how to find out.Quality Principles Most people on a job. exhortations. Version 6. Institute a vigorous education and self-improvement program for everyone What an organization needs is not just good people. ensuring that the workes have the means necessary to do thier jobs effectively and efficiently and the abolishment of the annual merit rating and of management by objective. It is necessary. Many of them are afraid to ask questions or to report trouble. The responsibility of supervisors must be changed from stressing sheer numbers to quality. do not understand what the job is. it needs people that are improving with education. Substitute aids. This means. Advances in competitive position will have their roots in knowledge. helpful leadership and use statistical methods for continual improvement of quality and productivity. slogans. Another related aspect of fear is the inability to serve the best interest of the company through necessity to satisfy specified rules. that people feel secure. Moreover. Break down barriers between departments People in user areas must work as a team and learn about the problems encountered with various technologies and specifications in system design and operation. as the bulk of the causes of low quality and low productivity belong to the system and thus lie beyond the power of the work force. Remove barriers that rob people of pride of workmanship Remove barriers that rob people (including those in management and engineering) of their right to pride of workmanship. Why not have users spend time in the IT department to see the problems and hear about them? 10. exhortations. Otherwise. Eliminate slogans. such education should keep up with changes in technology and methods and. among other things. They indicate desperation and incompetence of management. 11. There is a better way. therefore. or what is right versus wrong. 12. there will be losses in production from necessity of reruns and from attempts to use systems unsuited to the purpose. 13. or to satisfy a production quota. or only numbers? Do they help anyone do a better job? Eliminate work standards that prescribe numerical quotas for the workforce and numerical goals for people in management. The economic loss from fear is appalling. An inspector incorrectly records the results of an inspection for fear of exceeding the quota of allowable defects.1 1-17 . or to cut costs by some specified amount. Posters and slogans never helped anyone do a better job and numerical goals often have a negative effect through frustration. Eliminate numerical quotas for the workforce Do your work standards take account of quality. Such things only create adversarial relationships. and targets for the work force Eliminate targets.

It is necessary to consistently produce conforming products and services at the optimum price. 1-18 Version 6. their only purpose is to warn management of serious situations. 1. Management Commitment Clarify where management stands on quality. sales. Management must ensure that no one is exempt. 2. They should be used to identify specific problems needing corrective action. every function must participate in the quality improvement effort. After all. and their obligations to implement all of these principles. 3. which are broken down by operating areas of the plant.Guide to the CASQ CBOK if advantageous. it is useless. The degree of participation is best determined by the particular situation that exists.5. Take action to accomplish the transformation The transformation is everybody's job. Since every function of an operation contributes to defect levels. Since most companies have such systems. and others. These are: 1. and the quality department should report them. purchasing. quality control. new hardware. Create a structure in top management that will push every day on the preceding thirteen points. everyone has the opportunity to improve. and are keen to continue improving. It should be mentioned that unless this data is reported properly.2. The device to accomplish this is the use of defect prevention techniques in the operating departments: engineering. By comparing the rejection data with the input data. Clearly define top management's permanent commitment to ever-improving quality and productivity. and take action in order to accomplish the transformation. it is possible to know the rejection rates. The Quality Improvement Team They run the quality improvement program. However. Quality Measurement Communicate current and potential nonconformance problems in a manner that permits objective evaluation and corrective action. Basic quality measurement data is obtained from the inspection and test reports. it is not necessary to go into them in detail.1 . recruitment should not be based just on past records but look for people who are improving. Finally.2 Philip Crosby Philip Crosby has developed 14 steps for an organization to follow in building an effective quality program. manufacturing. 14.

By the time a company is ready for the quality awareness step. but because of the significance of it. Version 6. or wherever. Zero Defects Planning Examine the various activities that must be conducted in preparation for formally launching the Zero Defects (ZD) program . Goal Setting Turn pledges and commitments into action by encouraging individuals to establish improvement goals for themselves and their groups. Problems that are identified during the acceptance operation or by some other means must be documented and then resolved formally. Zero Defects is a revelation to all involved that they are embarking on a new way of corporate life. 5. from the board chairman down. See “Quality Concepts and Practices” on page 6 where COQ was defined and examples provided. Therefore. management must make sure it is conducted properly. will provide a clean energy flow into an organization-wide ZD commitment.Quality Principles 4. These steps. the supervisor must be given primary consideration when laying out the program.2.1 1-19 . it is not difficult. Working under this discipline requires personal commitments and understanding. Corrective Action Provide a systematic method of permanently resolving the problems that are identified through previous action steps. Therefore. placed on a schedule and assigned to members of the team for execution. ZD Day Create an event that will let all employees realize through personal experience. but individual classes are essential to make sure that they properly understand and can implement the program. The departmental representatives on the task team will be able to communicate much of the planning and concepts to the supervisors. The supervisor gives the individual employees their attitudes and work standards. whether in engineering. The supervisor. Since it is a natural step. Quality Awareness Provide a method of raising the personal concern felt by all personnel in the company toward the conformance of the product or service and the quality reputation of the company. 9. they should have a good idea of the types and expense of the problems being faced. is the key to achieving improvement goals. The quality measurement and COQ steps will have revealed them. it is necessary that all members of the company participate in an experience that will make them aware of this change. 10. 8. 6. computer programming. Supervisor Training Define the type of training supervisors need in order to actively carry out their part of the quality improvement program. 7. that there has been a change.The quality improvement task team should list all the individual action steps that build up to ZD day in order to make the most meaningful presentation of the concept and action plan to personnel of the company. sales. The Cost of Quality Define the ingredients of the COQ and explain its use as a management tool. About a week after ZD day.

but once the salary has been established. Primarily concerned with measurement and reporting. 13. Suggestion programs are some help. and fully 75% can be handled at the first level of supervision. Error-Cause Removal Give the individual employee a method of communicating to management the situations that make it difficult for the employee to fulfill the pledge to improve. This is not only because of the importance of the work itself but because those who submit work unconsciously draw a great deal of their performance standard from the professional evaluator. since the worker generates savings every time the job is done better or quicker. When the worker has stated the problem. Quality Councils Bring together the professional quality people for planned communication on a regular basis. It is vital for the professional quality people of an organization to meet regularly just to share their problems. Sometimes supervisors don’t listen anyway. People really don’t work for money. They go to work for it. Recognize their contribution publicly and noisily.2. Recognition Appreciate those who participate. their concern is appreciation. it is easy for them to become influenced by the urgency of activity in their work areas. One of the most difficult problems employees face is their inability to communicate problems to management. Try to get two goals from each area. These goals should be specific and measurable. feelings. 12. isolated even in the midst of many fellow workers. 11. with each other. The number of ECRs that save money is extremely high. but don’t demean them by applying a price tag to everything. 1-20 Version 6. Error-cause removal (ECR) is set up on the basis that the worker need only recognize the problem.1 . Sometimes they just put up with problems because they do not consider them important enough to bother the supervisor. but in a suggestion program the worker is required to know the problem and also propose a solution. Studies of ECR programs show that over 90% of the items submitted are acted upon.Guide to the CASQ CBOK individual supervisors should ask their people what kind of goals they should set for themselves. and experiences. Consistency of attitude and purpose is the essential personal characteristic of one who evaluates another’s work. the proper department in the plant can look into it.

The planning process should also attempt to avoid costly deficiencies. with everyone using the same feedback loop. • Quality Control Quality control takes place at all levels in the organization. and the occasional spikes (those representing special causes of variation) should be investigated.3 Dr. Dr. When the process is being designed. These are known as “The Juran Trilogy” or “The Quality Trilogy. Do it Over Again Emphasize that the quality improvement program never ends. processes must have numerical measures and adjustment capabilities. • Quality Improvement At some point in the quality control phase. the continuous loop of product deficiencies will be traced back to the planning process. Juran believed that managing for quality required the same attention that other functions typically receive. Typically quality control will be performed to prevent defects from worsening. and then incorporate those needs into the product and process designs. quality planning should identify customers and their needs.1 1-21 . In developing the process. control should be part of the planning process. 1.5.” • Quality Planning The purpose of this phase is to create a process that enables goals to be met. there will always be some acceptable (inherent) variation. he developed a trilogy consisting of three interrelated. such as rework.2. quality control and quality improvement. Juran believed that business processes presented a major opportunity for improvement. To ensure that adequate attention was given. He refers to this as “self-control”. He developed a structured approach for improvement. and optimize the company performance. It is necessary to construct a new quality improvement team. There is always a great sign of relief when goals are reached. which included the following list of responsibilities for senior managers that could not be delegated: Version 6. basic managerial phases/processes: quality planning. and the problems will be seen as an opportunity to improve. When products are produced from the process.Quality Principles 14. This phase occurs before the process is used to produce a product. Juran believed that in order to achieve control. Dr. the entire program will end at that moment. Joseph Juran Dr. Management should strive to give process users the capability of making the necessary adjustments to control the process. and problems will become less than originally planned. Improvements will be made to revise the process. and to let them begin again and create their own communications. it will not focus on the process. If care is not taken.

Prior to total quality management (TQM) there was only “little-Q”. Dr. such as a team of people and their manager improving a specific work process. appoint teams and provide facilitators • Provide training in how to improve quality • Review progress regularly • Recognize winning teams • Propagandize the results In addition to his Trilogy. Dr. make it part of every job description Create the infrastructure: establish a quality council. While the scope of “little-Q” quality is a specific departmental mission. but it has a limited scope and impact. Juran is also known for his distinction between “little-Q” quality and “big-Q” quality. which he called the narrow focus on quality. Juran referred to “big-Q” quality as the new focus on quality. the scope of “big-Q” quality emphasizes the coordination of various activities conducted in other functions and groups so that all plans contribute to achieving the organization’s goals for quality.Guide to the CASQ CBOK Create an awareness of the need and an opportunity for improvement Mandate quality improvement. select projects to improve. • • • 1-22 Version 6.2. An example of “big-Q” quality is cross-functional teams throughout an organization working to prevent problems.1 . “Little-Q” quality is considered important.

This category describes the management processes used to establish the foundation of a quality-managed environment: T Leadership Concepts Quality Management Infrastructure Quality Environment page 2-1 page 2-3 page 2-10 2. Quality management integrates fundamental management techniques. It is a culture change. Quality management is the application of quantitative methods and human resources to improve the products and services supplied to an organization. It is management’s responsibility to establish strategic objectives and build an infrastructure that is strategically aligned to those objectives. and the degree to which the current and future needs of the customer are met. all processes within an organization. existing improvement efforts.Skill Category 2 Quality Leadership he most important prerequisite for successful implementation of any major quality initiative is commitment from executive management. and technical tools under a disciplined approach focused on continuous improvement.1 2-1 . Version 6.1 Leadership Concepts Quality management is a philosophy and a set of guiding principles that represent the foundation of a continuously improving organization.2.

middle management is the weakest link in most quality management efforts. Having management commitment does not guarantee quality management success. will wait to see where executives prioritize quality management.1. Every employee. including other managers. a leader determines where the organization needs to 2-2 Version 6.1. commitment from the organization’s executives is vital. 2.1. Ideally. The need for a champion lasts a minimum of two to three years. and be included in all aspects of quality management planning and implementation. Ideally a champion will emerge during the planning for quality management implementation.2. The champion will be emotionally committed to quality management and will see it as a cause. the initial champion would be the top executive.2 Middle Management Commitment As the slowest group to accept the process. This “can do” attitude may be the most important consideration.Guide to the CASQ CBOK 2. This is the person who accepts personal responsibility for the success of quality management without being assigned the responsibility. The enthusiasm and energy of champions are important factors in the success of an organization. One way to assure their support is to assign them the task of determining their own role. Executive management sets the tone for the whole effort by visibly supporting quality management. Special effort is required to assure them they have a role as important players.1 .1. Champions happen naturally. 2. but several managers may assume this role at different times.1 Executive Management Commitment While overall management commitment is necessary to the success of quality management. have high quality standards and believe that the organization needs to improve. they are not appointed. How to include middle management is an important consideration for executive management.2 Quality Champion There is a need for one or more people to champion the cause of quality management. A quality management champion may assume the day-to-day management responsibility for successfully implementing quality management.1 Leadership Leadership and management are two different things. 2. They should have input to the statements that executive management prepares. it only improves the odds for successful implementation. it will not be with anyone else.1 Executive and Middle Management Commitment Management commitment is the single most important requirement for successful implementation of quality management. because obtaining quality management support from first-line managers and employees is relatively easy.2. There is no precedent of successful quality improvement without executive management and the management team leading the effort. If quality improvement is not "number one" with executive management. The champion should be respected in the organization.1. 2. While a manager works within the system following the accepted practices of the system.1. The entire organization must eventually become committed to quality management.1.

2. measurement. Some large companies opt for more than one level of Quality Council. the Quality Council: • • Initiates and personally commits to the quality management philosophies and practices. planning. some level of commitment and organizational structure exists that could be called quality management. 2.1 Quality Council A Quality Council is composed of the organization’s top executive and his or her direct reports. The Quality Council acts as the steering group to develop the organization’s mission. These serve as critical input to process mapping. Establishes committees at lower levels to focus on functional and cross-functional improvement efforts. to develop or revise processes. and to oversee and manage the quality management process on a daily basis. and ensuring resources are available for both ongoing and upcoming IT projects and internal process improvement projects. and to give employees a positive perception of their role within the business.2 Quality Management Infrastructure The reason a quality management environment is established is to assure constancy of purpose in promoting quality as a major IT goal. each organization’s mission and vision tie into that specified by the top council.2. technical prowess. etc. This section focuses on the infrastructure and initiatives. No organization has a perfect quality management environment. values. Recommends critical processes for analysis.1 2-3 . leadership is the ability to build the commitment of employees. • • • Version 6. In a business context. 2. By forming a quality function. There are two components to that environment: belief and commitment from management and staff. All are striving to achieve the optimum management philosophy. and the organizational structure and quality initiatives to support that environment. The first section of this Skill category focused on the belief and commitment of the management team. Specifically. and management ability may be important qualifications for toplevel IT management. Incorporates this decision into the strategic planning process. or approve the committees’ charters. and organizations can be anywhere along the quality management continuum.Quality Leadership be. While programming experience. and quality policy. leadership ability is the critical element. allocating resources in the budget for the deployment of quality management. It may also be referred to as an Executive Council. They may develop charters to serve as job descriptions for the committees. to endow an organization with a positive perception of itself. vision. and then does what is necessary to get there. Defines and deploys policies. goals. When multiple levels exist.

standards. They: • Work with the Quality Council to understand the organization’s mission. Establish teams or work groups (they may participate on the teams) to define and improve processes.). and improve processes. coaching. and identify those that need priority attention. goals. Develop. They may pilot new or revised processes. Process teams use standard approaches (such as flowcharts. Pareto analysis) to define. standards. Provides review and oversight of progress. or table (pending further investigation) new or changed processes. etc. Committees should represent all the skills and functions needed to work on the specific processes or activities. They also develop and maintain a deployment plan that identifies and prioritizes which key processes need to be defined and improved. They either review the charter provided by the Quality Council or develop one. Common names and functions are: • Process Development Teams develop processes. • • • 2.1 .2. The process teams are composed of a representative group of process owners. and priorities. Approaches and tools used by the team depend on its purpose. 2-4 Version 6.Guide to the CASQ CBOK • • • Makes the decision regarding whether to approve. They may also serve as process consultants to process owners and others using the process. or testing. inspection.3 Teams and Work Groups Teams are formed under any number of names depending on their purpose. and maintain a process inventory and process maps (see Skill Category 6). • Work Groups perform specific tasks such as JAD. etc. and quality control methods. but suppliers and customers are likely to participate as team members or as reviewers. It may also be desirable for a QA analyst to participate on the team. help deploy them to the rest of the organization. checklists.2. facilities. reject. They monitor team progress and review/approve the resulting processes. study. and provide process training. This includes proposing new processes and/or revising existing processes. standards. and are responsible for deploying quality management practices throughout the organization. etc. approaches. and provide support to the teams (training. standards. or commission the development of.2 Management Committees Management committees (also called Process Management Committees) are composed of middle managers and/or key staff personnel.2. Members of work groups will vary depending on the purpose. Analyze processes. tools. procedures. • Process Improvement Teams improve existing processes. 2. Acts on unresolved process problems and issues referred by the committees. at the direction of the Quality Council. One or more committees may be needed depending on the organization’s size and functional diversity.

author of The New Manager's Guide to Success defined these six basic attributes of executive management: Version 6. watch somebody go ahead and do it. Each team should have a work plan that outlines the tasks to be performed and assigns tasks to team members. many managers believe it is easy to buy technical capabilities. 2. The core team should be small. Image (visual intelligence about a person) is how others perceive a person. The team must reach consensus before submitting results to the process management committee for approval. Being male or female is irrelevant regarding how an image is projected. By looking at the dress of corporate officers or other very successful people. an image is developed of how successful people should look and act. Each team should have a chairperson. Baehler. containing 3-5 people. James R. A person has an image problem if peers appreciate his or her skills more than superiors do. Management relies on technical people for their technical capabilities. but the difference between a good technician and a good manager is frequently image. It is difficult to obtain good managers.2. The team should meet regularly to review work performed by individual members and to review progress. However. Teams Don’t Always Work When it's definitely determined by a team that something cannot be done. One would expect some differences in dress and actions. but basically the same attributes of good image apply to both male and female. If management has a high image of that person.3.1 2-5 .Quality Leadership Guidelines for teams include the following: • • • • • • • The process development committee selects teams. Their perception normally depends on how that person is viewed within the corporation. Different team members may draft different portions of the processes and procedures. Everyone has an image of what an executive looks like.1 Personal Persuasion People receive most information through visual intelligence.2. That image is normally shaped through role models. the probability of having his or her recommendations accepted and being promoted is significantly higher than when a negative image is projected.

Dr. Always talks straight. apologize for causing this type of problem to occur. not obstacles. You have not accepted the problem as your problem. Organizes thoughts before speaking. Talks about challenges. listens to the answers. Looks people directly in the eye. a human relations consultant. Stays calm. React positively to the customer's concern. Does more asking than telling. For example. in his book Contact: The First Four Minutes. specific. Does not accept generalities." show sincere interest.2. Get on your customer's wavelength – The first step in the resolution process is to show concern about the customer's problem by performing the following acts: • Get on the same physical wavelength. 2. "Do not interrupt us. Establish a position for mutual discussion. or ask the customer to sit and then sit after the customer complies. is brief. Quality promotes turning problems into opportunities. • • • 2-6 Version 6. Thus. States the problem. Assume a body position.2. execute the following four-step complaint-resolution process: 1. Does not appear rushed or harried. Knows the art of casual conversation.Guide to the CASQ CBOK Purposeful Competent Analytical Decisive Confident Appearance Stands and sits straight. such as. Is not tense with superiors.3. Does not waste time. To resolve the customer's problem. the customer will give up on you. Walks briskly. Research shows that complaints must be resolved within four minutes and the customer should be receiving a solution to his or her problem. Leonard Zunin. They will sense that you have not accepted the urgency of the problem they are expressing to you. Give undivided attention to the customer. Knows where he or she is going and how to get there. if the customer indicates you have caused great inconvenience to your customer's staff. then the solution. simple. Physically display interest. Comments to a secretary or receptionist. showing empathy. Dresses up one level. and direct. while resolving a customer complaint the opportunity can be used to improve customer relationships. and tone of voice that show concern.2 Resolving Customer Complaints Complaints are the customers' way of indicating they are having a problem.1 . states that unless a customer is satisfied within four minutes. gestures. Stand if your customer is standing. and you are not the one to solve their problem.

• Obtain feelings and attitudes.3. Get the facts – The problem cannot be dealt with until the problem (not symptoms) is known. still be empathetic. State the solution again. admit it and apologize for it. Find out how a person feels about what has happened. Establish and initiate an action program – Even if the complaint does not appear reasonable. order numbers. • Listen carefully. 2. through the words. 3. The problem could be a difference in believing what the solution was. As a second step: • Ask probing questions.Quality Leadership 2. the report must be comprehensive. talk about the steps that you will take to get the issue resolved. If the customer remains unsatisfied. • Take detailed notes. The value of the quality function can be rated on whether management accepts the report. The QA report is designed to convey information and to change behavior. to what is being said so that the real problem can be identified. Follow up with the customer – After the agreed upon action has been taken. identifying the scope. • Immediately take the action that was agreed to. amounts in question. See “Achieving Effective Listening” on page 14 for details. and sources of information. dates or times at which events happened. • Negotiate a satisfactory resolution with the customer by suggesting a solution and getting agreement. The problem may be more emotional than factual. Just as it is important to begin communicating a solution within four minutes. An angry person more likely tells symptoms than problems. follow up with the customer to ascertain that the result is satisfactory. In taking action: • If you are responsible for the error. Request an example of the problem. QA analysts write a report.1 2-7 . and to pacify the complainer. Thus. The solution may be to conduct an investigation and follow-up with the customer to determine next steps. make sure to communicate the name of that person and his or her contact information to the customer. samples of defective products. Good ideas are of little value unless they are accepted and implemented. Write down names. distribute it.2. action still needs to be taken to determine the validity of the facts. reports to management are the focus of this section because written reports are often used to judge the QA analyst’s ability to write. Do not minimize the seriousness of the error. and specific products and parts of products where problems occurred. Version 6. it is important to resolve the action quickly. find out what his or her colleagues or boss feels about the problem. but emotions need to be dealt with. to ensure customer agreement. • Note . and follow up on the recommendations.if you are not personally responsible for the problem. 4. Words do not always convey exactly what was meant.2. If resolution of the issue requires another person. return to step 3 and renegotiate a solution.3 Written Reports While QA analysts write many types of documents.

Provide enough information to make implementing the recommendations possible. Gather factual data (i. Have the report reviewed for readability At least one person other than the author should look at the report objectively. Failure to include this will adversely affect the credibility of the quality function and management will almost certainly disagree with the factual information in the report. Appearance. Develop a report outline A good report has no more than three objectives and three actions. If several items need reporting to management. 5. To write a good report the QA analyst should perform these ten tasks: 1. Review the draft for reasonableness The author should review the report to verify that the data gathered adequately supports the findings and recommendations. and explain any technical jargon of quality. 3. Too much data or too many requests overwhelm the reader. and must include all information necessary to attain that end. and report only the three most important. from the perspective of the target audience. rank the objectives according to priority. 6. and suggesting recommendations. 4.1 . and the impact it will have in changing managerial behavior. Draft the report General principles of writing any report apply to a QA report. The QA analyst should also remember the following potential problem areas: • • • Keep the quality-oriented language at a level that can be understood by management. to assess the impression the report will make on its readers. 2-8 Version 6. findings) and recommendations Ensure that relevant evidence supporting the data and recommendations is incorporated into the report. and that the information is presented clearly. 2. List small items in an appendix or a supplemental letter. The report must be written clearly and effectively enough to cause action to be taken. Establish report objectives and desired management actions Writing a successful report requires a clear understanding of both the report objectives (what the QA analyst hopes the report will accomplish) and the desired action (what the QA analyst wants management to do after reading the report). Ensure there is adequate time to write the report.e.. Consider using the presentation tools discussed in Skill Category 4.2.Guide to the CASQ CBOK explaining the factual findings.

Review the report with involved parties To recognize the importance of findings and recommendations. and effectiveness of the report are evaluated by considering the following questions: • • • Does the report appear to have been developed by a professional and knowledgeable group? Do I understand what the report is trying to tell me? Would a person associated with the report topic find the information in the report offensive or disparaging? If so.1 2-9 . 8. Review the report with management When the report is complete. Version 6. would they be more concerned with developing countermeasures than with implementing the recommendations? Does the report recommendations? adequately build a case for implementing the • • Does the report clearly differentiate between important and less critical items? 7.2. they should be discussed with affected parties before issuing the final report so that their support can be solicited. the QA analyst should meet with management to explain the report and to obtain their concurrence. Any issues should be addressed and corrected.Quality Leadership wording.

For example. 10. For the purpose of this skill category. In business and accounting literature. 2.Guide to the CASQ CBOK 9. and risks assessed. The quality environment has a pervasive influence on the way business activities are structured. the environment is referred to in many different ways. and follow up to ensure that appropriate action is taken. Distribute the report and follow up Distribute the final report to the appropriate parties.3.2. policies. It is the attitudes.” and sometimes just the environment in which work is performed. they listed several control objectives that if implemented would define each of the six attributes. procedures and behavior of management that sets the example for work in the organization.” other times the “management environment. ethics. formed a group known as COSO (Committee of Sponsoring Organizations). to provide guidance on evaluating internal control. For example. On the other hand. The six attributes are briefly described below: 2-10 Version 6. 2. American Accounting Association. employees will be lax in protecting their passwords and securing confidential information. if IT management conveys a lack of concern over security. Likewise. If IT management recognizes that they would not have any work task to perform if not for the users. The Institute of Internal Auditors. project personnel most likely will not follow that system development methodology. The COSO Framework identified the six quality attributes. objectives established.” What is important to understand is that the environment significantly impacts the employee’s attitude about complying with policies and procedures. and the Institute of Management Accountants). we will refer to it as the “quality environment. In accounting literature it is sometimes called the “control environment.1 The Six Attributes of an Effective Quality Environment Five major accounting associations (Financial Executives International. For each attribute. Finalize the report After incorporating any review comments make any final edits to the report. They issued this guidance as the COSO Internal Control Framework. values. then the users will be treated as very important people and their desires would be important to the IT organization. they will be treated as unimportant to the IT organization. if IT users are viewed as not knowing the requirements and over demanding.3 Quality Environment The quality environment is the totality of practices that management uses that effects how workers perform. American Institute of Public Accountants. they will be refunded the overpayment. if IT management conveys that they do not believe that following the system development methodology is important.1 . if management is ethical and customers over pay their accounts.

delegation of authority and establishment of related policies provide a basis for accountability and control. Executives should adequately understand their control responsibilities and possess the requisite experience and levels of knowledge commensurate with their positions. until recently. However. and sets forth-respective roles in the organization. Management must continually demonstrate. In most corporations. such as sales and community activities. Organizational Structure • The organizational structure shouldn’t be so simple that it cannot adequately monitor the enterprise’s activities nor so complex that it inhibits the necessary flow of information. Human Resource Policies and Practices • Human resource policies are central to recruiting and retaining competent people to enable the entity’s plans to be carried out so its goals can be achieved. intangibles. Management’s Philosophy and Operating Style • The philosophy and operating style of management has a pervasive effect on an entity. when suppliers invite them to lunch or offer them gifts.1 2-11 . employees are trained in how to perform their job responsibilities.3. Commitment to Competence • Management must specify the level of competence needed for particular jobs. The Code of Conduct of the corporation represents the manner in which the corporation expects the employees to act. It attempts to define the type of situations that employees may be faced with. A Code of Conduct is one of the most important documents involved in corporate governance. but one can look for positive or negative signs. and when they are faced with that situation. when they’re dealing with other employees.2 Code of Ethics and Conduct Employees need to know what is expected from them in the performance of their day-to-day activities. they were rarely trained in how to react in situations in which ethics and values are involved. of course. These are. and when they represent the corporation in outside activities. through words and actions. Version 6. when they acquire a second job. a commitment to high ethical standards. Assignment of Authority and Responsibility • The assignment of responsibility. and translate the desired levels of competence into requisite knowledge and skills. and employees must receive and understand that message. how they should respond to that situation. For example. Many employees face situations where ethical conduct guidance is needed.Quality Leadership The following six attributes are the key attributes of an effective quality environment: • Integrity and Ethical Values • Management must convey the message that integrity and ethical values cannot be compromised.2. and either has. • • • • • 2. or is trained in. needed job skills.

2. Before criticizing someone’s performance. in a proposal. • • 2-12 Version 6. For example. You should not leave an individual feeling that they have performed poorly or unsure as to how to correct that performance. if a code is to be effective. fix it" or "Your program does not make best use of the language or technology" leave people feeling confused and helpless.3.1. However. the senior officers of the corporation must set the example of how to perform. state specifically how it should and should not be used. indicate that a return-on-investment calculation was not made. That Code of Conduct must be taught and senior officers must live by the Code of Conduct. agents.2.” You must be prepared to help fix it. suppliers. Be prepared to train the subordinate in the area of deficiency. The Code of Conduct applies to all officers and employees of the organization. It is best done in a private location. 2. but must include communication upward from the lowest levels to senior management. Repeatable meaning that if individuals in a specific job are changed. Effective communication is planned.3. Effective communication is repeatable.3. 2.Guide to the CASQ CBOK It is not enough to have a Code of Conduct.1 . Have the Facts General statements of undesired performance are not very helpful. Communication must not only be downward from senior management. have specific items that are causing the deficiency or undesirable performance.3. and other stakeholders involved with the corporation.1 Providing Constructive Criticism In giving constructive criticism.1 Guidelines for Effective Communications The following are some guidelines that quality assurance personnel can use to improve their communication effectiveness. you should incorporate the following tactics: • Do it Privately Criticism should be given on a one-on-one basis. in a conference room or while taking someone to lunch. not spontaneous. rather than in the boss' office. the same types of communication will occur with new individuals. then that tone must be communicated to all involved. Many times it is more effective if it is done in a neutral location. For example. Only the individual being criticized should be aware that criticism is occurring. or if a program failed to use the language properly. for example.3 Open Communications If the “tone at the top” is to drive corporate governance. Communication would not only be to employees. statements such as "That proposal is not clear. but to partners.3. Be Prepared to Help the Worker Improve Their Performance It is not good enough to ask the worker to "fix it.

indicate that you expect a return-on-investment calculation included in all proposals.2. be as specific as possible in the things you like.Quality Leadership • Be Specific on Expectations Be sure your subordinate knows exactly what you expect from him or her now and in the future. get agreement from the individual on specifically why it might be disorganized. Avoid accepting agreement just because you are the boss. Always try to get the employee to propose what needs to be done. For example. One last recommendation for criticism: Avoid making threats about what will happen if the performance does not change. in a proposal. This will not cause any positive behavior change to occur and normally produces negative behavior. Be very specific in what you expect. Most people will try to do what they are expected to do—if they know what those expectations are. For example. Before criticizing indicate what you like about their performance. Ask the subordinate for advice on how to improve their performance. indicate that a report is disorganized. never indicate that an individual is disorganized. Again. Never criticize the individual. If the subordinate is unable to solve the problem.1 2-13 . If the employee’s suggestion is consistent with what you have decided is a realistic method of improvement. suggest the course of action that you had determined before performing the actual criticism. the "contract" should include your participation. Follow a Specific Process in Giving Criticism The specific process that is recommended is: • • State the positive first. they have great difficulty when you attack their personal work ethic. The individual being criticized must agree there is a problem before proper corrective action can be taken. Your expectations should be as clear as possible so there can be no confusion. Leave the individual with the assumption that he or she has the capability for improvement. probe the need for improvement with the subordinate until you actually feel there is agreement that improvement can be achieved. If the employee is uncertain how to do it. if you believe a report or program is disorganized. People can accept criticism of their products and services. as a vehicle to ensure what will happen. only the work performed by the individual. when and where you expect it. and that you know he or she will improve. Get agreement that there is a problem. you have finished the process. • • • • • • Version 6. Again. Indicate the deficiencies with products or services produced by the individual. Make a specific "contract" regarding what will happen after the session.

Some facts about listening include: Many Fortune 500 companies complain about their workers' listening skills. for example. we may pick up bad habits. • They are too busy rehearsing what they will say next. in learning to listen. 2) attending to the speaker. Step 1: Hearing the Speaker Hearing the speaker requires an understanding of the five channels of communication incorporated into speech.2 Achieving Effective Listening Throughout school.3. writing. • External stimuli. • Listening is the most frequently used form of communication. The listener must be aware of these detriments to good listening so they can recognize them and devote extra attention to listening. 2. Let's look at the five channels through which a speaker delivers information to his/her audience: • • • 2-14 Version 6. • They lack the motivation and responsibility required of a good listener. however.1.2. It is also important to understand why people do not listen. such as random thoughts going through their mind. an airplane flying overhead.1.3. Mastering each of these steps will help improve your listening abilities. but rarely is much emphasis placed on listening. and arithmetic. reading. diverts their attention.3 The 3-Step Listening Process The listening process involves three separate steps: 1) hearing the speaker. People do not listen for one or more of the following reasons: They are impatient and have other stimuli to respond to.Guide to the CASQ CBOK 2. This is particularly true in the practice of software testing – oral communication is rated as the number-one skill for the quality analyst. Listening is the first language skill that we develop as children. students are taught the importance of speaking.3. in ads a computer company emphasizes that they listen). • Listening is the major vehicle for learning in the classroom.1 . • The speaker’s topic is not of interest to them. • Salespeople often lose sales because they believe talking is more important than listening (thus. The shift in society from industrial production to information management emphasizes the need for good listening skills. it is rarely taught as a skill.3. in response to someone. The practice of listening requires these three listening steps to occur concurrently. Thus. • They are self-conscious about their communication ability. Much of listening occurs beyond merely hearing the words. and 3) understanding the speaker.

For example. or the use of a thumbs-down body channel signal will also indicate that John cannot do it. making it easy to lose the subject being covered on the information channel. Devote your full attention to the speaker to confirm that what you heard is what the speaker intended you to hear. The body movements and gestures associated with the information being conveyed. you must pay attention to all five channels. etc. you will not hear what the person is saying. If the subject is important to you. and listening to and watching the speaker to be sure we are receiving what the speaker is saying through all five channels. vocal. Hearing the speaker involves an awareness of all five channels. we will discuss the importance of feedback to confirm the subject being covered on the information channel.1 2-15 . you may hear that John can do it. while the speaker said that John could not do it. The words used by the speaker. that the speaker uses to emphasize or illustrate the material being discussed. but the vocal and body channels modify or emphasize the importance of those words. Active listening involves a lot of response and dynamics.” However. Speakers normally use the information. The pictures. You must first understand yourself and your situation. In Step 2. To master the hearing step. the tone of the vocal channel might indicate that John cannot do it. The vocal and body channels impact the importance of the verbal channel. but the speaker is boring. "John says he can do it. charts. Listening requires that there is a meeting of the mind on the information channel. For example. it will require significantly more effort on your part to be a good listener. Some people view the Version 6. The tone of voice associated with the various words. "John can do it" are stated. Step 2: Attending to the Speaker Attending to the speaker is sometimes referred to as being an active listener. if you are only paying partial attention to the speaker when the words. Speakers sometimes skip around to different subjects. The most important part of attending to the speaker is establishing an active listening ability. attending to the speaker. and body channels in speaking. they also use the graphic channel. If you miss one or more of the channels. In some instances. the words in the verbal channel may be.2. verbal. You must evaluate your motivation for wanting to listen to this speaker. The verbal channel includes the choice of words used to present information..Quality Leadership Information Channel Verbal Channel Vocal Channel Body Channel Graphic Channel The speaker’s subject.

It is very irritating to a speaker who is providing information to have the listener stray from the subject. Provide continuous feedback to the speaker.1 . Some suggestions to help in attending to the speaker are: • • • • • Free your mind of all other thoughts and concentrate exclusively on the speaker's communication. • Type 2: Comprehensive Listening 2-16 Version 6. but also can reduce or terminate the listening process. Feedback can be a nonverbal response. Periodically restate what you heard the speaker say. This is fine for hearing the speaker. and the listener changes the subject and asks where the speaker is going to have lunch that day. While people can listen several different ways concurrently. Maintain eye contact with the speaker for approximately 80 percent of the time. For example. normally listening is limited to one of the five types. Feedback is very important to the listening process.Guide to the CASQ CBOK listening process as a passive skill where you sit back and let the other person talk. For example. Step 3 . particularly in this step. It is very important to send the right type of feedback to the speaker. the speaker might be describing a quality problem. listen more to the nonverbal expressions rather than the verbal channel. The type chosen will have an impact on the ability to understand what the speaker is saying. such as nodding your head. but not for confirming what the speaker has said.. Move periodically to the understanding step to ensure that the information passed has been adequately understood. one may be listening to determine if an individual did a specific step in the performance of a task. To get this. listening must be adjusted to ensure that we get the message we need. The wrong type of feedback not only doesn’t confirm what the speaker said.Understanding the Speaker There are five types of listening. The five types of listening and their impact on understanding are: • Type 1: Discriminative Listening Directed at selecting specific pieces of information and not the entire communication.e. and ask the speaker to confirm the intent of the information spoken. When one has deciphered the information channel (i. what the subject is) and related the importance of that subject to the audience.2. or a verbal response such as a question or a statement of confirmation.

and quality policy must be what all levels of management truly believe and practice in their day-to-day activities. values. 2. Organizations need to map their course of direction. Developing the statements cannot be delegated.4 Mission. It is very helpful to use this type of listening when you want to gain the speaker's confidence and understand the reasons why a particular act was performed or event occurred. It clearly and concisely describes the work that is done. or does not know the complete facts of a situation. providing direction and a sense of purpose. goals. One must establish which type of understanding is wanted and then listen from that perspective. Values. as opposed to comprehensive listening where you want to find out what has happened. which is the corporate vision. A quality policy is a statement of principles. and what it is designed to accomplish. or activity exists. This type of listening requires a lot of feedback and summarization to fully understand what the speaker is communicating. • Type 5: Appreciative or Enjoyment Listening One automatically switches to this type of listening when it is perceived as a funny situation or an explanatory example will be given of a situation. The statements of mission. • Type 3: Therapeutic Listening The listener is sympathetic to the speaker's point of view. the mission is constrained by the vision and values. • Type 4: Critical Listening The listener is performing an analysis of what the speaker said.3.1 Mission A mission statement explains why a company. and Quality Policy The mission statement tells why a company or an organization exists.2. organization. During this type of listening. Goals. Values are like an organization’s code of ethics – they help establish the corporate culture and shape the foundation for making decisions. This listening type helps understand real-world situations. and a broad guide to action. Thus.Quality Leadership Designed to get a complete message with minimal distortion. nor is it a quick task. During implementation. This is most important when it is felt that the speaker is not in complete control of the situation. This type of listening is normally done in fact gathering. 2. Vision. The mission should focus on products and services and be customer-oriented. vision.3. Version 6. the audience uses this type of understanding to piece together what the speaker is saying with what has been learned from other speakers or other investigation. the listener will show a lot of empathy for the speaker's situation.4.1 2-17 . Goals convey how the vision will be achieved.

4. the owners of our business." From the Ford Motor Company (listed in their Ford Q-101 Quality Systems Standard. Well-developed programs are necessary to achieve the goals. and which will give us unique advantages over our competition.1 .3. allowing us to prosper as a business and to provide a reasonable return for our stockholders. and objectives of the company. goals. Examples of visions include: • • From the Quality Assurance Institute: “Our vision is to produce competent and successful quality assurance analysts. and will also go beyond boundaries we can see today. and integrity in all relationships. communications.Guide to the CASQ CBOK Examples of mission statements include: • From Arco Transportation Company Information Services: "The mission of information services is to provide the appropriate computing network. Organizations without a vision flounder. QA analysts should have a vision of improving quality." President Kennedy had a vision of putting a man on the moon before 1970. January 1986): “Ford Motor Company is a worldwide leader in automotive and automotive-related products and services as well as in newer industries such as aerospace. and financial services. ensuring how it contributes to the business is clear. known for reliability. productivity. The vision establishes where the organization desires to move from its current state. and customer satisfaction. • • • 2. Our business will be based on technologies that have evolved over our long history.” From the Eastman Kodak Company: "We see ourselves now and in the future as a company with a strong customer franchise. These technologies will span our core businesses. which is a clear definition of the result to be achieved. a goal might be to have no more than one defect per thousand lines of code. Goals change as an organization moves closer to accomplishing the vision. For example. and services and support of the strategies.” • 2.3 Goals Goals explain how the vision will be achieved. products.” From the Ford Motor Company: "A worldwide leader in automotive and automotiverelated products and services as well as in newer industries such as aerospace. A vision is simple and concise. trust.2 Vision Leaders provide a vision. 2-18 Version 6. Senior management should establish the vision.2. It gives everyone a direction to work towards.4. if an organization's vision is to produce defect-free software. and financial services. and it should be understood and supported by all.3. communications. Our mission is to improve continually our products and services to meet our customers’ needs.

They involve improving customer satisfaction and greater matching of the products and services to the true customer needs.) Must have management commitment Clearly identify the role each individual plays in accomplishing the goal Should be linked to programs established to accomplish the goals Strategic quality management goals must focus on both the producer and the customer. etc. thus creating an environment receptive to quality processes Understanding what must be done in order to deploy a new process Improving compliance to processes Sustaining quality effort.S. however.e.2. Goals: • • • • • Are consistent with the vision Are established by operational management (manager of systems and programming. goals tend to be more global and non-quantitative. Objectives come from goals. These goals could include.-based multinational companies in net earnings” Version 6. and tend to be more specific and quantitative. but should not be limited to: • • • • • • • • • • High customer satisfaction activities Management establishing a need for quality. shorter schedule and less resources) Provide return on investment from short-term programs Long-term goals should be customer oriented.Quality Leadership Goals and objectives are often used interchangeably. manager of computer operations. Short-term goals should: • • • Reduce defects Reduce cycle time (i. as a result of managing quality Involving all employees in quality processes Recognizing the need for the quality analyst Establishing a quality infrastructure with adequate resources to perform the assigned mission Having adequate resources to perform quality activities such as continuous process improvement A doable and measurable plan of action enabling the quality processes to demonstrate accomplishments based on approved objectives Financial goal statements from the Eastman Kodak Company are: • “To rank among the top 25 U..1 2-19 .

cooperative relationships. shareholders.1 . January 1986) include: 2-20 Version 6. customers. average for manufacturing companies” 2. ethical. quality management.” “Ethical behavior – so Kodak can earn and deserve a reputation that is beyond question. and suppliers. and they should be integrated into the organization's work program. and risk-taking. innovative. competitive.” “Teamwork – through open communication that gives everyone a sense of personal involvement in the company's performance. Values are established by senior management and respect the integrity of the individual.” • From the Ford Motor Company (listed in their Ford Q-101 Quality Systems Standard.3. Deming's 14 quality principles (see Skill Category 1). If really believed.” “Job satisfaction – in an environment that encourages people to grow to their full potential.” “Winning attitude – in knowing that through hard work. and market conditions.4. values help focus the organization on a shared behavioral model. social.” “Flexibility – recognizing the need to anticipate and respond to changing economic. Values should be consistent with Dr.” “Integrity – requiring honest relationships with colleagues. Kodak people make up a ‘world-class’ team.S.2. Examples of values include: • From the Eastman Kodak Company: “Quality – to strive for continuous improvement through personal contributions and teamwork. They help define an organization’s culture and personality by clarifying what behavior is expected in order for the organization to achieve its vision and mission. Examples of values are: customer-focused. pride and confidence.4 Values Values or guiding principles tell how to conduct business.” “Creativity – fostered by an atmosphere that challenges employees to seek new solutions and to take intelligent risks.Guide to the CASQ CBOK • • “To approach a return on equity of 20 percent” “To increase worldwide productivity at double the U.” “Trust – characterized by treating everyone with respect and dignity. employee empowerment.

Quality Leadership “People – Our people are the source of our strength.” “The objective of this policy is to provide products and services. so are we viewed. inside and outside the company. and policy letter should be reviewed to assure that it aligns with the new quality policy." 2.4. which must be aimed at the employees and written so they can understand it. As our products are viewed.” From Baxter: “We will reach agreement on requirements with our customers and suppliers. every existing regulation.” “Profits – Profits are the ultimate measure of how efficiently we provide customers with the best products for their needs.. no rework due to defects). procedure. They provide our corporate intelligence and determine our reputation and vitality. Quality improvement is the job of every Xerox employee.2. Profits are required to survive and grow. Eventually.” “Everyone must learn to do his or her job right the first time (i.” “Products – Our products are the end result of our efforts and they should be the best in serving customers worldwide. which are defect free. and meeting those requirements without error. Examples of quality policies are: • From Xerox: “Quality is the basic business principle of Xerox.” Key components from IBM Corporation’s quality policy are: “Quality is the cornerstone of the IBM Corporation business.3. Quality means providing our internal and external customers with innovative products and services that fully satisfy their requirements. on time. The policy should be concise and cover all aspects of quality. Total quality performance means understanding who the customer is.1 2-21 . We will conform to those requirements and perform defectfree work at all times. Involvement and teamwork are our core human values.5 Quality Policy Executive management's commitment to quality should be expressed in writing to all employees in the form of a quality policy.e.” • From Arco Transportation Company Information Services relating to people: "Information services maintain a productive and challenging work environment to foster personal growth and career development. Management should work as a team to develop the policy. what the requirements are.” • • • Version 6.” From Corning Glass Works: “It is the policy of Corning Glass Works to achieve total quality performance in meeting the requirements of external and internal customers. every time.

Guide to the CASQ CBOK “Each stage of a job must be defect free.” “Quality is everybody's responsibility.1 .” 2-22 Version 6.2.

1 3-1 . O Quality Baseline Concepts Methods Used for Establishing Baselines Model and Assessment Fundamentals Industry Quality Models page 3-1 page 3-6 page 3-12 page 3-16 3.Skill Category 3 Quality Baselines rganizations need to establish baselines of performance for quality. In order to establish a baseline. productivity rates such as function points per person-month. The metrics used for baselining should be metrics. and levels or amounts of documentation (number of words of documentation per line of code). to determine the baseline. if improved. For example. would help drive the management-defined results. These baselines are used to document current performance and document improvements by showing changes from a baseline. which. productivity and customer satisfaction. Version 6. if one were to baseline the software development process the baseline could include quantitative data on defect rates. a model and/or goal must be established for use in measuring against. resources expended by phase.1. Baselines normally are quantitative and describe multiple attributes of an activity/process.1 Baselines Defined Developing a baseline is performing an analysis/study to determine the current level of performance in a specific activity.2.1 Quality Baseline Concepts 3.

it is still a defect and warrants attention. This method is normally most effective when the information to be analyzed is automated. While sampling should be done statistically. 3-2 Version 6. program. to establish a baseline from which quality improvement objectives can be established and improvements in quality quantitatively measured. The practices used to improve the process are covered in Skill Category 6. Many IT managers do not know the severity of their quality problem. it is not essential in these studies for valid statistical samples to be drawn. or attitude.1. many of the individuals who chair the study group later become the quality assurance manager. • Sample survey Using this method. In many instances these study groups precede the establishment of a quality assurance function. They are attempting to establish a “base” from which a quality improvement need can be identified and measured.3 Conducting Baseline Studies The three logical groups to conduct baseline studies are: • Quality assurance groups If one has been established. In fact. they should conduct baseline studies in areas of quality concern. Once a need can be identified from analyzing a baseline. Baseline studies can be conducted in one of the following two manners: • Evaluate entire population This means all of the parties or products involved will be surveyed. when looking at factors such as schedule and budget. the need for a quality initiative to address the problem normally becomes obvious. It both shows “how you are doing” and provides a quantitative base from which change can be measured.1 . These quantitative quality baseline studies help establish the need for a quality initiative.Guide to the CASQ CBOK A baseline needs to be established for two reasons: first.2. 3. For example. and second.1.2 Types of Baselines A baseline study is one designed to quantitatively show the current status of an activity. The studies included in this manual should be viewed as baseline studies. The reason is that quality is attempting to eradicate all defects. and even if the defect is only perceived by a part of the population. 3. • Quality task forces A special task force or committee established to study quality/productivity in an information services function. This approach is normally most effective when there is a large population to be surveyed and the data is not automated. to identify perceived quality problems. only a part of the population of people/products are surveyed.

1. The quantitative data should be subject to analysis and interpretation. Version 6. Note that in many instances the study is actually conducted by a subordinate to that manager. 4. 2. Identify survey population The population of data/people to be surveyed needs to be identified. computer programs as a product. we rarely try to go below a sample size of twenty. and customer/user interaction as a service. but the general areas of nonconformance will need to be identified in order to gather nonconformance data. the data is used to substantiate what people intuitively know. It is always cheaper to look at fewer than more.1 3-3 . This is a critical step because the definition of nonconformance will vary significantly depending upon who is considered the population. but to the end user not meeting needs is a defect. then an additional study might be undertaken to validate the reliability. programming defects would look significantly different to the programmer than the end user of the program results. Identify products/services to be surveyed This is the requirements phase of the baseline study. The following are typical steps needed to perform a baseline study. For example. The objective is to identify quantitatively potential problems. If the data appears unreliable. The question that needs to be decided is how few can be examined to give valid results. For example. Statistically. Identify size of population to be surveyed This step is one that involves economics.2. 3. The programmer may only consider variance from specs a defect. Define conformance and nonconformance The individuals developing the survey instrument must have defined (at least on a preliminary basis) the expected conformance and nonconformance. Note that in many instances the survey will be used to help establish nonconformance. but in surveying people we may be able to drop below this limit and still get valid results. Baseline studies need not be time-consuming or costly. because in most instances.Quality Baselines • IT management Specific managers that have a concern over quality may wish to perform a baseline study. Studies should be directed at specific products or services. It is generally good practice to get the data as quickly and cheaply as possible.

consolidated. if we wanted to know how many lines of executable code were in a program. Take action and notify participants of that action All surveys should result in some specific action. 3. There are very few objective studies that can be conducted within information services.1. may really be more subjective. if we ask people to keep time records. Not to inform the participants of the action will reduce cooperation in future surveys.Guide to the CASQ CBOK 5. 3-4 Version 6. the hours count is subjective because many will record their hours worked at the end of each month. We need to note that what may appear as objective. and let us assume we can define what is meant by a line of executable code. Normally this should not exceed one week. a decision should be made based on the survey results. This can be done through written instructions. Develop survey instrument A specific survey instrument must be developed to meet the survey objectives. If the surveys have not been returned by that time. the validity of the results will increase when extra time is spent to explain the intent and purpose of the survey. the participants expect some action. but it is normally better to do it verbally. 6. Even if that action is to do nothing.2. Surveys need to be customized to the specific needs and vocabulary of the population being surveyed. For example. we must make the assumption that the count is accurate. 8. In most instances. Follow up on incomplete surveys The survey should have a time limit for response. Objective means performed by counting. If the population group is small.3. Conduct survey The survey instruments should be completed by the surveyed population. and thus the hours count is a subjective measure and not an objective measure.1 . From a quality perspective. they can be called together for a meeting. or have the survey instruments hand delivered and explained. then we could count those lines and have an objective baseline. 7. For example. then the surveying group should follow up and attempt to get as many surveys completed as possible. 9.1 Conducting Objective Baseline Studies Objective baseline studies are ones which are viewed as factual and non-argumentative. and put into presentation format. Note that whenever a survey is conducted. and then record hours worked based on those times. Accumulate and present survey results The survey information should be accumulated. Generally. and what can be counted must be considered non-argumentative. it is helpful to train the population on how to complete the survey questionnaire. Suggestions on how to do this follow in a later part of this quality initiative. Note that it is generally not realistic to expect every survey to be returned. That decision should be given to the survey participants.

2.2 Conducting Subjective Baseline Studies Subjective baseline studies will be the most commonly conducted studies in measuring quality and productivity. there is a great deal of difference between a “1” rating and a “3” rating for dissatisfaction. It is important to recognize that there are very few objective measures. Examples of objective baseline measures that can be used for baselines include: • Project completed on schedule • Lines of code • Number of programs • Number of people assigned to a project • Number of abnormal terminations Again. but it is indicative of a problem. Subjective means that judgment is applied in making the measure. We noted in the discussion on objective measures that when the individual involved in recording time has the option of applying judgment. we are looking for ways to put this information into a quantifiable format. Examples of products/services that can be measured subjectively for developing a baseline include: • Customer satisfaction • Effectiveness of standards/manuals • Helpfulness of methodologies to solve problems • Areas/activities causing the greatest impediments to quality/productivity • Causes for missed schedules/over-budget conditions • Understandability of training materials • Value of tools • Importance of activities/standards/methods/tools to individual activity Baselines can be conducted for any one of the following three purposes: 3.1 3-5 . we now have conveyed a lot more information. 3. For example. This does not convey a lot of information. If our scale rates “1” as very poor service. if we develop a five-point scale for unresponsiveness. Version 6. because quality conformance and nonconformance must be defined by people.2. Baselines should be quantitative even if it is a subjective measure.1. However.1 Planning To determine where detailed investigation/survey should be undertaken. and thus we are forced to use subjective measures in measuring quality and productivity. and ask your dissatisfied customer to complete that scale.3. the exactness of the counting will actually determine whether the above measures are objective or subjective.3. and “5” as very good service.Quality Baselines Objective measures are those.1. which can be accomplished by counting. but quantitatively subjective. then the measure becomes subjective.

To help understand the type of baselines that are used in IT. and commitment (e.Guide to the CASQ CBOK 3. 10% correctness. product is available when you need it) • Correctness – Accurate and meets business and operational requirements (e.. Benchmarking Comparison against external organizations. • • • Customer surveys Benchmarking Management established criteria 3-6 Version 6.g.. 3. to develop a customer satisfaction score you would need to assign importance to each factor.g. Each one of these will be discussed individually. as agreed (e.. you might assign the percentage of importance to your end user/customer of these eight quality factors as follows: 10% availability. automated information is accurate) • Flexibility – Easy to customize for specific needs (e. 5% flexibility.1. Once the problem/area has been identified. 10% caring.2.2 Internal analysis To identify problems/areas for quality improvement. and relevant (e.3 If you want to develop a baseline of customer satisfaction. 20% dependability..g. 20% competence.g. you might ask your customer to rate the following factors using “very satisfied” to “very dissatisfied” as a scale: Availability – Accessible to the customer...3.1 . project team successfully completes work on schedule) • Responsiveness – Providing prompt turnaround of customer inquiries and requests (e.2 Methods Used for Establishing Baselines IT organizations have established many different baselines to evaluate current performance and to measure improvement. and 15% responsiveness. project team responds quickly to your written requests) Next.g. easy to enhance product with new features) • Usability – Easy to learn. • 3. 10% usability.2.g.3.2. then no additional effort need be undertaken to formalize the results. easy to use. courtesy.g.g. project team provides correct answers to questions) • Dependability – Meeting cost and schedule commitments (e. the way you interact with components of the product is consistent) • Caring – Demonstrating empathy. For example.. QAI has categorized four most commonly used baselines as listed below.1. project team is sensitive to your needs) • Competence – Having and applying the appropriate knowledge and skills (e..

you will know that they have the same definition for defects and the same definition for lines of code. These can be internal customers. Skill Category 1 provides two definitions of quality. We can divide customer surveys into report cards and surveys.” Customer surveys use the “fit for use” definition of quality. most baselining processes have these four steps: Version 6. The subjective response is defined in enough detail so that there is consistency among individuals regarding the response. were they satisfied with the user manuals provided by IT.1 Customer Surveys The customer needs to be defined as some group or individual receiving IT products and services. If the definitions are different.2.2. The factors and attributes are then given rates so that a total customer-satisfaction score can be developed. However. or external customers.Quality Baselines • Industry models 3. The report card will ask them on a scale of 1-to-5. if you wanted to establish a baseline goal for up-time in your computer center. There are many different steps that organizations follow in benchmarking.2. you might want to go to what you believe is a leading organization and use their up-time percentage as your baseline goal. Questions are then constructed to support the assessment of that factor and attribute. Customer surveys measure attitude and satisfaction of customers with the products and services they receive. Customer surveys are subjective baselines. Because the report card is not controlled. Let us assume that you are baselining the number of defects per thousand lines of code created during development. Because they are subjective it is important that customer surveys are properly constructed. on what they think of the user manual. such as programmers receiving program specifications from systems analysts. Through your baseline of your organization you will have to carefully define what a defect is and what a line of code is. For example. then the benchmark that you get from that organization will be meaningless as the goal for your organization. it is sometimes difficult to know how the user calculated a particular rating. for example. Report cards ask the customer what they think about something. 3.1 3-7 . These were “meets requirements” and “fit for use. such as the payroll department using IT products and services to produce payroll. What is important in benchmarking is that you have a well established baseline measurement in your organization so that you can ensure the other organization’s baseline is comparable.2 Benchmarking to Establish a Baseline Goal A benchmarking baseline is one determined by another organization. one meaning very pleased and five meaning very displeased. Surveys should be constructed around specific factors and attributes of a product or service. Once you have done that and you look at another organization’s defects per thousand lines of code for benchmarking purposes.

When you find a variance between the baseline calculation in your company and the baseline calculation in the organization you are benchmarking against. Identify discussion areas • 3-8 Version 6. • Comparing quality programs to those in other companies can identify gaps in current processes and lead to obtaining more effective quality practices. In our example of defects per lines of code. and you benchmarked against an organization that only had 10 defects per thousand lines of code you would want to identify the cause of the difference. 4. Visiting another company is a five-step process. For example. as follows: 1. 2. Learning this. Learn from others and don’t “reinvent the wheel”. If you cannot identify the cause of difference. 3. you need to identify the cause of variance. This will provide the quality professionals with these benefits: The cost and effort to develop new and innovative quality approaches within IT is cost-prohibitive for most companies. Benchmarking is only effective when you benchmark against an organization who has calculated their baseline using approximately the same approach that your organization used to calculate the baseline.Guide to the CASQ CBOK 1. Compare baseline calculations. Develop a clearly defined baseline in your organization.1 . Identify the cause of baseline variance in the organization you benchmarked against. Identify the organizations you desire to baseline against. do you want to benchmark what you believe are leading organizations. Those colleagues will not exist internally unless the company is large. A less formal method for benchmarking is to visit other organizations. and do you want to benchmark against organizations with a similar culture. Compare how your baseline is calculated versus the baseline calculation in the company you want to benchmark against. you may choose to adopt JAD in your organization as a means for reducing your developmental defect rates. such as do you want to benchmark within your industry. there is little value in benchmarking. clearly defining what is meant by defect and a line of code would meet the objective of this step. • Interfacing periodically with other quality individuals is good for professional development.2. Many factors come into this decision. This means that all of the attributes involved in your baseline are defined. For example. do you want to benchmark an organization that uses the same tools that are used in your organization. if your organization was producing 20 defects per thousand lines of code. assume they use JAD (joint application development) and you did not. Let us assume that the company you benchmarked against had a different process for requirement definition than your organization.

reworking. distributing. convert it into a visit objective. what you can share. or to other organizations or corporations. The visit concludes by thanking management of the other company.3 Assessments against Management Established Criteria Management can develop a baseline for anything they feel needs to be measured and/or improved. and.) is given. Put new practice into use Select the one best idea from the visit for implementation. such as identifying the three most effective quality practices used by the other company. Determine the objective of the visit. 2. Identify constraints that need to be considered as part of the selection process. 4. There are no industry models or standards Version 6.1 3-9 . Do not try to implement too many things at once. 5. One-half to two days is recommended. Offer to provide a letter requesting the visit if your colleague needs it to get the visit approved.Quality Baselines As it is important for the visit to be mutually advantageous. the information you would like to receive in exchange and how the visit would be mutually advantageous. Consider items such as whether information can be exchanged with competitors. a restatement of the objectives of the visit and a tour of the other company’s facilities to put the size and purpose of the IT function in perspective. • • Identify a specific area such as conducting software reviews or independent testing. State the purpose for the visit.2. leaving some memorabilia or some of your company’s products as a token of remembrance. Conduct the visit Visits typically begin with introductions. and obtain management's approval to schedule a visit with them. get management agreement on what oral and written information can be shared with the company being visited.2. Written documentation is exchanged and agreement for usage (such as copying. 3. etc. the availability of travel funds. Agenda items are then discussed in detail. Demonstrating a positive outcome from these visits will increase the chance of management approving other visits. prioritizing them by the desirability. and whether the maturity of the company's quality function lends itself to the topics selected (starting a quality function requires a company that has gone through the process). whether the size of the target company should be similar. If the objective is to gather general information. Set a date and time for the visit. if possible. These are baselines that are organizational dependent. 3. Schedule the visit Contact your peers at the targeted companies. Identify yourself and your company. Select three target companies. Identify target companies Visits can be within divisions or subsidiaries of the corporation.

The organization may have developed a training program for tool x and want to develop a baseline on how the students taking that course perceive the value of the course. they should develop a baseline to evaluate performance. and the opportunities for growth in the organization. 3-10 Version 6.) Assess the mission and goals of the organization. should be periodically evaluated to discover whether the satisfaction level of the employees is positive or negative. etc. responses of employees to one another. Generally. expectations. The baseline can also measure improvement due to changing that particular item. committee. Look at the organization (group. and. interpersonal conflicts. task force.1 . perception of events. The following is an example of a baseline for IT climate. what it is supposed to produce. An example of calculating such a baseline follows below. An evaluator can use the six steps below to assess the climate of a specific organization. It is a composite of human behaviors. therefore. if management wants to know the efficiency or effectiveness of something in their organization. etc. The climate is crucial in creating and maintaining an effective organization. group. For example. Climate is a component of the environment relating to the individual worker’s perception of the “climate” in which they perform their work activities. and the overriding principles by which it operates.Guide to the CASQ CBOK against which a baseline can be determined.2. task force. committee. The organizational climate is the workers’ attitude toward their organization. 1. your management may want to develop a baseline on how effective an organization developed training program is.

2. Create a dialog with the members of the group Interact with each employee asking a series of hypothetical questions to identify the employee's true feelings toward the organization. whether they follow or ignore the requests of their manager. and promotion than accomplishing the organization’s mission.e. can help draw out the true feelings of each employee. 6. ask if the employee is doing what should be done. Rate organizational climate Based on the responses to steps 1-5. Determine whether each employee within the group likes his or her manager. • Average (3 points) The organizational climate is one of accomplishing the organization's mission and goals.1 3-11 . Assess employees’ performance Evaluate each employee’s performance in relation to the organization’s mission and goals. and whether they attempt to protect their manager (i. and whether it is important in accomplishing the mission and goals of the organization. 3. whether it makes full use of the employee’s capabilities. Examine the jobs Examine each job in the organization. 5. For each job being performed. is using his or her skills effectively. Ask whether the job is necessary. • Good (4 points) Some concerns about the health of the climate. but no more. 4.Quality Baselines 2. evaluate the climate on the following five-point scale: • Ideal (5 points) A fully cooperative environment in which managers and staff work as a team to accomplish the mission. make their manager look good). development. and has enthusiasm and interest in performing the job. likes his or her job. Evaluate how employees feel about their manager or leader Good organizational climate requires good leadership. Questions such as.. but overall it is cooperative and productive. • Below average (2 points) The individuals are more concerned about their individual performance. “do you feel the organization supports your suggestions”. • Poor (1 point) Version 6.

3. for example. Models are usually developed under the auspice of national or international standards organizations and may be customized or ‘tailored’ to meet new or changing needs. as the climate rarely improves without procedural changes. If the workers’ abilities are not being utilized effectively. try an incentive program or one tied to short-term rewards. reassign tasks to take advantage of their capabilities. a day off. such as a paid lunch. and through discussion and compromise agree upon the mission and goals for the organization. Develop a plan for accomplishing the organization's mission that is understood and acceptable to its members.Guide to the CASQ CBOK There is open hostility in the group and a non-cooperative attitude.3 Model and Assessment Fundamentals 3.4 Assessments against Industry Models An industry model can be used to measure your IT organization against that industry model. Get feedback from the employees.1 .2. etc. Compliance assessments can be through first-party.3. The following section describes the more common industry models used by the IT industry. Organizations choose to adopt a model for any or all of the following reasons: 3-12 Version 6. Your current level of performance against that industry model provides a baseline to use to measure improvement. This is normally accomplished if the members help develop the plan. the mission and goals are typically not met. • • • • 3. A negative climate of three points or less often results from individuals having too much responsibility without the authority to fulfill those responsibilities. second-party or third-party audits or other types of evaluation.1 Purpose of a Model A model is an idealized concept to be accomplished.2. Negative organizational climates can be improved with the following: • Develop within the organization a shared vision of what needs to be changed. Develop new methods. or management’s failure to recognize the abilities of the employees. As a result. Most industry models define the minimum that has to be accomplished for compliance. Change the organization's procedures. and allow compliance to be measured in "pass/fail" terms.

are discussed below.2 Types of Models (Staged and Continuous) The two types of models that exist. adopting a model can refocus the IT organization to meet those goals and objectives. ISO15504 (SPICE) is another example of a staged model. and. The Software Engineering Institute’s Capability Maturity Model Integration (CMMI®1) is an example of a staged model. Version 6. The processes themselves are staged and serve as foundations for the next process.3. although it now has a continuous representation.2. For example.2. ® CMMI are registered in the U.3. processes are individually improved along a capability scale independent of each other. as such. 3. key customers may direct the IT organization to adopt a model. For competitive reasons External customers may only do business with an IT organization that is in compliance with a model. the project planning process could be at a much higher capability level than the quality assurance process. provides a road map for improvement.3. As a guide (road map) for continuous improvement A model usually represents the combined learning of leading organizations.1 Staged models Staged models are composed of a number of distinct levels of maturity. Likewise. staged and continuous. • • • 3. 3.2 Continuous models In the continuous model.3 Model Selection Process There are a number of software process improvement models publicly available. 3. This is particularly true of continuous models.1 3-13 . Each level of maturity is further decomposed into a number of processes that are fixed to that level of maturity. See “Software Engineering Institute Capability Maturity Model Integration (CMMI®)” on page 16 for more information.3. Malcolm Baldrige National Quality 1.Quality Baselines • Satisfy business goals and objectives If the current infrastructure and processes are not meeting business goals and directives. Requirements are imposed by customer(s) In an effort to improve efficiency and effectiveness. The most notable are the Capability Maturity Model Integration (CMMI®).S.2. each level of maturity is the foundation for the next maturity level. Patent and Trademark Office.

The MBNQA was designed to identify world-class quality organizations in the United States. a model may propose the use of customer and employee surveys. the Deming Prize. Quality models are designed for specific purposes. An assessment determines the baseline against the model (goal to be accomplished). the model may either be rejected or modified. need goals and objectives to measure that improvement. If management does not commit budget. then measurable goals and objectives can be defined that describe movement toward meeting the model criteria. A particular model may include criteria that are inconsistent with the organization’s goals or objectives. • Management commitment The implementation of any model may require a significantly different management style than currently exists.Guide to the CASQ CBOK Award (MBNQA). • Need for measurable goals and objectives Organizations that desire to improve. and selecting a model does not have to be an all or nothing decision. and ISO/IEC 12207. the effort is doomed from the beginning.4 Using Models for Assessment and Baselines A baseline is a current level of performance. A specific model may or may not fully apply to an IT organization.3. Measuring the object against a model improves the objectivity of a baseline assessment. can be a major step in improving the productivity and quality of an IT organization as it moves toward world-class status. • Need for baseline assessments An organization may need to know how effective and efficient it is. It is important that each criterion in a model be directly related to accomplishing the organization’s goals and objectives. the third criterion required for continuous improvement is a method for moving from the current baseline to the desired goal. Without measurement against specific criteria. If a model is accepted as the means for continuous improvement. used with ISO/IEC 15504 as the assessment model. Having a model that is applicable. and that management is committed to achieve.2. In that case. schedule and resources to implement and sustain the model. The new management style may involve different programs or methods of doing work. 3-14 Version 6.1 . and so forth. In addition to the baseline and the model. 3. An organization should consider these four criteria when selecting a model: • Applicability of the model to the organization’s goals and objectives. any efficiency and effectiveness assessment is likely to be subjective. The effort and resources required to do this may not be consistent with the organization’s current work plan. For example. the CMMI was developed to evaluate a software contractor’s capability to deliver quality software. ISO 9001. independent audits to determine compliance to the model. For example.

The results may be used to drive process improvement activities or process capability determination by analyzing the results in the context of the organization's business needs and identifying strengths. in many countries. such as building a single software system. There are two general categories of auditors. including determining baselines and regular assessments. In addition.1 Assessments versus Audits An assessment is a process used to measure the current level of performance or service. For the purpose of this study guide. These are auditors external to the organization (external auditors). The objective is to establish a baseline of performance.2. and then assessments are used again to show whether or not performance is being accomplished. The external auditors are normally engaged by the organization’s Board of Directors to perform an independent audit of the organization. have no means of determining whether performance has changed.4. A report or opinion by the auditors relates to whether or not the activity under audit has or has not met the objectives/requirements for that activity. The effective use of a model. and risks inherent in the processes. Sequencing of the improvements is determined by the organization. Audits assess an activity against the objectives/requirements established for that activity. Internal auditors are employees of the organization that they audit. While assessments may be done by the individuals involved in conducting the work. can help IT management continually improve effectiveness and efficiency.3. the external auditors (frequently referred to as Certified Public Accountants or Chartered Accountants) are covered by stock exchange and governmental regulations. an audit is performed by someone independent of the activity under audit. as a result. 3. They frequently perform what are called “test implementation audits. as opposed to an overview of all the information systems built within a specified period by the organization.” Most of these audits look at a specific activity.1 3-15 . You can measure against an internationally accepted model or an organization’s own defined criteria. and. While internal and independent auditors are subject to their professional standards.Quality Baselines The assessment against the model characterizes the current practice within an organization in terms of the capability of the selected processes. and auditors internal to the organization (internal auditors). Many organizations put new programs into place without a model. Both internal and external auditors are governed by the standards of their profession. For example. Many audits within the IT organization are conducted by quality assurance professionals. weaknesses. An audit is an independent appraisal activity. Version 6. we will define an audit as an independent evaluation of an IT related activity. auditors within an organization are not held to professional standards. information systems may be audited after they are placed into operation to determine whether or not the objectives of those systems were accomplished.

Most baselines are determined by.1 . In 1992. IT management will establish achieving the criteria of a model as a goal.0 was released and became the framework for software development and control. they would determine their current performance baseline against the CMMI model. when industry models are adopted the organization has a choice to bring in outside consultants to establish the baseline. the CMMI® framework is a method for organizing these steps into five levels of maturity. Each maturity level defines an evolutionary plateau of process improvement and stabilizes an important part of the organization’s processes. Continuous improvement is based upon long-term objectives accomplished through the implementation of short-term evolutionary steps. 3-16 Version 6. Although this can be costly. and measured by. CMM version 1.1 Maturity Levels The CMMI® influences an organization’s culture by instilling discipline and continuous process improvement into the workplace. if an IT organization wanted to be at CMMI Level 3.1 Software Engineering Institute Capability Maturity Model Integration (CMMI®) In 1986.Guide to the CASQ CBOK 3. the Software Engineering Institute (SEI). maturity levels organize the process areas and provide a recommended order for approaching process improvement in stages.2. IT personnel. One of the most commonly used models for developing a performance baseline is CMMI model.1. In 1991. On March 1.4 Industry Quality Models There are many industry models available against which your organization can establish a baseline. 2002. the integrated model for systems and software engineering. 3. For example. and supplier sourcing (CMMI®-SE/ SW. As shown in Figure 3-1. However. began working with the Mitre Corporation to develop a process maturity framework that would help organizations improve their ability to build software solutions. 3.” In the staged representation. version 1.4. This has two different representations – staged and continuous. Integrated Product and Process Development.1 was released with improvements from the software community. The following section describes the most commonly used models in the IT industry.1) was released. and then establish the baseline. Normally.4. the Capability Maturity Model (CMMSM) version 1. a federally funded research and development center. it does provide an independent view. It is a framework that enables an organization to “build the right products right.

3.4. with no more than two awards being presented per category per year. stabilizes specific aspects of the software development process. • • • • • Manufacturing Service Small Business Educational Organizations Health Care Organizations Version 6. named after a former Secretary of Commerce.2.2 Malcolm Baldrige National Quality Award (MBNQA) The United States national quality award originated from the Malcolm Baldrige National Quality Improvement Act (Public Law 100-107). The framework is also an aid to quality planning as it affords organizations the opportunity to prioritize improvement efforts. That act. resulting in an overall increase in the process capability of the organization.1 3-17 . when satisfied. A maturity level is a well-defined evolutionary plateau for achieving a mature software process. Each level contains a set of goals that. called for the creation of a national quality award and the development of guidelines and criteria that organizations could use to evaluate their quality improvement efforts. signed by President Ronald Reagan on August 20. Awards are given in the five categories below. Achieving each level of the maturity model institutionalizes a different component. 1987.Quality Baselines Figure 3-1 The SEI CMMI® Framework The five maturity levels define an ordinal scale that enables an organization to determine its level of process capability.

This survey revealed the need to substantially re-engineer the model itself.3. Under ISO protocols. with an emphasis on four needs: A common structure based upon a Process Model Promote use of continuous improvement and prevention of non-conformity Simplicity of use. revised. Involvement of People 3-18 Version 6. In 1997. the ISO 9000 family of standards integrated the five 1994 Standards into four primary documents as follows: ISO 9000 – Gives an introduction and a guide to using the other standards in the 9000 family. These use similar categories and variations of the scoring methods used by the MBNQA. install and service products. • ISO 9004 – Helps companies implement quality systems. produce. Leadership 3.4.4. 3. ease of understanding.4. develop.2. there are also the European Quality Awards and the Australian Quality Award in their respective regions.Guide to the CASQ CBOK 3. • ISO 9001 – Describes a quality system model. Customer Focus 2. • ISO/IEC 90003:2004 Software Engineering – Guidelines for the application of ISO 9001:2000 to computer software • • • • 3.2. all standards must be reviewed at least every five years to determine whether they should be confirmed. ISO conducted a large global survey to better understand their needs.1 Other National and Regional Awards Other countries and regions have also created quality awards.1 . and use of clear language and terminology • Consistency with the Plan-Do-Check-Act improvement cycle In 2000.3 ISO 9001:2000 The ISO 9000 family of standards contains 3 standards and many supporting documents. or withdrawn. For those interested in applying ISO 9001:2000 to Software the following will be useful. 1. The Deming Prize is awarded in Japan. These are supported by the following Standard: • ISO 19011 – Includes guidelines for auditing quality systems.1 Model Overview The ISO 9001:2000 standards are based on the following eight quality management principles. including quality system requirements applicable to organizations that design.

how these processes interface with each other. is a nine part Technical Report referred to as SPICE. Process Approach 5. it does not detail how to perform the activities and tasks.4. format. and the high. ISO/IEC TR 15504:2004 Information Technology – Process Assessment. System Approach to Management 6. this Technical Report has become an International Standard comprising of five parts. It contains a framework for managing.2.5. It has subsequently been amended twice. 3.1 Model Overview ISO/IEC 12207. is the international standard that covers the software life cycle from concept through retirement. Factual Approach to Decision-Making 8. For each process. Mutually Beneficial Supplier Relationships 3.4 ISO/IEC 12207: Information Technology – Software Life Cycle Processes 3.5 ISO/IEC 15504: Process Assessment (Formerly Known as Software Improvement and Capability Determination (SPICE)) In June 1991. and improving the software life cycle activities. controlling.level relations that govern their interactions. As a result. or content of documentation. The standard describes the major processes of the software life cycle.4. Since it is a high-level standard. defining specific responsibilities and identifying outputs of activities and tasks. organizations applying ISO/IEC 12207 should use other standards and procedures to specify these types of details. 3. nor does it specify names.1 Model Overview ISO/IEC 15504: provides a framework for the assessment of software processes that is appropriate across all application domains and organization sizes. This framework can be used by Version 6. ISO approved a study period to investigate the needs and requirements for a standard for software process assessments.Quality Baselines 4. The standard does not assume a specific life cycle model. Continuous Improvement 7.4. Based on a series of trials that have been held around the world since 1994.4.1 3-19 . The resulting conclusion was that there was an international consensus on the needs and requirements for a process assessment standard.4. which was published in 1995. Amendment 1 defines a Software Engineering Process Reference Model for use with ISO/IEC 15504 Process Assessment. the standard also describes the activities and tasks involved.

The results of the audit will be used to both improve the software system and to improve the process that builds software systems. supply.4. development. control. evolution.2. and support of software. operation. maintainability) were achieved Post-implementation audits are a quality assurance activity. or all. monitor.g.1 . of the following: • The system objectives were met • The system specifications were implemented as specified • The developmental standards were followed • The IT quality objectives (e. and improve the acquisition.6 Post-Implementation Audits A post-implementation audit is conducted to determine any. manage.Guide to the CASQ CBOK organizations that plan. 3-20 Version 6. The post-implementation audit is used to assess the ability of the IT organization to perform effective and efficient work. an organization with the objectives of: • • • Understanding the state of the organization’s processes for process improvement (establish process baselines) Determining the suitability of the organization’s processes for a particular requirement or class of requirements Determining the suitability of another organization’s processes for a particular contract or class of contracts 3. or on behalf of. The SPICE standard provides a structured approach for assessing software processes by..

and pays for itself by actually reducing costs. Version 6. Quality functions have demonstrated that quality can be defined and measured.1 Establishing a Function to Promote and Manage Quality Inadequate attention to quality in IT normally results in high systems maintenance costs and customer dissatisfaction.1 4-1 .Skill Category 4 Quality Assurance Q uality Assurance is a professional competency whose focus is directed at critical processes used to build products and services. The profession is charged with the responsibility for tactical process improvement initiatives that are strategically aligned to the goals of the organization. This category describes the management processes used to establish the foundation of a quality-managed environment: Establishing a Function to Promote and Manage Quality Quality Tools Process Deployment Internal Auditing and Quality Assurance page 4-1 page 4-6 page 4-28 page 4-31 4. operation. and can improve customer satisfaction. Experience has shown that effective quality does increase productivity. the quality function can reduce the cost of systems development. The quality function has been implemented in hundreds of companies. Through the promotion of effective quality practices. and maintenance.2.

the real impediment. so that scope changes cause the resources or schedule dials to move. For example. The four project variables are scope. but if management holds schedule and resources firm while moving the scope. Until management believes there is a need for quality improvement. which get set according to the project criteria as illustrated in Figure 4-1. a staff function for quality is unnecessary. The ultimate responsibility for quality rests with senior management. the project team agrees to install a new requirement. what must give is quality. 4-2 Version 6. The management challenge in completing the project can be illustrated as a dashboard of four system attribute dials. While individual workers have responsibility for the quality of their products and services. so movement of one dial affects one or more of the other dials. cannot be dealt with. and quality. A quality function exists when a specific individual/group is assigned the responsibility to assist in improving quality. The four dials are interconnected. Figure 4-1 The Four Interconnected Variables of IT Development All goes well until one of the variables needs to be changed. or fewer controls. resources.2. it is management's responsibility to ensure that the environment is one in which quality can flourish. schedule. less documentation. Reduced quality occurs as less testing. The following example shows how this happens in a typical systems development project. That argument is theoretically correct.1 . something must give. Many people argue that because everyone has some quality responsibility. One role of the quality function is to hold the quality dial firm. the pressures of other priorities such as meeting schedules and budgets frequently takes precedence over quality. management. but in practice unless there is a group charged with responsibility for ensuring quality. and not cause a reduction in quality.Guide to the CASQ CBOK The key concept to establishing a quality function is establishing the need for quality. Since the dials are interconnected.

stabilizing.1. The SEI of Carnegie Mellon University states that it takes at least three years to move from SEI capability maturity Level 1 to Level 3.1.1. measuring. It normally takes at least two years in this phase of maturity for management and staff to recognize that quality cannot be tested into a product. While process improvement occurs before and after this phase. it is during this phase that resources are allocated to make process maturity happen.3 Final Phase During this phase.1 How the Quality Function Matures Over Time The MBNQA program has stated that it takes five to seven years to mature a quality management system. 4. The emphasis is on defining. it must happen through process maturity. These processes are integrated to obtain maximum customer satisfaction at minimum cost." Herodotus 484-424 4.1 Three Phases of Quality Function Maturation The maturation of the quality management system can be divided into three phases: 4. are being continually improved. It takes between two and four years for significant process maturity. An organization in this phase is results-driven. In this phase: 4. It is during this maturation period that the role of the QA analyst significantly changes from quality control to quality assurance to quality consulting. objectives such as consulting. At this level products and services achieve consistency in product quality (consistency being the prerequisite to improved quality and productivity). In this phase: 4. Version 6. and are producing high-quality results that yield high customer satisfaction.1.1.1. World-class begins when work and management processes are well defined. the bitterest is this: to know so much and have control over nothing. and improving work processes. and benchmarking move the organization toward optimization.1 4-3 . motivating. The MBNQA program defines such an organization as a “world-class” organization.2.1 Initial Phase The initiation point of the quality management system can be considered the formalization of quality control activities. focusing on defining and controlling product quality.Quality Assurance "Of all men's miseries.1.1.2 Intermediate Phase In this phase an organization's objectives move from control to assurance.1.1. Other individuals performing these roles also change as the organization's quality management philosophy matures.1.1.

Guide to the CASQ CBOK

4.1.2 IT Quality Plan
Quality planning is discussed in Skill Category 5. The IT Strategic Business Plan should include the following: The mission, giving a detailed description of what business IT is in Long-term IT goals giving direction for IT in the next five years Short-term objectives for the next business year How the objectives will be accomplished, including IT projects for the next business year • Organizational renewal programs that will assure the long-range success of the organization • The resources necessary to accomplish the short-term objectives and the organizational renewal activities that will enable the long-term goals to be achieved An IT Quality Plan has two objectives: • Supporting the organizational quality policy • Ensuring the high quality achievement of the IT Strategic Business Plan. The IT Quality Plan should include the following: A reference to the organization's quality policy An assessment of where the organization currently stands in relation to accomplishing the quality policy • Long-term quality goals - these are discussed below • Short-term quality objectives (i.e., programs) - these are discussed below • The means of implementing the quality objectives • Resources required in order to accomplish the short-term objectives and long termgoals A major reason for the failure of quality initiatives is a lack of action. Many organizations have introduced quality principles through generalized education and other departmental-wide awareness and motivational sessions. However, at the end of these sessions nothing has changed. In fact, these sessions often demoralize the staff because they recognize the benefits that could be achieved if action was taken. Quality is a long-term strategy, and any successful quality program must balance the long-term strategy of building a quality management environment with the short-term need for quick payback. Long-Term Actions The quality function should have a long-term plan of action to become a champion for moving the IT function to a world-class organization. While the short-term plan focuses on specific work tasks, the long-term plan is more complex and should incorporate these three objectives: • • • • • •


Version 6.2.1

Quality Assurance Building a quality management environment Supporting the implementation of the IT function's quality policy Assisting management and staff in closing the two quality gaps (see Skill Category 1 One of the best case studies in long-term planning is the Ford Motor Company. In a short period, from the late 1970s to the mid-1980s, Ford Motor Company went from losing hundreds of millions of dollars per year to becoming the most profitable corporation in the world. This happened under the guidance of Dr. W. Edwards Deming. His plan required Ford Motor Company to develop a mission, vision, guiding principles, and corporate values, and then live by them. Dr. Deming placed William Scherkenbach in the Ford Motor Company as the quality manager. His duties were to make sure that the management and staff of the Ford Motor Company were doing those things necessary to accomplish Ford's vision. Short-Term Actions Each quality manager must assess his/her own organization to identify quick-payback opportunities. The following short-term actions have proven beneficial in demonstrating that positive results can be achieved from quality initiatives. • Involve Management in Quality The first of this two-part action is to help management get beyond quality as an abstract concept by educating them in the "how-to" aspects of quality management. The second part gets management involved in performing quality tasks, large or small. For example, management can post a paper on a bulletin board asking IT staff what prevents them from doing their job effectively, and then select tasks to perform from this list. • Redefine a Problem Process Process definition should not take longer than just a few days to accomplish. In some organizations a small process can be defined in a day or two. Redefining a process should not usually take longer than five days. It is best to select a process that is performed in a variety of different ways, none of which appears to be completely successful, and to select a team that consists of individuals with a vested interest in the process. The process definition team chooses the best pieces of all of the processes, and puts them together to create the best of the best practices within the newly defined process. The team reviews and evaluates the process based on the following criteria: • Its value to the users (criticality of process) • How current the process is • Usability of the process • Measurability of the process • Attainability of adherence to the process in the existing environment Find and Fix a Problem • • •

Version 6.2.1


Guide to the CASQ CBOK The cost of quality in most IT functions is about 50%. This means half of the budget is already devoted to finding and fixing problems. Unfortunately, the fix is usually to the work product and not the process. Measurement (see Skill Category 8) is essential to determine what problems need to be fixed, and is a prerequisite to improvement. It requires identifying a product, and then counting the number of defects, providing an accurate record of the types and frequencies of product defects. Analyzing this record identifies recurring defects and their root causes. Eliminating the root causes eliminates the defects, and results in process improvement. • Improve quality control An important step in process definition is to identify the most logical points in the process to add a control. Skill Category 6 discusses this further. "Overriding all other values is our dedication to quality. We are a marketdriven institution, committed to our customers in everything we do. We constantly seek improvement and we encourage the unusual, even the iconoclastic." Louis V. Gerstner, Jr., CEO, IBM Corporation

4.2 Quality Tools
A tool is defined as a vehicle that assists in performing a task. Some tasks that a quality management organization will be performing where quality tools can be used are: • Defining a mission, vision, goals, and objectives • Defining Do and Check processes • Defining measures • Collecting data • Problem-solving • Designing solutions • Improving processes • Measuring results Quality tools can be categorized many different ways. For this presentation the following three groups have been selected. The tools described in each of these categories are a subset of existing tools. They have been included because they are more common and experience has demonstrated their effectiveness. • Management Tools


Version 6.2.1

Quality Assurance These tools are based on logic rather than mathematics, to address idea generation and organization, decision-making and implementation. • Statistical Tools These tools have a mathematical focus, usually related to data collection, organization, or interpretation. They may also be separated into tools used for counting and tools used with measures. Presentation Tools These tools are used during presentations to summarize or graphically illustrate data. These tools may be used in the development of written materials, such as proposals or reports, or to accompany oral presentations.

The three steps needed to select and use a quality tool are: 1. Select the Tool The general process for selecting a tool is to first define the objective (what is needed to perform the work task more effectively and efficiently). Next, study the tool description to determine whether the need objectives match the tool objectives. Finally, assure that the user of the tool believes it meets the objectives. 2. Learn the Tool If applicable, the person using the tool must receive some training. Reading through the tool’s documentation is the minimum. If possible, get classroom training or take a selfstudy course. Many tools are not only valuable in quality improvement, but can help individuals in the performance of their day-to-day work. Dr. W. Edwards Deming frequently stated that individuals knowledgeable in quality tools tend to be better onthe-job performers. 3. Use the Tool in Performing the Work Practice The tool should be utilized in the manner in which it is taught. The user should ensure that there is an approach for deploying and using the tool, and that the results meet the objectives.

4.2.1 Management Tools
Tools in this category are used to help generate ideas and information, to organize the ideas or information, to facilitate making decisions about the information, and to aid in the implementation. These tools are broad in nature. While they are not based on statistics, some, such as cause-andeffect diagrams and benchmarking may be used in conjunction with statistical tools. The tools to generate or organize ideas and information are: • • • Brainstorming Affinity Diagram Nominal Group Technique

Version 6.2.1


Guide to the CASQ CBOK • • • • • Cause-and-Effect Diagram Force Field Analysis Flowchart and Process Map Benchmarking Matrix Brainstorming Brainstorming is a technique used to quickly generate a quantity of creative or original ideas on or about a process, problem, product, or service. Brainstorming can be used to: • Develop a vision • Review inputs, outputs, and flows of existing processes • Create a list of products or services • Eliminate wasteful and redundant work activities • Re-engineer a process, product, or service • Design a new or improved process • Establish standards, guidelines, or measures • Identify the internal and external customers served • Improve the work environment • Gather data for use with other tools A brainstorming session begins with a facilitator establishing basic ground rules and a code of conduct. Typical brainstorming rules state that all members have an equal opportunity to participate, there is no criticism or pulling rank, people should think creatively, no idea will be treated as insignificant, and there should be only one conversation at a time. Members need to be active participants, willing to share their ideas, opinions, concerns, issues, and experiences. Next the team agrees on the topic to be brainstormed and whether to give ideas verbally or written on individual index cards, or any other easily manipulated medium. Either a structured (round table) or unstructured (free-flowing) approach is selected. Ideas should be generated quickly (5-15 minutes) and are recorded clearly on a flip-chart or board. The process stops when ideas become redundant or infrequent. Recorded ideas are reviewed for duplication and clarification, and eliminated when necessary. Remaining ideas are then evaluated with an open mind and may be used with the affinity diagram, nominal group technique, or cause-and-effect diagram. Affinity Diagram The affinity diagram is an orderly extension of a structured brainstorming session. Teams use this tool to help create order out of chaos, by categorizing large numbers of ideas. Rather than having teams react logically to a group of ideas, this technique helps to identify more creative solutions or to structure ideas for a cause-and-effect diagram.


Version 6.2.1

Quality Assurance Possible topics or problem statements where affinity diagrams could help are: • Why policies don’t exist • Why standards are not adhered to • Why QA failed • Why objective measures aren’t used • Understanding the leadership role in quality management • Why employees are not involved or lack empowerment • Why quality doesn’t work • Improving teamwork in the workplace • Understanding the issues to automation and use of CASE tools To generate affinity diagrams. Randomly place each index card on a flat surface. Generate a list of ideas. Sample ideas that could be ranked with the nominal group technique are: • Which defect is the greatest? • Who are our customers? • What are our impediments to quality improvement? • What new standards are needed? • What are our key indicators? • What tool is not being used effectively and how can we increase usage? • How do we get a quality tool used? Nominal grouping uses a round table (verbal) or index card (written) method for equal participation of teams or groups. and then achieve consensus by combining the individual rankings. wallboard or flipchart. 4. 3. issues.3 Nominal Group Technique The nominal group technique is a structured. As a team. discuss each category.1. Brainstorming can be used if the list is not readily available. It is a good technique to gather large amounts of information. concerns. discuss and then label each category with a title. facilitated technique where all team members participate by individually ranking ideas. and solutions. The steps for the nominal group technique are: 1. As a team. or solutions to prioritize. using cause-and-effect diagrams. issues. if needed.2. Write each idea on a separate index card. 5.2. Version 6. team members move the cards into meaningful groups until consensus has been achieved (the group stops moving the cards). 4. concerns.1 4-9 . continue with these steps after a brainstorming session: 1. In silence. 2.

such as: A – list item 1 B – list item 2 C – list item 3 Table 4-1 Results from Nominal Grouping List Items A – item 1 B – item 2 C – item 3 Member 1 2 1 3 Member 2 3 1 2 Member 3 1 2 3 Total 6 4 8 4. products. One represents the lowest (least important) ranking.1 . 4. link. clarify. A diagnostic approach for complex problems. Team members individually rank the list by assigning a number to each line item.1.2. Total the rankings of all team members. Through understanding problems within the work processes." “why-why diagrams. and services. If the list contains more than about 35 items. As shown in Table 4-1.” or "Ishikawa diagrams" (after Kaoru Ishikawa. In this example. teams can identify root causes of a problem. Through a series of "why-why" questions on causes. item “C” is the most important. a quality expert from Japan). A cause-and-effect diagram visualizes results of brainstorming and affinity grouping through major causes of a process problem. 5. it may be desirable to shorten it using Pareto analysis to make it more manageable. They are also referred to as "fishbone diagrams.Guide to the CASQ CBOK 2. this technique begins breaking down root causes into manageable pieces of a process. Figure 4-2 displays a sample cause-and-effect diagram. and classify possible causes of problems in processes. 4-10 Version 6. prefacing each item with a letter of the alphabet.2. record the remaining listed items in a location visible to the team. this process can uncover the lowest-level root cause. and higher numbers signify increasing importance. identify.4 Cause-and-Effect Diagram Teams use cause-and-effect diagrams to visualize. 3.

6. Materials. Review the completed cause-and-effect diagram with the work process to verify that these causes (factors) do affect the problem being resolved. Work on the most important causes first. which become “big branches”. 2. 4. Write the effect at the right side of the paper. 7. 5. but the categories vary with the effect selected. Fill in the “small branches” with sub-causes of each major cause until the lowest-level sub-cause is identified.2. Identify a problem (effect) with a list of potential causes. People.Quality Assurance Figure 4-2 Cause-and-Effect Diagram Cause-and-effect diagrams are applicable for: • Analyzing problems • Identifying potential process improvements • Identifying sources of defect causes • Analyzing improper use of test routines/testing problems • Scheduling problems/cycle times Developing a cause-and-effect diagram requires this series of steps: 1. Verify the root causes by collecting appropriate data (sampling) to validate a relationship to the problem. This may result from a brainstorming session.1 4-11 . Machines. Identify major causes of the problem. Teams may opt to use the nominal group technique or Pareto analysis to reach consensus. Six categories of causes are often used: Measurement. Version 6. and Environment. 3. Methods.

4. The technique is used to develop a common vision of what a process should do or look like. 4. Flowcharts may be a simple high-level process flow.2. driving forces on one side and restraining forces on the other.2. or workflow. or anywhere in between. Select the most significant forces that need to be acted upon using the nominal group technique to prioritize and reduce the number. 2. Brainstorm and list all possible driving forces. They are standard tools for any IT organization. ultimately the root cause.5 Force Field Analysis Force field analysis is a visual team tool used to determine and understand the forces that drive and restrain a change. Continue this process to identify all validated causes. 6.1. Following are sample processes that could benefit by a force field analysis: • Implementing a quality function • Implementing quality management in IT • Developing education and training programs • Establishing a measurement program/process • Selecting a new technique or tool • Implementing anything new • Establishing meaningful meetings • Empowering the work force The steps below show how a team uses force field analysis: 1.Guide to the CASQ CBOK 8. process. Flowcharts are most useful when applied by a team to obtain knowledge of a process for improvement. Determine the relative importance of reaching consensus on each force. Opposing forces prevent or fight the change. inefficiencies and redundancies can be identified and reduced. and. Draw a force field diagram that consists of two columns. Brainstorm and list all possible restraining forces. Proceed to a plan of action on the forces selected in the previous step.1 . 4. a detailed task flow. Once the process is documented. 7. 3.2. Driving forces promote the change from the existing state to the desired goal.6 Flowchart and Process Map A flowchart is a diagram displaying the sequential steps of an event. 4-12 Version 6. 5.1. Understanding the positive and negative barriers helps teams reach consensus faster. if there are too many. Establish a desired situation or goal statement.

Figure 4-3 shows a sample process map. testing processes.2. and their owners. change management. 5.1 4-13 . 6. Connect the tasks and steps for the function or activity. review processes. and events with no trigger activities. such as internal or external design. 2. Identify the tasks to support the function or activity. 4. 3. their relationships. The display of relationships and owners helps identify wasted steps. configuration management • Customer surveys or interviews • Supplier agreements or contracts • Service level agreements A flowchart is constructed using the following steps: 1. Sequence the tasks and steps for the function or activity. Identify the major function or activity being performed or needing to be performed. Figure 4-3 High-Level Process Map for Project Management Sample processes where flowcharts are useful include: Life cycle activities. redundant tasks. • Version 6. Create a flowchart of the tasks and steps for the function or activity using directional arrows or connections. Determine the steps needed to do the tasks.Quality Assurance A process map is a more detailed flowchart that depicts processes.

The three types of benchmarking are identified below. or functional (against “best in class” or “best in breed” within any industry). • Design a professional career ladder for information technology professionals.2. services. Benchmarking should be integrated with an organization’s process improvement process. Product Benchmarking • 4-14 Version 6. measurement.Guide to the CASQ CBOK 7. It helps achieve process improvement.2. The use of benchmarking is not normally associated with cost cutting or a quick fix. Flowcharts should reference the following information: • • • • • • • • • • • Process owners Suppliers Customers Key deliverables Decision points Interfaces Tasks and task sequence Policies Standards Procedures Tools used 4. • Process Benchmarking This benchmark is used to plan for business process improvement. It is the highest level of performance that fully meets customer requirements. Reach team consensus on the process depicted in the flowchart. competitive (against competitors within the same industry).1 . Benchmarking enables a company to identify the performance gap between themselves and the benchmarking partner.1. Benchmarking partners can be internal (against other company units). and practices measure up against others. motivation and a management process for improvement. • Identify and install measurements for quality and productivity.7 Benchmarking Benchmarking is the process of determining how well a company’s products. and is documented as part of business plans and quality improvement projects. Benchmarking has been used to: • Evaluate and upgrade the customer requirements process. and to determine a realistic improvement goal (some set higher goals) based on industry practices. The first two types account for about 80% of all benchmarking that is done.

or performance and those of the benchmarking partner.7. studying industry reports. obtain their agreement to participate. and interviewing consultants. make a managerial decision regarding performance goals for the organization. Steps 2 through 5 are iterative. professional groups. agenda. 4.7. 4. involving four phases.1.7. product. 4.Quality Assurance This benchmark is used to help in product planning and development. Develop questions and meet with the process owners to collect and record the data. and communicate the potential future performance levels to gain acceptance to move forward.1. as described below.1 4-15 .3 Integration Phase Step 6: Communicate Findings and Gain Acceptance Describe the benchmarking results to the process owners and involved parties.1. etc. using product documentation that includes the product performance goals and design practices identified through benchmarking.2. and confirm the visit time. Step 7: Establish Functional Improvement Objectives Version 6.2. and attendees with the benchmarking partner.2. and to project improvements required to close the benchmark “gap. Step 2: Identify and Select Benchmarking Partners Determine viable candidates for benchmarking. Step 5: Project Future Performance Levels Based on the competitive gap. interaction with industry groups.” Benchmarking is a ten-step process.2 Analysis Phase Step 4: Determine Current Competitive Gap Identify the difference between the attributes of the organization’s process. • Performance Benchmarking This benchmark is used to set and validate objectives to measure performance. Step 3: Collect Data Document the organization’s process to be benchmarked.1 Planning Phase Step 1: Identify Benchmarking Subject and Teams These internal or external candidates come from personal knowledge.2.

and develop strategies of multi-dimensional factors in solving problems. and continue the improvement cycle. especially in conjunction with the JAD process. not being met. and research is required to evaluate opinion versus fact. and process documentation requirements in advance. Matrix diagrams can be used to: • • Research or survey customer preferences Compare skill levels versus experiences in job 4-16 Version 6. Prepare objectives. meeting and benchmarking protocol. this approach could support the determination of system requirements. such as when teams need to understand and analyze customer preferences to drive out requirements or improvements on a product or service. Step 10: Re-calibrate and Reset Benchmark Performance Levels Based on the analysis. and make adjustments as necessary.8 Matrix A matrix is a structured. For IT organizational teams. problem-solving technique used to show the relationship between groupings. and clarify the strength of the relationships. 4. attendees.4 Action Phase Step 8: Develop an Action Plan Plan improvement using the organization's process improvement process.Guide to the CASQ CBOK In conjunction with the process owners. The matrix data analysis is frequently used to identify whether customer needs are being met. Step 9: Implement Plans and Monitor Progress Perform the plan. evaluate.2. establish specific improvement objectives to be achieved. 4.1. These are generally short-term objectives not to exceed one year. agenda. The matrix data analysis allows a team to test. and facilitate the benchmarking session to keep it on track.1.2.7. measure progress. It always takes longer than you think. focus on what is important. benchmark again to find better ways of executing the process. data. For multi-dimensional problems. This tool helps view a problem as a whole for clarity.1 . this approach focuses on the essential factors in problem areas.2. It is not easy to identify the IT “best of the best” because good performance data is not readily available. different. set new objectives. Lessons learned from the benchmarking leaders include the following: • • • • Focus on a specific objective and process. It helps teams sort language information for discussion and consensus. or no longer exist. It is also known as a matrix check sheet or a matrix diagram.

Listed along the top half of the left side is one set of items (such as causes) and listed along the bottom half of the left side is the other set of items (such as results). effort. such as two characteristics of a process or product. The relationship between the attributes and objectives helps clarify how to prioritize the objectives. The resulting matrix is in the shape of a “T”. An L-type matrix compares two sets of items to each other or compares a single set to itself. Create a matrix (tabular) structure with enough rows and columns to accommodate the two lists.Quality Assurance • Evaluate tools available versus usage • Correlate defect rates. For the T-type matrix.2. showing attributes of an improvement objective. and skills • Understand tasks in a process versus goals and resources The two common types of matrices are the L-type matrix and the T-type matrix. Typically this is shown with numbers or with relationship symbols such as shaded circles. a T-type matrix is generated the same as an L-type matrix. 4. Table 4-2 is an L-type matrix. none). view. use the following steps: 1.2 Statistical Tools The tools covered in this section are used to collect. probable. A T-type matrix is used to compare two sets of items to a common third set. 2.1 4-17 . 5. Complete the matrix by putting the relevant symbols in the corresponding boxes. circles and triangles (indicating strong.2. cycle times. With the exception of the format. 4. Total the scores if applicable. and analyze numbers. Determine the (usually) two lists of items to be compared. Put one list across the top of the matrix and one down the left side. They are: Version 6. Determine how the comparison will be symbolized. Table 4-2 L-type Matrix Improvement Objective Implement Unit Testing Define Coding Standards Implement Design Reviews Build Usability Lab Contribution (1-5) 5 3 5 2 Readiness (1-5) 5 1 5 1 Capability (1-5) 1 4 5 1 Cost/Benefit (1-5) 3 2 3 4 Score 14 10 18 8 To produce an L-type matrix. 3. the common set of items is listed in a row across the middle of the matrix. such as observed attributes between causes and results.

4.1 Check Sheet A check sheet (also called a checklist or tally sheet) of events or occurrences is a form used to gather and record data in an organized manner. It may follow a Pareto analysis or cause-andeffect diagram to validate or verify a problem. such as defect occurrences. Establish a format for the data collection that is easily understood by the collector.2. or type • Documentation defects by type or frequency • Cycle times. This tool records the number of occurrences over a specified interval of time to determine the frequency of an event. 5. 4-18 Version 6. record. Clarify what must be collected objectively.2. 3. Ensure those involved understand the objectives so the collection process is accurate. The data is recorded to support or objectively validate the significance of the event. such as requirements to design or design to implementation • Conformance to standards • End user complaints of all types • End user surveys • Late deliveries To use a check sheet: 1. Observe. Figure 4-4 shows a sample check sheet (Daily System) Failures Week of dd/mm/yy Day 1 Day 2 Day 3 Day 4 Day 5 Total Figure 4-4 Check Sheet Check sheets can be used to record the following types of information: • Project review results. 6. Instruct or train data collectors for consistency. location.2. Establish the sample size and time frame of data collection. 2.1 . or it may be used to build Pareto charts or histograms.Guide to the CASQ CBOK • • • • • • Check Sheet Histogram Pareto Chart Run Chart Control Chart Scatter Plot 4. and collect data.

cycle times. build a Pareto chart or histogram or evaluate the results to determine whether the original analysis is supported. Figure 4-5 shows a simple histogram. Tally the results. Histograms are also useful for displaying information such as defects by type or source.2. or end user survey responses. Disadvantages might be their applicability or limiting of other questions. 4.2. Version 6. and bias should be avoided. It provides a way to measure and analyze data collected about a process or problem. Depending on the purpose. The person using the check sheet should understand the reason for the questions and be able to anticipate a response. Interval 0-3 3-6 6-9 Tabulation 11 111111 111 Frequency 2 6 3 Cumulative Frequency 2 8 11 Figure 4-5 Histogram A histogram requires some understanding of the data set being measured to consolidate or condense it into a meaningful display. Gather data and organize it from lowest to highest values. experience or skill levels. To create a histogram.2. and documented approach. Questions on check sheets should be organized by topic and tested prior to use.2 Histogram A histogram (or frequency distribution chart) is a bar graph that groups data by predetermined intervals to show the frequency of the data set. and may provide a basis for what to work on first. and provide a consistent. perform the following steps: 1.1 4-19 .Quality Assurance 7. delivery rates or times. 8. limit the scope. A response of “I don’t know” should be allowed for. Advantages of check sheets are that they pre-define areas to discuss. organized.

and postimprovement charts Sample problems for Pareto analysis include: • Problem-solving for the vital few causes or characteristics • Defect analysis • Cycle or delivery time reductions • Failures found in production • Employee satisfaction/dissatisfaction The process for using Pareto charts is described in the following steps: • 4-20 Version 6. cogwheel. isolated island. 4. Count the data points of each cell (frequency) to determine the height of the interval. 6. 7. determine the number of cells. Calculate the interval or width of the cells.2. Other shapes such as double peak. Based on the number of observations. The vital few problems and their respective root causes can then be focused on. Sort the data or observations into their respective cells.3 Pareto Chart The Pareto chart is a special type of histogram. 3.Guide to the CASQ CBOK 2. used to view causes of a problem in order of severity from largest to smallest. and create a frequency table. time.1 . cliff. This technique provides the ability to: Categorize items.2. The distribution is normally a bell-shaped curve. etc. position. which is normally between 7 and 13.. within plus or minus 5 of budget) to show whether the actual values lie within the acceptable range.g. and skewed can provide insight on the variability of the process. manpower.2. A Pareto chart is typically used early in the continuous improvement process when there is a need to order or rank problems or causes by frequency. Then add the range of acceptable values (e. 5. process. usually by content (type of defect. which is the range divided by number of cells.) factors • Identify the causes or characteristics that contribute most to a problem • Decide which basic causes of a problem to work on first • Understand the effectiveness of the improvement by doing pre.) or cause (materials. operating methods. Joseph Juran refers to this Pareto principle as the separation of the “vital few” from the “trivial many”. which is the largest value – smallest value. measurement. It is a simple statistical tool that graphically shows the 20-80 rule where 20% of the sources cause 80% of the problems. etc. 4. Plot the results. Calculate the range. One variation on the histogram is to create a graph by drawing a line from the midpoints of the bars.

5. or cause-and-effect diagrams to define the problem clearly. counts. where the “x” axis is the problem category and the “y” axis is frequency. 4. Collect a sufficient sample size (at least 30 occurrences) of data over the specified time.2. or use historical data. Determine the vital few causes to focus improvement efforts. if available.4 Run Chart A run chart as shown in Figure 4-6 is a graph of data. affinity diagrams.2. Sort the data in descending order by occurrence or frequency of causes characteristics. The data represents measures. Compare and select major causes. 3. repeating the process until the problem’s root causes are reached sufficiently to resolve the problem. 6. or percentages of outputs (products or services) from a process. Run charts can be used as input for establishing control charts.Quality Assurance 1. Run charts are often used to monitor and quantify process outputs before a control chart is developed. in chronological order that displays changes and trends in the central tendency (average). Construct the Pareto Chart and draw bars to correspond to the sorted data in descending order.2. Figure 4-6 Run Chart Run charts can track events such as: • • • • • • Total failures Complaint levels End user satisfaction levels Suggestion levels Training efforts Production yields Version 6. 2. Use brainstorming. 4.1 4-21 .

Guide to the CASQ CBOK • Number of invoices • Number of system errors • Down time (minutes. There are two types of variation: • Common or random causes of variation These are inherent in every system over time. 3. Resolving common cause problems requires a process change. 5. 4. 4. Note: Variation is briefly described here to put control charts in perspective. Processes containing special as well as common causes of variation are referred to as unstable processes. 4-22 Version 6.2. and are a part of the natural operation of the system. improve process analysis and allow process improvements to be based on facts.2. They result from some special circumstance and require changes outside the process for resolution. Accepted practice uses a width of three standard deviations around the population mean (µ ± 3) to establish the control limits. A process containing only common causes of variation is considered stable. Figure 4-7 and Figure 4-8 show control charts for stable and unstable processes.1 . Label the chart both vertically (quantity) and horizontally (time). tracking the data chronologically in time. Connect data points for easy use and interpretation. percent) The steps for developing a run chart are as follows: 1. Decide which output of a process to measure. The intent of a control chart is to determine if a process is statistically stable and then to monitor the variation of stable process where activities are repetitive. the process control limits and current process capability can be determined. which implies that the variation is predictable within the statistically established control limits. • Common causes of variation are typically due to many small random sources of variation. 2. The sum of these sources of variation determines the magnitude of the processes inherent variation due to common causes. Plot the individual measurements over time (once per time interval or as they become available). Skill Category 8 provides additional details on the topic of variation. They establish measures on a process.2. From the sum. Special causes of variation These are not part of the system all the time.5 Control Chart Control charts provide a way of objectively defining a process and variation. Note: The special cause falls outside of the control limits of the unstable process. Monitor the data points for any obvious trend.

1 4-23 . Use of control charts follows: Version 6. and cause-and-effect diagrams. it is typically evaluated first by brainstorming.Quality Assurance Figure 4-7 Control Chart of a Stable (In Control) Process Figure 4-8 Control Chart of an Unstable (Out of Control) Process Control charts are suitable for tracking items such as: • Production failures • Defects by life cycle phase • Complaint/failures by application/software • Response time to change request • Cycle times/delivery times • Mean time to failure When there is reason to believe a process is no longer stable. Pareto analysis.2.

Fifteen consecutive points between the centerline and inner one-third of the chart. such as how many or over what time frame. Construct the control chart based on the statistics. 2. From the initial evaluation. cycle times. upper control limit. standard deviation. It shows whether a relationship exists between two variables. etc. or maintenance. figures.2. Monitor the process for common and special causes of variation. If observed. Data that is variable (measured and plotted on a continuous scale such as time. Collect sample data. Check sheets can be used to gather data. such as defects.2. Determine the methods for sampling.) may require different charts. The five rules constituting a special cause are: • • • • • Any point outside the upper or lower control limit. 4-24 Version 6. 5. the process needs to be evaluated and analyzed for causes related to the situation.6 Scatter Plot A scatter plot is used for problem solving and understanding cause-and-effect relationships. 3. by testing how one variable influences the response (other variable). Six or more consecutive data points. cost. Any run of eight or more data points above or below the average value (centerline). 4. Select the appropriate type of control chart based on the characteristic to monitor. cost. indicating the average has changed.2. 4. and lower control limit. Analyze and calculate the sample statistics: average. The process is in control when observations fall within the control limits. Scatter plots are also called scatter diagrams or correlation diagrams. which are increasing (trend up) or decreasing (trend down).Guide to the CASQ CBOK 1. identify the characteristics of the process to monitor. failures. Five rules are used to determine the existence of special causes. 7.1 . 6. Two out of three consecutive points in the outer one-third control limit.

Version 6. or want to see reports during and after implementation. Circle repeated data points as many times as they occur. 5. Select the variable and response relationship to be examined. determine the appropriate scale to plot the relationship. such as: • Defect Level versus Complexity • Defects versus Skill Levels (Training) • Failures versus Time • Cost versus Time • Change Response versus People Availability • Defect Cost versus Where Found (Life Cycle) • Preventive Cost versus Failure Cost The steps for creating scatter plots are: 1. curvilinear. sometimes called stakeholders. 2.2. cluster. Figure 4-9 Types of Scatter Plots 4. The involved parties. Plot the results. Be careful when interpreting results – a frequent error in interpreting results is to assume that no relationship exists between a variable and a response because a relationship isn't immediately apparent. determine the sample size of the paired data. etc. Figure 4-9 shows a few typical patterns. linear. It may be necessary to take additional samples. Stakeholders include management. negative. 4. and the individuals impacted by the changed process. 3.Quality Assurance Scatter plots may be used to look for relationships. random. the individuals that will use the changed process. Gather data on the variable and response. or use specialized axes such as logarithmic scales. need to be convinced that a proposed change is beneficial. The pattern of the plots will suggest correlation: positive.3 Presentation Tools Presentations are an integral part of the change process.2.1 4-25 .

Figure 4-10 shows how maintenance costs for Project 1 fluctuate over time.1 . For example.000 $3. 4-26 Version 6.000 $1. which depicts the dollars spent on maintenance for three different projects of about equal complexity over a four-month period.000 $3. Table 4-3 shows a sample table.000 $9.000 $2.000 $1.2.000 $3.000 $8.000 Project 2 $2.Guide to the CASQ CBOK The five tools below are the more common methods for graphical presentation: • • • • Table Line Chart Bar Chart Pie Chart 4.2. Table 4-3 Sample Table Month January February March April Total Project 1 $1.000 Project 3 $2.000 $1.000 $2. The information is presented in row and column format.000 $2.3. Spreadsheet software can prepare these types of graphical presentations. Line charts can also be used to compare: Figure 4-10 Line Chart • Like Units – There could be a line for each of the three projects.2.000 4.1 Table Quality reports often use tables as worksheets presented to management.2 Line Chart A line chart is used to show direction of events.3.000 $6.

4. project 3 maintenance costs are illustrated. In Figure 4-11.1 4-27 .Maintenance for Project 1.2. Note that the same type of information can be presented in a tabular format.The total or average maintenance could be shown as another line on the chart. Figure 4-11 Bar Chart Version 6. or a bar chart.Quality Assurance • • Related or Fixed Variables .3 Bar Chart A bar chart is normally a two-dimensional chart using bars to represent values or items. a line chart. for the first four months of this year could be compared to the same time period last year. Like Periods .2.3.

representing different values or items so that the bar permits easy distinction between the two items 4. Figure 4-12 Pie Chart Pie charts can be used to view: • • • • Segments visually differentiated from one another Segments showing the percent of the whole (uses 100% of the total pie) Dollar volumes. stability is the norm and change is abnormal. It is not necessary to be highly precise if the numbers are placed within the pie. where each pie piece indicates how many dollars of the total dollars are included Items. That cycle needs to be reversed. The size of each pie segment should reflect the portion of the total represented by that piece.Guide to the CASQ CBOK A bar chart is particularly advantageous when: • • • A large graphic is desired Specific time frame periods are to be emphasized Two or more items are to be included. if quality and productivity are to be constantly improved.2.4 Pie Chart A pie chart graphically presents the components of a total population. In most organizations. 4-28 Version 6.3 Process Deployment One of the most difficult tasks facing any IT function is changing the way that function operates.2.3. such as claims processed by the processing district 4. The pie chart illustrated in Figure 4-12 uses percentages to show how four categories of information are spread among the population.1 . where each piece shows the number of items.

3. strategic.S. There are three deployment phases . there is no way of effectively implementing a change. but not whether they can be successful using the changed process. In a software engineering environment. This resentment sometimes leads to overt action to stop the change.S. Check and Act components of the PDCA cycle.1. thus deployment is a critical component of making the software engineering environment work. Change brings risk. Curt Reimann. (This is closely associated with the “not-invented-here” syndrome. stated that less than one percent of U. Dr.Quality Assurance People resist change for the following reasons: • • It is significantly more difficult to implement change than to develop the approach that will cause the change. Deployment is the process of implementing a new or improved approach to completing a work task.2. and tactical. National Quality Award Program. • • • • 4.1 4-29 . The person(s) who developed the current process may resent having that process changed. Management may be more committed to meeting schedules and budgets than to implementing change.3. Deployment is normally only effective in an environment that uses processes.assessment. Starting a quality management program forces management to rethink its quality responsibilities. and then make routine mistakes while learning. If there are no processes. it discourages them from wanting to try the changed process. The assessment and strategic deployment phases represent the Planning component of the PDCA cycle. Corporations have an effective quality program. People do not like change imposed on them. past director of the U. This change process is called deployment. and there is a higher probability that they will make errors using it.) Workers know they can be reasonably successful using the current process. coupled with the lack of measurement processes to assess the results of quality programs. Most quality experts believe the cause of ineffective quality programs is attributable to poorly designed or nonexistent deployment efforts. there is a natural resistance because the change is not his or her idea. 4. This phase answers the question “Where am I?” Version 6. When people spend time and effort to learn a new process. If they are not actively involved in making the change.1 Deployment Phase 1: Assessment The first step in the deployment process is to establish the current level of performance for both the environment (via general assessments) and the goal to be accomplished (via specific assessments). compliance to process is enforced.1 The Deployment Process Initiating change is only effective when change is implemented through a process. The tactical deployment phase represents the Do.

which is normally a step towards accomplishing the vision. • Someone must take the lead for making the identified approach happen. The tactical phase answers three questions: • • When the process is initially implemented. Deployment is a team effort. Deployment champion(s) is in place. If the deployment resources are inadequate. This phase results in a goal. or that its use will be minimal. it is always advantageous for the champion to be a senior manager. 4-30 Version 6. installation costs should be between $50.3. The questions “Where do I want to be (goal or vision)?” and “How am I going to get there (process)?” are answered in this phase. While the champion can be a highly respected staff member.1.3 Deployment Phase 3: Tactical As previously stated. • 4.2 Deployment Phase 2: Strategic The deployment strategy establishes the framework in which deployment will occur.1 . Approach is an intellectual exercise. There are five intangible attributes called critical success factors that help make deployment work. 4. For example. if a capture/playback testing tool is purchased for $ effectively performing the strategic deployment activities helps ensure the success of the deployment tactics. These five critical success factors for an effective deployment process are: 1.000 and $250. compliance is attempted. Without strategic planning. deployment is a people-intensive process. answering the implementation question “How do I get people to follow the process?” Measurement is performed to answer the question “Does the process work (is it under control) and is the organization moving in the right direction to support the vision or goal?” Based on the measurement. and a definition of the process or approach to accomplish that goal. there is a high probability that the approach will fail.2 Critical Success Factors for Deployment Deployment is much harder than defining an approach. It takes three to ten times more resources to deploy an approach than to develop it. and if so. which together enable approaches to be effectively implemented. Deployment is a series of integrated tasks.3.000.Guide to the CASQ CBOK 4.2. how?” is answered. 2. • These integrated tasks are a deployment process that should be customized as needed for each organization and each approach. 3. that tool will normally fall into disuse. deployment rarely works. the question “Does the process need improvement. If those funds are not expended.

The individual does not have to like the approach.e..4 Internal Auditing and Quality Assurance Both internal auditing and QA are professions.e. In this activity an individual accepts the approach as the way business will be done.1 Types of Internal Audits Internal auditing is a management control directed at measuring and evaluating an activity to determine if it is performed in accordance with the policies and procedures of an organization (i. and goes on until the approach is discontinued. There is buy-in by the affected parties. During that time. but can rarely deploy that approach. It is essential that new people involved in the work tasks have the same enthusiasm and desire that existed in the initial deployment effort.. meets the intent of management). deployed) likely will change over time. Deployment responsibilities are effectively passed between individuals and between teams.1 4-31 . • Deployment is a continuous process that begins prior to developing the approach. 4. 5. It is generally recognized that a profession has the following criteria: • Code of ethics • Common body of knowledge • Statement of responsibilities • Certification program (including continuing education) The differences between the auditing and QA professions are in the common body of knowledge and the statement of responsibilities. The specific types of auditing are: • Financial Auditing Financial auditing is performed in accordance with generally accepted accounting procedures and other applicable laws and regulations to determine that the accounting records are reasonable. • Tasks that transfer ownership of the approach to the users of the approach involve a buy-in.4. A team of people including instructors. 4. 4. Version 6. but does have to wholeheartedly support its use in performing the effective work tasks. People involved in ensuring that the approach is followed (i. colleagues and management must implement the deployment process. the level of enthusiasm for the approach will vary. technicians. It is an independent appraisal activity.Quality Assurance • A single individual can develop an effective approach.2.

QA. it strives to be independent of the activities being appraised. Internal auditors must be knowledgeable of the Standards for the Professional Practice of Internal Auditing and are required to comply with those standards in the performance of their work. Internal auditing cannot get involved in developing procedures. • Internal auditors review the means of safeguarding assets and verify the existence of assets.4. • Program Auditing Program auditing is performed to determine that the objectives of specific business activities are being properly fulfilled. • Internal auditors normally coordinate their activities and work in conjunction with the organization’s firm of external auditors. While QA performs many appraisals. Confusion between the two roles frequently leads to a negative image of QA. while the role of QA is to find and implement solutions for those problems. QA should be a leadership position. • Internal auditors verify compliance to corporate policies. should have a positive role.Guide to the CASQ CBOK • Operational Auditing Operational auditing is performed to determine that operations are performed in an efficient. effective and economical manner. plans. • Internal auditors have direct lines of communication to senior corporate officers and frequently to the organization’s board of directors. There are three important characteristics in the performance of an internal audit: 1. Some key activities performed by QA analysts that are not normally performed by internal auditors are: • 4-32 Version 6. 4. 2. has a negative role. or usurp the roles and responsibilities of other employees.1 . Auditing. 3. The internal auditor is to evaluate the interaction of all company groups with regards to meeting objectives. procedures. the regular company operations would go on in a normal manner for the time being. by practice. and applicable laws and regulations.2 Differences in Responsibilities The main role of auditing is to identify and report problems. emphasizing the strong interpersonal activities involved in making improvement occur.2. Some of the skills and activities that an internal auditor has are not applicable to QA analysts. A good practical test is that if the internal auditing activity were temporarily discontinued. The work of the internal auditor needs to be detached from the regular day-to-day operations of the company. by nature. standards.

2. and presenting analyses Performing process analysis (i. procedures. recording. statistical process control) See Skill Category 3 for a discussion of quality audits. Version 6.e.1 4-33 . summarizing. and standards Acquiring and implementing tools and methodologies Marketing or creating awareness of quality programs and concepts Measuring quality Defining..Quality Assurance • • • • • • Developing policies.

4-34 Version 6.Guide to the CASQ CBOK This page intentionally left blank.2.1 .

Skill Category

Quality Planning


xecutive management establishes the vision and strategic goals. Planning is the process that describes how those strategic goals will be accomplished. Quality planning should be integrated into the IT plan so that they become a single plan. In simplistic terms, the IT plan represents the producer, and the quality plan represents the customer.

Planning Concepts Integrating Business and Quality Planning Prerequisites to Quality Planning The Planning Process Planning to Mature IT Work Processes

page 5-1 page 5-4 page 5-5 page 5-6 page 5-13

5.1 Planning Concepts
Planning is the totality of activities that determine, for an individual or organization, what will be done and how it will be done. Quality planning is a component of overall business planning. Quality planning focuses on the policies, processes and procedures which assure that the defined requirements are implemented, and the implemented requirements meet the customer’s needs. The following two concepts epitomize the importance of planning. • • If you do not know where you are going, all roads lead there. This means that without a plan, any action is acceptable. If you fail to plan – plan to fail. This means that without a good plan which defines the expectations of work, activities may be performed which provide no benefit and lead to customer dissatisfaction.

Version 6.2.1


Guide to the CASQ CBOK Two important components of quality planning are the management cycle and the planning cycle. The management cycle, frequently referred to as the Plan-Do-Check-Act Cycle, is repeated here to emphasize the importance of planning as a management activity.

5.1.1 The Planning Cycle
Planning is a management responsibility. The responsibility commences when management establishes a vision for the IT organization, and works through the development of a tactical plan which defines the detailed work activities to be performed. The planning cycle is a decomposition of the IT vision into work activities which will help accomplish that vision. Table 5-1 shows that decomposition. It also shows the do, check, and act activities that follow when planning is completed. The planning cycle must be integrated with the do, check, and act activities because planning is a continuous activity. While the PDCA cycle implies that you plan, then do, then check, and then act, that concept is misleading. While the plan should be complete before work activities commence, business requirements may change, and problems or opportunities may be encountered. These events which affect work activities should be incorporated into a new version of the plan. These changes to the work activities can have any or all of the following impacts on the plan: • • • • • • Change the schedule Change the budget Change the number of resources allocated Change how one implemented component of software will affect other components of the software Change in work priorities Addition or deletion of work activities to accommodate the needed changed work activities


Version 6.2.1

Quality Planning Table 5-1 Planning Cycle Example to Show Decomposition from Vision to Rework
Planning Activity Establish IT Vision Define Mission PDCA Phase P P Example of Planning Activity IT deliverables and service exceed customer satisfaction. We will work with our customer to assure satisfaction. On a scale of five to one -- from very satisfied, satisfied, neither satisfied nor unsatisfied, dissatisfied, very dissatisfied – our goal is 90% of our customers very satisfied or satisfied. Involve users in the software development process. Conduct reviews at the end of each development phase with users as part of the review team. For project “x” conduct a requirements phase review on November 18, 20xx. Did the requirements phase produce testable requirements? Make non-testable requirements testable.

Set Goals


Strategic Planning Tactical Planning Execution Monitoring Rework


The planning cycle is illustrated with a customer satisfaction example showing the decomposition from the first listed planning activity to the last planning activity. • • • Establish IT vision A vision is broad in nature, probably un-achievable but it is the ultimate goal. Define Mission The responsibility of an organization unit related to achieving the vision. Set Goals A target established by management to be achieved by the staff of the implementing organization. Strategic Planning A description of what must be done in order to meet the goals set by management. Tactical Planning The detailed “how-to” work activities that need to be undertaken to accomplish the strategic planning objectives. Execution The execution of the tactical plan as written. Monitoring An ongoing assessment to assure that the plan is followed, and problems encountered in following the plan, are appropriately addressed. Rework Actions approved by management based on problems uncovered through the monitoring process.

• •

• •

Version 6.2.1


Guide to the CASQ CBOK

5.2 Integrating Business and Quality Planning
Quality planning should focus on two major activities: process management and quality control. Process management is discussed in Skill Category 6 and quality control is described in Skill Category 7. The quality professional will do other activities, most quality activities that require planning are related to these two quality activities. Business planning should focus on accomplishing business objectives. Let’s look at these two planning processes. The IT organization develops a business plan. The purpose of the business plan is to define the work and work activities that will be conducted during the planning period. These work activities are designed to accomplish the business objectives. The quality professionals will develop a quality plan focusing on quality activities that will help assure the outputs from the business plan meet the defined output specifications and meet the needs of the users of those deliverables.

5.2.1 The Fallacy of Having Two Separate Planning Processes
Many IT organizations develop both a business plan and a quality plan. However, they do not integrate these plans and, by not integrating the plans, undesirable results may occur. For example, the quality plan may call for system development reviews to occur prior to the end of a software development phase. However, if the business plan does not allot time and resources to participate in that development review, the review may never occur. In many organizations, the quality professionals who organize the reviews are not informed when a software development phase is concluding.

5.2.2 Planning Should be a Single IT Activity
Both the business staff and the quality staff should be involved in IT planning. Involvement is in both strategic and tactical planning. The objective of this single planning cycle is to ensure that adequate resources and time are available to perform the quality activities. The net result is that the individuals executing the business plan cannot differentiate the quality planning from the business planning. For example, if business planning calls for a quality review prior to the end of each phase of software development, the business staff will assume, that is a logical part of the software development process. If it is not integrated, the review is owned by the quality professional and may not be adequately supported by the IT business staff, such as, system designers and programmers. Figure 5-1 illustrates the integration of quality planning into IT business planning.


Version 6.2.1

Quality Planning

Figure 5-1 Integrating Quality Planning into IT Business Planning The figure gives an example of business planning and an example of quality planning. While the blocks show that strategic and tactical business planning and strategic and tactical quality planning occur, the end result is a business plan that incorporates quality activities.

5.3 Prerequisites to Quality Planning
Quality planning is a process. Quality planning should be a defined process indicating who is involved in planning and the specific work procedures and deliverables included within the planning process. Individual IT staff members should not create their own planning process. Before effective quality planning can occur these prerequisites should be met: • IT vision, mission and goals documented Those planning need to know specifically what the plan is to accomplish. The planning process begins by knowing the desired output from the plan. If those performing planning understand the IT vision, the IT mission and the specific goals related to that vision and mission, they can develop a plan that hopefully will meet those goals. Defined planning process

Version 6.2.1


Discussed below are the planning activities that are most frequently identified as important to effective planning. There are literally hundreds of different books on planning. or working under a mentor to teach effective planning. the systematic planning process described in this section provides a great wealth of 5-6 Version 6. planning standards. Reliable information required The plan will be no better than the information used to create the plan. Normally both should occur. they should take whatever steps necessary to assure them that they’re working with valid and reliable information. The competency can be achieved by training.4 The Planning Process The planning process is the same for business planning and quality planning. If the process is not working or not effective the process should be changed. Planners competent in the planning process Those who will use the planning process to plan need to be competent in using the planning process.Guide to the CASQ CBOK The IT organization needs a planning policy. Compliance to the plan If a planning process exists.4. refer to Skill Category 6. If those performing quality planning cannot rely on the information provided them. and planning procedures.2. management should require compliance to that process. these three areas: • • • Planning process overview Basic planning questions Common planning activities 5. • • • • 5.1 Planning Process Overview While there is no standard off-the-shelf plan for planning that can be universally applied to every situation. Management support means that the plan must be completed and approved before resources will be allocated to accomplish the work defined by the plan. as well as. Maintenance of the planning process Planning processes should continually be improved. • Management support for planning Effective planning will only occur when management requires the plans be developed using the IT organization’s planning process. To better understand the attributes of an effective process.1 .

But planning will still be poor in spite of good information unless each manager is able to coordinate adequately with his/her associates and consolidate his/her planning to support the IT objectives. Figure 5-2 The Planning Process This planning process is divided into the following ten planning activities: • • • Business or Activity Planning Environment Planning Capabilities and Opportunities Planning Version 6. retrieve it and distribute it on a controlled basis for decision-making. This process can be adapted to most organizations and thus avoid the necessity to “reinvent the wheel.Quality Planning experience. economical way to collect the process data. retain it.2. concepts and materials. This planning process as illustrated in Figure 5-2 provides an easy.1 5-7 .” The quality of planning and decision-making cannot consistently rise above the quality of the information on which it is based.

The planning process then documents the answers to these six questions: • • • • • Where are we? Where do we want to go? How are we going to get there? When will it be done? Who is responsible for what? 5-8 Version 6. The primary purpose of planning is not to produce a rigid plan. The whole must be greater than the sum of its parts. but to facilitate intelligent delegation with responsible participation by providing a better method of reaching and revising agreements. 5. many managers abhor planning. The practical.2 The Six Basic Planning Questions The ten planning activities described in Figure 5-2 were designed to answers six basic planning questions as listed below. whatever ability it has gained to tackle new tasks has been achieved by making things simple through system.” Obviously. A stagnating one soon dies. to reduce to principles and concepts what has been left to experience and ‘rule of thumb.4. Whatever progress the human race has made. it minimizes surprise and optimizes performance in a changing environment. The common reason heard most frequently is that planning systems cannot be installed because “our business is a dynamic one which is changing daily. there is probably not a single. If one does seemingly fall into such a category.1 .’ to substitute a logical and cohesive pattern for the chance recognition of elements.” The following material in this section identifies the contents of the planning activities illustrated in Figure 5-2 above. Most specifically. Weaknesses and Tactics Planning • Priorities and Schedules Planning • Organization and Delegation Planning • Budgets and Resources Planning Like most important things in life. viable business today which is not changing. Yet.Guide to the CASQ CBOK • Assumptions and Potentials Planning • Objectives and Goals Planning • Policies and Procedures Planning • Strengths. systematic approach to planning was best described by Peter Drucker in “The Practice of Management:” “There is only one answer: the tasks must be simplified and there is only one tool for this job: to convert into system and method what has been done before by hunch or intuition. it is impossible to describe the scope and purpose of this approach to planning by enumerating the bits and pieces.2. it is stagnating.

profits. Identify specific milestones to measure progress on a month-by-month basis. Define tactics. information Capabilities and Opportunities Planning 2. Plan now for your organization requirement 2-3 years from now so you have the right person.2. at the right place. What are important in understanding an effective planning process are the activities that must be performed and the information needed to develop an effective plan. Where are we? (Historic and current information.1 5-9 .g. From a quality profession perspective. products. Select tactics that maximize strengths and minimize weakness. productivity objectives. expenses. and facts) Environment Planning (External to Company) Planning Information Needed Nature of Business-Purpose. Political. present time. and Tactics Planning 4. weaknesses – internal/controllable) Problems (external/partially controllable) Opportunities Analysis by Key Result Areas Temporary future estimates of probable developments beyond our control. How are we going to get there? Strengths. Products. cannot be predicted with accuracy) Assumptions and Potentials Planning Objectives and Goals Planning Policies and Procedures Planning 3. Industry Regulations and Laws Identify and Analyze input on other organizations Capabilities (strengths. in the right way at the right time. any planning process that works would be an acceptable response to any question on the CSQA exam Version 6. Table 5-2 The Six Basic Quality Planning Questions Six Basic Questions Planning Activities Business or Activity Planning 1. For example. Who is responsible for what? Organization and Delegation Planning 6. How much will it cost? Budget and Resources Planning There are a large number of planning processes that are effective. Specify organizational relationships.Quality Planning • How much will it cost? Table 5-2 shows how each question links to one or more of the ten planning activities and the information that is needed to answer the question and perform the activities. Organization and IT work environment Economic. Where do we want to go? (Dealing with the future. market potentials. Capital budgets by month and by year List of major resources—dollars. Quantified measurable objectives (5-year and fiscal year month-by-month). Assign order of accomplishment for programs. The operational budget should place price tags on the tactics. Temporary estimates of desirable results achieved by our own efforts. populations. facilities. Monthly operating budgets by department. Social. e.. revenue. interest rates. doing the right work. History Management Philosophy Profiles of Business-Revenues. organizational charts. and responsibility profiles. etc. government regulations and impact of competitive actions. When will it be done? Priorities and Schedules Planning 5. Weaknesses. Scope. Specify who is responsible for the program of action and identify areas of decision-making and the accompanying authority required to accomplish the programs. Profits. Current policies/procedures hindering performance Required policies/procedures to improve performance Strategy is a course of action selected from among alternatives as the optimum way to obtain major objectives.

The planning activity needs to identify the capabilities and competencies of the individuals who will develop the products. Second to perform a self-assessment on your organization’s planning process to identify potential improvement opportunities.1 Business or Activity Planning The purpose of this planning activity is to both define the planning objectives. Specifically.4. this planning activity should address: • • • The environment established by the organization and the IT function that impacts the means by which work is performed Laws and regulations affecting the products produced and operated Other organizations and systems that are interfaced or impacted by the products being developed and operated (e. The criteria need to be ranked so the proper emphasis can be placed on the most important criteria.. 5. The description of each of the planning activities that follows can be used for two purposes.g.4. 5. mission and goals Who are the customers/users What are the business needs of the customers/users Interfacing software systems Profile/description of customer/user activities 5.1 . The business opportunities that can be achieved by the project also need to be identified.Guide to the CASQ CBOK about planning. QAI recognizes that quality professionals should follow the planning process adopted by their organization.2. payroll systems automatically sending tax information to governmental agencies) Capabilities and Opportunities Planning This planning activity needs to identify criteria that are required for the success of the project. Specifically. and to relate the plan to other documents that involve individuals. this planning activity should address: Critical success factors 5. while others will combine the ten into a fewer number. Environment Planning Planners need to know the environment in which the deliverables will be developed as well as the environment in which the deliverables will be operated. Specifically.3 The Common Activities in the Planning Process The ten planning activities listed in Figure 5-2 are common to most planning processes. this planning activity should address: • • • • • Vision.3.3. First to understand the planning model presented in this section. Some planning processes have more activities.3 • • 5-10 Version 6.

1 5-11 .3. These developments should be as specific as possible in terms of how much and when.4.2. If goals are not measurable it will be difficult to determine whether or not the project is successful. Specifically. some management philosophies believe in using stretch goals to push people to achieve a higher level of performance. Goals and objectives should be measurable and achievable.. procedures and practices) Changes needed to processes Existing processes or parts of processes. not applicable to this project Process variances needed and how those variances will be obtained Version 6.4 Assumptions/Potential Planning This planning activity identifies those probable developments that will affect the ability to achieve the project objectives. Specifically.) 5.4.g. or alternative strategies can be determined Quality and productivity goals 5.6 Policies/Procedures Planning This planning activity needs to identify all of the policies. this planning activity should address: • • • Assumptions which if not correct will impact the success of the project Current opportunities received from implementing the project How future opportunities will be identified during the implementation and operation time frame of the project 5. turnaround time.3.. If the goals are not achievable the assigned staff are in a “no win” situation. this activity should address: • • • Project objectives and goals expressed in quantitative terms Any qualifications for objectives that can impact the sequence of work.3.5 Objectives/Goals Planning Setting the project objectives/goals is an important component of the planning cycle.4. In this activity the planners are trying to describe what will happen in the next month or years so that implementation can seize any new opportunities that may develop.Quality Planning • • Strengths and weaknesses of the assigned staff IT’s ability to meet the project goals (e. procedures and practices that will impact the implementation and operation of the project. number of clicks to get information. standards. This analysis needs to determine whether that impact can be positive or negative. this planning activity should address: • • • • Documenting the processes to be used in implementing and operating the project (i. etc. Specifically. However. policies.e.

capital items such as software and hardware. This planning activity is normally the most time consuming activity as planners need to explore multiple strategies and tactics.4.3. 5.9 Organization/Delegation Planning The only way to develop people is to give them an opportunity to make decisions. what decisions (and limits) and related responsibilities are delegated to accomplish the task. this planning activity should address: • • • Required and realistic completion date Milestones that need to be met to finish by the scheduled completion date Sequence in which activities must be performed to determine whether or not the scheduled date can be met 5. Therefore.3. this planning activity should address: • • • Responsibilities for each employee assigned to the project Responsibilities of support staff/individual Agreement by the individual that those responsibilities are adequate and reasonable 5.4. education and training. Specifically.Guide to the CASQ CBOK 5.1 . These resources can be incorporated into a budget. and the tactics how to do it.4.8 Priorities/Schedules Planning This activity develops precise milestones that identify steps or activities that are essential to accomplishing the project in the sequence in which they must occur. Specifically. this planning activity should address: • • • • • Select preferred strategy among alternatives Select best tactics among alternatives Select tactics that maximize strength and minimize weakness Document tactics Get buy-in from those involved in the project. Specifically. a responsibility profile should be developed by each staff member and discussed and revised with management so that understanding and agreement are accomplished. and other monetary needs. If the completion date is predetermined then the milestones must consider what can be accomplished within the available time span. Specifically.3. information. this planning activity should address: • • Monetary resources needed Skills/competencies needed 5-12 Version 6. Specifically. support staff.3.10 Budget/Resources Planning The resources needed for projects include employees. Many organizations use critical path planning or equivalent for this planning activity.4.7 Strategy/Tactics Planning The strategy defines what to do.2. supplies.

5 Planning to Mature IT Work Processes A major component of most quality plans will be defining. and how to implement that model to mature IT work processes. all IT organizations. motivation processes and customer/user-focused processes. The Malcolm Baldrige National Quality Award Model includes these.2. Therefore. Please note that no one model is appropriate for. assure adequate competency in this area. Quality professionals need a model that represents the entire IT organization because it is the environmental components of the IT organization that drive quality. and improving IT work processes. deploying. or applicable to.Quality Planning • • • • Hardware/software needed Support needed Information needed Training needed 5. For information on those models. The quality plan should identify specific processes to be defined. Many industry process models are designed for software development and not an entire IT organization. However. leadership processes.1 5-13 . such as handling customer complaints. understand that industry or equivalent models. A quality plan should include a process model whose achievement would be a quality goal. quality professionals need to know the logical steps that IT organizations should follow in maturing their IT work processes.3. training processes. testing processes. Specifically. refer to Skill Category 3. some of the components missing from many industry process maturity models are management processes. deployed and improved with specific objectives and goals defined for those work processes. 5. Quality assurance should be involved in identifying needed new processes and where existing processes need to be improved. Others have selected the ISO Model. Version 6. but does not focus heavily on IT processes. To better understand the changes in planning when work is outsourced.11 Planning Activities for Outsourced Work Most of the planning activities that occur for a project developed in-house must also be performed for outsourced projects. refer to Skill Category 10. since the planners do not have direct control over a lot of the resources in the outsourced organization.4. Many organizations have chosen the SEI CMMI Capability Maturity Model as that goal. many of the planning activities need to be incorporated into contract negotiations.

This is because organizations know how to do the routine processes. Since there is a continuum of work processes.1. but the ability to recognize whether it is done correctly exists. illustrates that the defined routine processes are comprised primarily of detailed written procedures to be followed with minimal variation. As processes move toward creative. and thus how to sequence the implementation of the maturity of the six process categories. there is less knowledge on how to do it. from well-defined and routine work processes to highly creative work processes. Thus.2 Relationship of Do and Check Procedures Work processes are a combination of Do and Check procedures. for creative processes there may be minimal guidance on how to do it. Twelve relationships are presented for use in maturing your IT function to meet your specific improvement objectives and timetable. a strategy is provided on the sequencing of maturity. The twelve relationships are: • • • • • • • • • • • People skills and process definitions Do and check procedures Individuals' assessment of how they are evaluated to work performed What management relies on for success Maturity level to cost to do work Process maturity to defect rates Process maturity and cycle time Process maturity and end user satisfaction Process maturity and staff job satisfaction Process maturity to an organization's willingness to embrace change Tools to process maturity 5. and they are very dependent upon people skills.Guide to the CASQ CBOK 5. the six were selected because they are frequently managed by six different individuals or groups. but well- 5-14 Version 6. the mix of written procedures and people skills change as this continuum moves. and then how to check that what was done was done correctly. For the defined routine work processes. The worker is told what to do.5.1 How to Plan the Sequence for Implementing Process Maturity The QAI Manage by Processes Tactical model identifies six process categories. For each relationship.1. This section explains some of the relationships and strategies that are important to understand in maturing an information services function. . As explained earlier. while at the creative level the procedures are very cryptic and generalized. as well as a discussion on skipping levels and reverting back to lower levels. the Do procedures are very detailed and are designed to be followed with minimal variation.1 Relationship between People Skills and Process Definitions Professional work processes are a combination of the skill sets of the individual performing the work process plus the procedures and standards within the work process. 5.

(Note: They could also use cost to produce a thousand lines of code. and controls has associated with it a search for the root cause of problems. As the process maturity level increases. 5. there is a decrease in the cycle time to build software products.1. Many organizations now measure cost in dollars to produce a function point. can be developed. 5.1. The maturity of processes.7 Relationship of Process Maturity and Cycle Time There is a high correlation between process maturity and cycle time. which also significantly reduces cycle time.6 Relationship of Process Maturity to Defect Rates There is a strong correlation between process maturity and defect rates.Quality Planning defined processes to check to see if it is done right.1. On the other hand. 5. 5. Thus. and as people become more motivated.5. extends the cycle time. As the maturity level increases.5 Relationship of Maturity Level to Cost to Do Work Maturity level has a significant impact on cost. Many of these checking processes involve groups of peers. improvements also focus on facilitating transitions of deliverables from work task to work task. which are addressed by process Version 6.1. and so forth.5. Thus. Because immature processes have poorly defined standards of work. organizations at low process maturity levels tend to rely much more on individuals.4 Relationship of What Management Relies on for Success Current literature emphasizes the use of teams. 5. As processes mature. at low levels of process maturity. End users are dissatisfied for many reasons. 5. Many organizations measure defect rate in defects per thousand function points or defects per thousand lines of code. empowered teams.3 Relationship of Individuals' Assessment of How They are Evaluated to Work Performed People do what they believe they are evaluated on.1.5. even if those individuals are members of teams. most assessments of individual performance at maturity Level 1 are highly subjective.5. the defect rate drops significantly. in turn.) It has been estimated that an increase of one level in process maturity doubles an organization's productivity.8 Relationship of Process Maturity and End User Satisfaction There is significant research to support the premise that end user satisfaction increases as process maturity increases. If they are evaluated on meeting schedules.2. people. It is these problems that lead to rework which.1 5-15 . if they are evaluated on producing defect-free code.5.5. people believe they are subjectively evaluated and focus their attention on organizational politics. they produce defect-free code.1. controls become more effective. while at the highest levels of process maturity their emphasis switches to the results they are paid to produce because those results can be measured against specific standards. the cost per unit of work decreases. and self-directed teams. the defect rate decreases. they meet schedules. As processes become better defined.

One is variability. reduces defect rates. The reason for this is that stabilized effective processes increase an individual's ability to be successful. The facts that process maturity increases end user satisfaction. and if the organization's processes make them successful they are motivated and anxious to stay with that organization. 5. People resist change for a variety of reasons. 5-16 Version 6. are all contributors to an individual's personal success.1. In addition.9 Relationship of Process Maturity and Staff Job Satisfaction Staff job satisfaction increases significantly as processes mature.2. However. 5. holds organizations at lower maturity levels. 5.1.1 . Individuals tend to want to work in an organization in which they are successful. and reduces rework and cost. End users are also dissatisfied by high costs and long cycle time. Levels 3 through 5 have a significant impact on value received from the information areas.5.Guide to the CASQ CBOK maturity.5.5. tools tend to be optional and not well taught. but others relate to a lack of confidence that the change will both improve the way work is done and have a positive impact on their career. At Level 1. meaning that the service or product they receive one time is significantly different from product received at a later point in time.11 Relationship of Tools to Process Maturity Tools are a vital component of maturing processes. people gain confidence in the processes and the willingness of management to reward for results. The lack of good tools. end users are dissatisfied when the information system does not provide significant value in accomplishing their mission. as processes mature. There is a strong relationship between the acquisition and integration of tools into the work processes and the movement from process maturity Level 1 to Level 5.1.10 Relationship of Process Maturity to an Organization's Willingness to Embrace Change There is a high correlation between process maturity and an organization's willingness to embrace change. Some of these are personal. The willingness to change parallels process maturity but lags slightly at the lower levels of maturity and accelerates during the higher levels of maturity. which are both reduced by process maturity. and the lack of consistent use of those tools.

1. deploying and maintaining the work processes used to achieve the IT mission. What is referred to as process management in this skill category is also called process engineering and the standards program. Defining and continuously improving work processes allows the pace of change to be maintained without negatively impacting the quality of products and services. Companies must constantly improve their ability to produce quality products that add value to their customer base.1 6-1 . 6.Skill Category 6 Define. The level of Version 6. quality and speed of delivery are now critical needs.1 Process Management Concepts Process management is a term used by many IT organizations to represent the totality of activities involved in defining. T Process Management Concepts Process Management Processes page 6-1 page 6-10 6.2. and Improve Work Processes he world is constantly changing. specifying the methods used to produce a product or service. building. It is the set of activities that represent the way work is to be performed.1 Definition of a Process A process is a vehicle of communication. Customers are more knowledgeable and demanding and. Implement. Build. therefore.

productivity.2 Why Processes Are Needed Processes add value to both management and the workers. and customer satisfaction by having workers define and improve their own work processes • Free management from activities associated with "expediting work products" to spend more time on activities such as planning. From a management perspective. work processes are important to: • • • • Increase the probability that the deliverables produced will be the desired deliverables Put workers in charge of their own destiny because they know the standards by which their work products will be evaluated Enable workers to devote their creativity to improving the business instead of having to develop work processes to build products Enable workers to better plan their workday because of the predictability resulting from work processes • • • 6-2 Version 6.2. and customer and vendor interaction From a worker perspective.Guide to the CASQ CBOK communication (detail of the process) is normally commensurate with the skill level associated with the job. Table 6-1 Sample IT Processes and Their Outputs Examples of IT Processes Analyze Business Needs Conduct JAD Session Run Job Develop Strategic Plan Recognize Individual Performance Conduct Project Status Meeting Process Outputs Needs Statement JAD Notes Executed Job Strategic Plan Recognized Individual Updated status information 6.1. Table 6-1 shows some sample IT processes and their outputs. although the reasons differ. processes are needed to: Explain to workers how to perform work tasks Transfer knowledge from more experienced to less experienced workers Assure predictability of work activities so that approximately the same deliverables will be produced with the same resources each time the process is followed • Establish a basic set of work tasks that can be continuously improved • Provide a means for involving workers in improving quality.1 .

3 Process Workbench and Components A quality management approach is driven through processes. As Figure 6-1 shows. A process is written with the assumption that the process owners and other involved parties possess certain skill and knowledge levels (subject matter expertise). skills. or work teams.1 6-3 . documenting how a specific activity is to be performed. Workbenches are also called phases. or management makes a decision to release a nonstandard product. Figure 6-1 Components of a Process Workbench From the perspective of the PDCA cycle. Depending on the maturity of the organization. the workbench is a graphic illustration of a process. the process engages in rework until the output products meet the standards. QA analysts. Implement. process workbenches may be defined by process management (or standards) committees. Build. and improved during the Act segment. The workbench transforms the input to produce the output. steps. and Improve Work Processes 6. A process can be viewed as one or more workbenches. or tasks.2.the intent) Standards (what . People. The workbench is comprised of two procedures: Do and Check. If the Check procedure determines that the standards for the output product are not met.1. A process is defined by workbench and deliverable definitions. A workbench definition contains: • • A policy statement (why . and tools support the Do and Check procedures. which correspond to the Do and Check phases of the PDCA cycle. the process workbench is created during the Plan phase.Define.the rules) Version 6.

they will be complied with. and procedures may refer to people. and if the process contains task procedures.2. and necessary. • Policy The policy states why a process exists or its purpose. 6-4 Version 6.Guide to the CASQ CBOK • Procedures (one or more tasks) in the form of procedural statements (how) A deliverable definition contains: • • • • A policy statement (why – the intent) Standards (what – the rules) Templates that specify document format Policies. and should link to the organization’s strategic goals and support customer needs/requirements. they will be performed. attainable. methods. Standards may relate to a deliverable produced in the process or to the task procedures within the process. The following components of a process represent the vocabulary of process management. • Standards The standards state what must happen to meet the intent of the policy. Note that some workbenches and deliverables may not contain standards or procedures. standards may specify things such as the time frame or a sequence that must be followed. standards. A standard must be measurable. Regarding deliverables.1 . Regarding the task procedures. • Inputs Inputs are the entrance criteria or materials needed to perform the work. It is assumed that if a defined process contains standards. the standard is used to determine that the delivered product is what is needed. A policy indicates intentions or desirable attributes or outcomes of process performance. and tools.

skills. capture/playback testing tools.4 Process Categories Approaches for controlling businesses have evolved over many decades. Examples of these controls are given in Skill Category 7. or results produced by the process. People. a process.S. which are implemented through the processes below. Deliverables can be interim or external. and Improve Work Processes • Procedures Procedures describe how work must be done . tools. and e-mail may be used to aid in the execution of the procedures. Deliverables serve as both inputs to. Implement. and outputs from. • Output or Deliverables Output or deliverables are the exit criteria. are not considered separate components of the workbench. and associated skill sets needed to execute a process.1. Version 6. responsibilities. Procedures indicate the "best way" to meet standards. • 6. and. These approaches have been accepted by the American Institute of Certified Public Accountants. Government).Define. People or Skills are the roles (such as suppliers. including human resources. chartered accountant societies worldwide. but never passed on to another workbench. owners. The business control model includes three general categories of control. therefore. A model for this system is the Malcolm Baldrige National Quality Award model. planning. Build. • Management Processes These are the processes that govern how an organization conducts business.2. may be used by one or more workbenches. templates. code compilers. budgeting. There are procedures to Do and Check work. External deliverables.1 6-5 . and the U. General Accounting Office (which has issued control guidelines for the U. Interim methods. techniques. products. Management processes are referred to as the quality management system. directing. organizational controls. For example. checklists. such as a requirements specification. and have one or more customers. • Manual and automated tools such as CASE tools. and customers). a programmer may require written communication skills and knowledge of Visual Basic. Models are discussed in Skill Category 3.S. and processes governing responsibility and authority. and people are applied to perform a process (transform the input into the output). such as JAD notes. and tools are incorporated into the Do or Check procedures. are produced within the workbench.

and the Check processes represent the Check component. For example: • With management processes at Level 1. not punish the people assigned to the work processes. They are also the source of data needed for continuous process improvement. or how management will react to work situations. rather than relying on work processes. As processes mature. predictability and consistency enter the workplace. Management processes at SEI Level 1 are very unpredictable and have great variability. and there is more reliance on management by fact. leaving management free to perform their planning responsibilities rather than acting as inspectors for the • • • 6-6 Version 6. so do the methods and approaches for managing them. As the Check and Do processes mature. management processes perform the Plan and Act components. the same concepts apply to management processes. In contrast. While the Software Engineering Institute (SEI) Capability Maturity Model (see Skill Category 3) is directed at the software development process. and trust increases. Management processes should be stabilized and matured in conjunction with the work processes. but they exist in management processes as well. management does not know what workers are doing. The need for status reports and status meetings decreases. program reviews. a mature. Immature management processes are usually managed through controls such as budget and schedule. When organizations have immature management and work processes. Politics can influence unpredictable processes. organizational politics become extremely important in the management style. such as systems development. acquisition of software.1 . The maturing of the Check processes help mature the management and work processes. stable management process is less influenced by organizational politics. They also assure that management processes are performed according to organizational standards and needs. contracting. and testing. Responsibility for resolving a noncompliance may be enforced automatically through controls. workers need less supervision. Check processes are typically associated with work processes. As management and work processes mature. while those at Level 5 are highly predictable.2. In the context of the PDCA cycle (see Skill Category 1). The organization tends to flatten as layers of middle management are eliminated. and workers may not know what management wants. with minimal variability. This means that management must resolve noncompliance to processes. work processes represent the Do component. Check processes are a major source of quantitative data for the tactical dashboards that are used by management (see Skill Category 8). and change management.Guide to the CASQ CBOK • Work Processes These processes include the standards and procedures that govern the performance of a specific work activity or application. • Check Processes These controls assure that the work processes are performed in accordance with the product standards and the customer needs. Examples include document reviews.

many repetitive tasks are performed by computer operations to produce similar products such as invoices and checks. are interrelated as shown in Figure 6-2.5. At the high end of the continuum the product may also be a service in the form of professional advice or consulting.1 6-7 . Build.Define.5 The Process Maturity Continuum Work processes.2. such as a software system. A study of quality models shows that the major factor in maturing the work processes is the addition of check processes.1. check processes. Figure 6-2 Process Management Continuum As the type of product changes on the continuum. Implement. The primary change in work processes is the amount of worker skill and personal contribution to the product or service produced. and dictates the level of customer involvement. and customer involvement. and Improve Work Processes work products and services. Version 6. 6. The type of product determines the work and Check processes.1 Product and Services Continuum The IT industry deals with a continuum of products. For example.1. so do the work processes. 6. ranging from similar to professional. In the middle of the product continuum are job shops that produce one-of-a-kind products with the same characteristics but are customized and unique.

Intent controls used in design processes would focus on such things as whether the design: • • • Uses the hardware configuration effectively Can be implemented given the skills of the implementers Can be easily maintained 6.Guide to the CASQ CBOK 6. the customer usually becomes more involved on an iterative or checkpoint basis. judge the quality of the completed product. There are normally few standards at this level. or negative. In the system design example above.5. The worker knows the standards that the product or service must meet. and may be referred to as crafts or art. the user becomes still more involved to verify that the products being produced are those wanted by the customer. with the promise of a reward. Or. 6. and must try to hire the best people and then encourage them to perform the job tasks needed. and the worker’s manager or customer. and to the customer's satisfaction. the probability of success is significantly increased at this end of the continuum because previous use of the process has proven it to work.5.2 Work Process Continuum Continuous work processes are process-dependent.3 Check Processes Continuum The continuum of check processes parallels that of work process methodologies. and is provided with the procedures and tools needed to complete the job on time and within budget.1. there is minimal or no customer involvement. For similar products produced by continuous work processes using literal controls. For example. and require a group of peers to assess their effectiveness. While individuals use experience and personal skills.1.1. This is sometimes called "inspirational management. It is important for the worker to follow the processes precisely so that each product produced is similar to the previous product produced. management depends upon the creativity and motivation of people. and represents the types of controls associated with the work processes. The process assumes that the designer has certain skills and creativity and utilizes them to create the design.2. the customer would rarely be involved unless problems occurred. For example. Professional work processes are people-dependent. As an IT function moves towards customized products and services.4 Customer Involvement Continuum The customer involvement continuum shows the level of involvement needed by the customer for the type of product produced. the process focuses on the way the design is documented and the constraints rather than the actual design. They depend as much on the skills of the worker as on the steps of the process. Controls of professional processes focus more on intent.1 . Continuous work processes use literal controls. the objective of placing controls on a worker’s skills and creativity is to enhance the probability that the skills and creativity will be used effectively. threatening to withhold rewards if the job task is not completed on time.5. When the products become 6-8 Version 6. With professional work processes." The inspiration can be positive. in daily computer operations. At certain points during the work process. within budget. in creating a design. a software developer might have C++ programming skills.

Implement. an order entry check might have a computer screen of the fields needed Version 6.1 6-9 . Process management is primarily a line (not a staff) responsibility. Other approaches have been used. but not to manage the processes.6 How Processes Are Managed The infrastructure in a quality management environment supports process management.Audit for process deployment and compliance.7 Process Template A process template is a pictorial representation of what is needed to comply with the process requirements. If the advice is ignored. Build. Occasional failure is the price of improvemen. when an organization is not able to separate the auditing and quality functions (separate functions are recommended) Occasional failure is the price of improvement. Their primary role should be to help and support the line organization. and give teams responsibility for the activities of building and improving processes.Define. and is shown in Figure 6-1 in Skill Category 2. etc. Generic purchased processes are not customized for the culture in which they are installed.2. The quality function may: • • • • Participate on committees.1. or used incorrectly. Either they fail completely. • 6. providing process management expertise Provide team support.1. but not as successfully. For example. the quality function also has some process management responsibilities. As a supporting staff function. Thus. coaching. or they feel that they know better ways to do work than those defined by someone else. and publishing and distributing process definitions.formatting. and facilitation Serve as a centralized resource for measurement analysis and reporting Play a "custodial" role for processes . the end product will likely be ineffective. heavy involvement is not only needed. such as training. and Improve Work Processes professional. the work process involves how the customer uses that advice. The most effective means for managing and building work processes is to have managerial responsibilities fall to a process management committee. or are only partially followed. As discussed in Skill Category 4. the involvement of the quality function varies with the maturity level of the quality management system. controlling access or change to process definitions. but the involvement actually shapes the final product or service. This frequently fails because the users do not feel ownership of the process. Some organizations engage a single individual or small group to write processes for the entire organization. the customer is heavily involved in the work or check processes. In consulting. editing. All levels of the organization should be involved in both establishing and using processes in their daily work. 6.

Process Planning sets priorities for process management projects (defining or improving processes). 6.2 Process Management Processes Process management is a PDCA cycle. and its deliverables (products and services). Process Inventory defines a list of processes that support an organization in accomplishing its goals.1 .2. Do Cycle The Do cycle is used by the Process Development Team and includes these processes: 6-10 Version 6. its functions (people). The computer screen represents the template used to accomplish the order entry process. 2. Process management processes provide the framework from within which an organization can implement process management on a daily basis. together with the infrastructure group that uses that process. Figure 6-3 shows how this set of practices can be viewed as a continuous improvement cycle. 3. • Plan Cycle The Plan cycle is used by the Process Management Committee and includes these processes: 1. Process Mapping identifies relationships between processes and the organization's mission/goals. Figure 6-3 Process for Process Management The process management PDCA cycle includes seven processes. These processes are summarized and then discussed in more detail below.Guide to the CASQ CBOK to complete the customer order.

standards. and second. Planning should occur before processes are defined to ensure that the most critical processes are defined first. 6. 6. and the cycle continues. to determine where the process could be improved. Build. people and skill requirements. Before producing an inventory. the scope of the effort must be defined. and acted upon. and incorporates tactical measurement into the appropriate processes. task procedures. This enables the process improvement process to focus on those process components that will provide the greatest benefit to the organization when improved. and tools.1 Planning Processes Figure 6-3 showed the following three processes within the Plan cycle of the process management process. Process Controls identifies the level and types of quality controls needed within a process. and Improve Work Processes 4. The Check and Act components cause new planning to occur. Implemented processes should then be measured to determine first whether they are repeatable (approximately the same product set is produced each time the process is used). The seven process management processes should be used in the sequence in which they are described. The process management PDCA cycle is continuously performed. checked.Define. but is also updated and improved on an ongoing basis. such as client/server or the Internet. Process Measurement determines what measures and metrics are needed to strategically and tactically manage by fact.2. Implement.1 6-11 .2.1 Process Inventory A process inventory is a major process management deliverable containing the "master list" of processes that support an organization in accomplishing its goals. 5. • Check Cycle The Check cycle is used by the Process Management Committee and includes this process: 6. The plan then redefines the sequence in which processes should be defined. prevent problems. deliverables. The inventory is developed as part of an overall process management framework. Process Improvement uses facts (measurement results) to identify root causes of problems and to change the processes in order to improve results.2. focusing on processes owned and used by the organization. and reduce variation. • Act Cycle The Act cycle is used by the Process Development Team and includes this process: 7.1. Process Definition defines a process's policies. and incorporates QC procedures into the process. as will the introduction of different technology and approaches. Inventories can be developed by: Version 6.

Sample processes include: • • • • • Develop Strategic Plan Purchase Tools Perform Internal Assessment Conduct Market Research Identify Product Requirements • 6. identifying an organization's mission and goals is difficult. The three main objectives of process mapping are to understand how a process contributes to meeting the organization's mission and goals. its functional units or roles (people). list the goals.g. If all processes contribute to all goals.2. On the left side of the matrix. functional units or roles or deliverables. Initial.Guide to the CASQ CBOK Referencing existing policies. Managed. or Information Systems Process Architecture) and updating to reflect organizational structure and terminology The inventory should list processes that produce a major outcome or deliverable. Identify the linkages between the rows and columns. Optimized can be used. who is responsible for the process. Put an X in the intersection to acknowledge a linkage only if the stability or quality of the process could influence meeting goals. 4. Defined. 3. 2. To map processes. Create a matrix. 6-12 Version 6. executive management (Quality Council) must have identified the organization's mission.2 Process Mapping Process mapping identifies or "maps" relationships between processes and the organization's mission and goals. Repeatable.2. Each process listed should contain a brief description and its status or state (e.1. The resulting Manage-by-Process matrix is a process map.1 . Processes should be mapped to mission and goals. to functional units (people). procedures. such as “run jobs” or “manage facilities”. The following generic process can be used for each mapping: 1.). and deliverables.. the CMM levels of Undefined. and its deliverables (products and services). organizational structure. and system development life cycle manuals • Conducting brainstorming and affinity grouping sessions (see Skill Category 4) • Surveying and interviewing employees • Starting with existing process inventories (such as other companies' inventories. as follows: • A process may support multiple goals. Optionally. List processes across the top of the matrix. and how the process interfaces to produce the organization's outcomes. long and short-term goals. decompose the mission and goals further. If formal strategic planning is not regularly performed. a high-level “class” could be included to categorize processes. standards. and to deliverables in separate matrices.

look for gaps where deliverables do not have processes or vice versa.2. Priorities are set based on the relative importance of the process to accomplishing the organization's mission and goals. For mission and goal mapping. A score of 1-3 represents the readiness of the organization. “R”. • Version 6. or the service is provided through the process.3 Process Planning Process planning allows priorities to be set for process management projects (defining or improving processes). In this matrix indicate the usage of the deliverable by placing a “C”.1 6-13 . Build. • 5. Implement. “D” means the deliverable is deleted. time and money) allocated to define or improve the process? • Status of each process This was determined as part of the inventory process. “S” or “C” in the intersection to distinguish between the roles. “U” indicates the deliverable is updated or revised in the process. and is there subject matter expertise in the methods and tools used to define or improve the process? • Resources: Are appropriate and adequate resources (people.Define. as they do not add value. look for gaps where goals are not supported by processes. add them to the inventory. For deliverable mapping.2. and Improve Work Processes • Processes have primary owners. Deliverables can be interim such as design specifications. look for gaps where units do not have processes. • Assessing organizational capability or readiness The degree to which an organization is capable of defining or improving each process should be assessed.1. “C” is used when the deliverable is created. Readiness is influenced by three main factors: Motivation: Are most of the process owners committed to managing by process and motivated to define or improve it? • Skills: Is the process understood. Identify a linkage in this matrix by using “O”. This involves weighting goals by importance and then applying the weighting scale to the one to three processes that most strongly align to each goal. Consider removing any processes that do not support goals. or retired. or external such as user manuals. suppliers and customers. etc. organizational constraints or readiness. and an assessment of the project’s status as follows: • Assessing mission alignment The degree to which each process contributes to the organization's mission and helps to accomplish organizational goals should be ranked. 6. “R” indicates the deliverable is referenced or used as input to the process. “U” and/or “D” in the intersection. For functional unit mapping. If the mapping identified any new processes. 6.

While each process definition or improvement team should develop its own work plan.2. develop the tactical plan by assigning resources and time frames to the highest-priority projects. and process status. The time box approach helps organizations use an incremental. which specifies a preset period of time. for the project's duration. standards. supplier.2 Do Processes The two processes within the Do cycle of the process management process as shown in Figure 6-3 are discussed below. manager of the process owner. Only the core activity is discussed within the scope of this guide. During Process Definition the following activities occur: • Define the Scope of the Process Use the Process Inventory. people or skill requirements. readiness. A simple scheme is to assign the top-scoring process a Priority 1. customer.1 Process Definition The process that a team uses to scope a single process by defining its policies. Repeatable or Defined to 2. deliverables. Each company should develop a prioritization scheme that fits their own needs. a representative group should be used with the others acting as reviewers.2. piloting the process. Roles such as team leader. After establishing priorities. but other activities include performing walkthroughs of the process before publication.Guide to the CASQ CBOK After assessing the alignment. marketing the process. The core team should contain 3-5 members (typically process owners). and tools is called Process Definition. and existing standards and procedures to clarify the scope of the process.2. and Managed or Optimized to 1. The core activity of Process Definition is defining the process. and so on. the process management projects can be prioritized. iterative approach to process definition.2. 6. The scope of the work effort and the specific tactical plan is then driven by this time frame. When multiple people or units exist for the same role. such as six weeks. The alignment and readiness assessments are represented by a numerical value. and assign priorities based on the scores. Total the assessment values to provide an overall score. procedures. process administrator. and scribe should be assigned and the team should be trained in consensus building (see Skill Category 4). This plan should contain a project title. Team members should include a process owner.1 . 6. Many organizations use a "time box" approach to process management. Process status values can be converted to numbers by setting Undefined/Initial to 3. the tactical plan is a higher-level plan that the Quality Council and process management committee use to establish and track multiple process management projects. etc. the resources (manpower and materials) and time frame (start and stop dates) for the project. Process Maps. and a process auditor. facilitator. This involves developing a high-level process flow (major workbenches 6-14 Version 6.

prerequisites. and indicates desired results (desirable attributes of process performance or desirable product quality characteristics). and easily maintainable so that it can be used throughout application systems development and maintenance. with each workbench containing 3-7 process steps or task procedures. correct. stating a desire that the organization is currently capable of accomplishing. adding missing tasks or deliverables and correcting flaws in the workflow. process management committee and/or process manager should review it.Define. The standard is necessary if it is considered important or needed (not a "nice to have”) in order to meet the intent of the policy. Policies should link to the organization’s strategic goals. major inputs and outputs (deliverables). • Develop the Workflow Brainstorm the tasks and deliverables in the process and then group the tasks. Select the current best practices. and be complete. Workbench standards deal with performance issues related to time frames. Define the workbenches internal to the process. • Sample Workbench Standard Version 6. Typically processes contain 3-5 workbenches. Sample Deliverable Policy Statement The requirements specification must reflect the true needs of the ABC organization. testable. their sequence. and major customers and their requirements. It is measurable if it can be verified that the standard has or has not been met. • A process management committee or the process manager usually develops a policy before establishing a process development team. Build. and support customer needs or requirements. the standard can reasonably be complied with every time. Standards are more specific than policies in that they convert intentions into specific rules. and Improve Work Processes and interfaces with other processes). • Sample Workbench Policy Statement A JAD session is conducted to uncover the majority of customer requirements early and efficiently. and to ensure that all involved parties interpret these requirements consistently. attainable. • Develop Standards A standard states what must happen to meet the intent of the policy. or sequencing of tasks while deliverable standards typically specify content.2. Implement. they may need to challenge the feasibility and appropriateness of the policy. If the process development team develops the policy. A standard must be measurable. • Develop Policies A policy states why the workbench or deliverable exists. A policy should be realistic. A standard is attainable.1 6-15 . if given current resources and time frame. and major interim deliverables. and necessary. When the team begins scoping and defining the detailed process.

efficient. and indicate the "best current way" to meet standards. Process development teams should consider the following guidelines when developing standards: • • • Standards contain the basis on which compliance is determined. • Develop Procedures Procedures describe how work is done. and templates for deliverables. reviewed. quality control procedures must be able to evaluate whether the standard has been met.2. Different types of standards may result in different QC methods. • Policy statements may need to be re-evaluated in light of standards development.1 . For example: • Literal or binary: Each dataflow line on a dataflow diagram must contain a dataflow descriptor of five words or less that identifies the data being passed between processes and stores. • Sample Deliverable Standard Each unit of data or information referenced in the requirements specification must be described in the data dictionary. interview customers to find out how they use the deliverables. • 6-16 Version 6. Task procedures should refer to people. Each time a workbench is executed or a deliverable is produced. this is not always possible.Guide to the CASQ CBOK Requirements uncovered in each JAD session must be formalized. • Since customer requirements often determine standards. and the most important quality characteristics. if the procedures are followed. the standards will automatically be followed and the intent of the policy will be met. Ideally. and approved by the JAD participants the following morning. known techniques or methods. Procedures may have many tasks. It is better to embed these standards in the supplier’s process. accessible tools. not the methods by which compliance is achieved.the process owner’s desires for effective. however. such as training or user’s manuals. It is easier to control quality with binary standards that require no subjective assessments or judgments to determine compliance. tasks can also refer to more detailed work instructions. If appropriate. A task is a single step in a procedure. Judgment or intent: Each dataflow line on a dataflow diagram must contain a dataflow descriptor that concisely and accurately identifies the data being passed between processes and stores. • Consider "internal requirements" . and consistent processes. any problems they have. create standards that serve as entrance criteria. • If information from other processes is a problem.

Define. A Do procedure explains in a step-by-step format the tasks needed to produce a product. Address each open issue when its reference column contains the item being covered. Check procedures are described in the next section on process control. Check procedures are defined in the same way as Do procedures. each line (task) is identified in its proper sequence. and designs and incorporates them as Check procedures into the process. • Sample Do procedure for the requirements definition process 1) Scribe: Use the XYZ tool to enter the requirements.1 6-17 . Version 6. Controls should be designed based on the criticality of the process and what needs to be checked. Sample Do procedure to write a computer program might have the tasks 1) Project manager: get a job number. For example. Build. Check procedures should be incorporated to the task flow diagram or flowchart at the appropriate spot. paraphrasing each item. As standards and Do procedures are perfected. and the actor (role or function) responsible for “saying” each line is noted. • • • 6. should be minimized. 2) Leader: Walk through the requirements. the need for extensive quality control is reduced. A task flow diagram or flowchart may be used to graphically depict the procedure. Implement. 2) Programmer: obtain the program specifications. The procedures should not be a substitute for skill set development or describe how to do something the person doing the task should know (given the prerequisites). Like a script in a play. "Was a flowchart produced that describes the processing of the program?" or “How much quality control is enough?” One of the quality management philosophy objectives is to continually improve process capability by reducing variation and rework.3 Check Processes The Check process incorporates the process controls needed to be assured the “Do process” was performed correctly.2. and so forth.2. Skill set and knowledge (subject matter expertise) requirements or prerequisites for performing the procedure should be identified. and Improve Work Processes Procedures are often written in play script format (see Skill Category 4). a quality control checklist associated with programming would have questions such as. Process control identifies the appropriate types of quality controls needed within a process. There are Do procedures and Check procedures. With this strategy quality is built into the products rather than tested in. Check procedures describe how to evaluate whether the right work was done (output meets needs) and whether the work was done right (output meets standards). Generate a list of any open issues using the XX template. A process development team develops procedures using the following guidelines: Procedures are not always required and unless critical. If the play script format is used.

policies. One way to address this issue is to identify and rank process risks. Risks that are found to be outside the scope and control of the process being defined are best controlled in other processes. process risks for an "estimate project effort" process may be: • Use of inaccurate historical data • Misuse of historical data • Inaccurate estimation algorithm or mathematical mistakes • Inadequate contingencies used • Wrong staffing ratios and loading figures used The team reviews the process workflow. Skill Category 8 contains additional information on risk. the team should identify where defects potentially originate. For example. 6. which is covered in Skill Category 1.3. and to minimize the risk that defects will go undetected and "leak" into other processes. The severity of the risk will also influence the selection of the control methods.1 . The challenge is to install the appropriate amount and types of controls to minimize cost. Ranking risks should be based on two factors: the probability that the risk might occur and the impact (cost of defect and rework) if the risk does occur. Process risks are those things that could occur during execution of a process that would result in defects and rework. that should also be noted.Guide to the CASQ CBOK Quality control procedures are considered appraisal costs. low ranking scheme could be used. quality control is necessary in a process to catch defects before they affect the outcome of downstream processes. If there is any risk that standards may not be followed. In general. effort. and then brainstorms risks. and cycle time impact Availability of appropriate resources and people Strength of the control method 6-18 Version 6. Appraisal costs are part of the Cost of Quality. but this is not always the most appropriate location. Using the ranked list of risks. and procedures. the better. Control methods should be considered based on the following: • • • • Risk severity Cost. and increase the cycle time for building products and delivering services. the judgment and knowledge of the process owners on the team is usually accurate. Where standards and Do procedures are not perfected. and cycle time. effort. The control point decisions should be adjusted if needed. increase the effort required. The first step in process control design is to identify the most logical points in the process to add controls. the closer the control is to the point of origin of the defect. While this is somewhat subjective without historical defect and cost data. Plotting where the top-ranking risks lie can help to determine the most appropriate point to insert controls. A high.2. medium. They add to the overall cost.2. standards.1 Identify Control Points Controls are often placed near the end of a process or workbench.

standards compliance. Methods in this category are: • Analysis tools. Checklists often mirror policies. 6. Implement.3. and deliverable standards.2. Some tools. Checklists. complexity analyzers. Without Process Improvement Figure 6-4 Control Methods Categories 6. Build.2. automation is the only way to force compliance. writing analyzers.1.Define.1 Automatic When performing a Do procedure. and cross-reference tools). to crosscheck his/her work. automatically enforce task completion and sequence.3. and code analyzers (e.1..2. standards. and Improve Work Processes • Impact on the overall culture Figure 6-4 shows five main categories of control methods that are discussed below.1 6-19 . such as CASE tools. which are worksheets containing a list of questions oriented towards determining whether the standards have been adhered to. which parse and analyze deliverables after they have been created. They are designed to provide a summary-style self-check for the author. such as spelling checkers.2 Self-Checking This is when the process owner (author) uses a method other than the Do procedure. but address each compliance issue in the form • Version 6.g. and procedures.

checklists.1. which are informal reviews conducted between the author/ owner and one other person. which validate that the actual results are the expected or desired results. and sampling. Methods used by third parties include informal walkthroughs. To implement measurement in a process. and how to measure each.2. from informal to formal. 6-20 Version 6.1 .1. 6.3.Guide to the CASQ CBOK of a question. and may be ineffective because the supervisor is not sufficiently skilled or knowledgeable about the work to make intent or judgment calls regarding compliance. Examples of independent groups are quality functions and independent test teams.2. it identifies the relationship of process contributors and results.whether it is moving towards its results. Measurement provides quantitative feedback to an organization about whether it is achieving its goals . formal inspections. analysis tools. The objective is to review against specifications and standards. Tests. Methods such as informal walkthroughs.3.2. It may also influence the supervisor unfairly regarding a worker’s performance. Desk-checks. As with supervisory controls. The fourth phase discusses measurement by fact. checklists. Supervisors may use the informal walkthrough.3 Peer Reviews One or more process owners review the results of the author. where the author or owner reviews the product against specifications and standards.5 Third Party An independent group evaluates the product.2. The program starts by building a measurement base. A "yes" answer means that the author has followed the process and has produced a result that matches the intent of the policy statement. One-on-one reviews. and checkpoint reviews are discussed in Skill Category 7. quality problems are noted. and incorporates measurement into the appropriate processes. • • • 6.2 Process Measurement Process measurement determines what strategic and tactical measures and metrics are needed to manage by fact. testing. Typically. This is problematic because it takes responsibility for quality away from the worker. or testing for control methods. and the author corrects them.2. and then identifies goals for the desired business results.4 Supervisory The author’s supervisor reviews the work and ensures that defects found are corrected. Various methods exist.1. A "no" answer indicates noncompliance.3. It is uncontrolled and subject to the time requirements of each individual. this is problematic because responsibility for quality is taken from the worker and the third party may not have the skills or knowledge about the work to determine compliance.3. 6. 6.

The purpose of process improvement is to reduce the frequency of defects. consider which processes should incorporate the requirements to address the contributors. These requirements become policies and standards. One measure of process effectiveness is compliance. Process requirements can be derived from analyzing the factors that contribute to achieving desired results. If using the play script format. without process improvement.2. Build. The concept and practices used to test are covered in Skill Category 7.3. • • 6. 6. and Improve Work Processes Results should play a significant role in process definition. then a standard may have been developed limiting code complexity and size. Line management uses the data to control the process and the quality of the products produced by the process. These factors include identifying: The desirable attributes that must be present when processes are performed The desirable characteristics that must be included when products are produced in order to achieve desired results Next. This is because the same defects will likely occur every time that product is built. standards. While the measurement has been selected. if "maintainable code" is a desirable outcome for the "develop unit" process. and form the basis for tactical measurements. the QA analyst is a secondary recipient of IT data. how it will be collected in order to evaluate the process’ effectiveness over time may not have been considered.Define. as shown in Figure 6-5.3 Testing Testing is the process that determines whether or not the actual results of testing equal the expected results of processing. They are needed to ensure that processes are being followed and are also facts that must be analyzed when measuring and interpreting results. For example. analysis. including process ineffectiveness. there is a continuous cycle of uncovering product defects and removing them from the product. Other process measurements must be derived from the policies. and procedures themselves. and the procedures added to the task flow diagram or flowchart if those forms are used. Figure 6-5 shows how.4 Act Processes The Act cycle of the process management process includes the one process of process improvement.2.1 6-21 .2. If the process being defined will be a measurement collection. Implement. and uses it to improve the process itself. Measurements may already be used to control the process. Measurement data should be developed as part of executing processes and collected for use by the line management responsible for those processes. Normally. the tasks that describe how to do these things must be defined. Measurement procedures are defined as are Do and Check procedures. or reporting point. measurement procedures should be incorporated at the appropriate spot. A self-check QC method to analyze and report code complexity and size may have been incorporated into the process. Process improvement uses facts (measurement results) to identify the root causes of problems and Version 6.

iterative process. Every member of the organization should become involved in the process improvement process. then there is no need to search out. Figure 6-5 Concept of Process Improvement The long-range objective for process improvement is to eliminate the need for quality control activities such as testing. then an improvement program can be started to drive down those defects. It involves finding the defects. and correct product defects. accumulating the defects in a manner that identifies the significant from the insignificant. 6-22 Version 6.4. which may or may not include members of the process development team Providing the teams with a process to use for process improvement 6. selecting a single defect. Process improvement is a continuous. and variation will be reduced. and inspections. Not everyone must be on a team. At that point. The two methods below are recommended for creating the teams. although as many teams as practical should be organized.1 Process Improvement Teams Process improvement must be accomplished by emphasizing teamwork. For example. find. reviews. The PIT component addresses this need. and identifying the root cause of that defect.2. problems will be prevented.Guide to the CASQ CBOK to change processes so that results will improve.1 . The objective of the program would be to change the products or processes in order to remove the cause of the data entry defects. Process improvement has two components: • • Establishing process improvement teams (PIT). Then the process selects the next most significant defect and repeats the improvement process. if an organization identifies data entry errors as a high-defect activity. an action program is put into place to reduce the frequency of defects or eliminate the root cause of the defect.2. If the processes do not generate defects.

quickest. Team member responsibilities should include: Identifying problems and selecting the ones on which to work Proposing solutions to problems Choosing an appropriate solution and improvement approach Implementing the chosen improvement Documenting required data regarding team activities in the PIT measurement system • Ensuring consistent use of a common set of statistical process control (SPC) tools and techniques • Presenting the implemented improvement to a quality improvement administrator for certification that the PIT processes were followed Team members should allocate 30-45 minutes per week for their duties. Smaller or larger groups can lose effectiveness. and the members have established working relationships. Implement. so appropriate reward and recognition would occur for these shared team efforts.5 million in their second (250 teams. designing. and implementing improvements. plus individual time spent on PIT activities. this small amount of meeting time can be very productive.200 employees). and McCormack and Dodge saved over $2 million • • • • • Version 6. Utilizing 30-45 minutes per week.) Some teams might meet weekly and others might meet for an extended session every other week. and Improve Work Processes • Natural Work Group Natural work groups. the improvement process should support the collaboration of multiple teams working on problems with a broad scope. characterize the way companies currently organize their employees. it is only recommended as a method of organizing when the PITs are mature. Generally. Using natural work groups for a PIT is the easiest.25 million in annualized savings in their first year. the Paul Revere Insurance Company was able to save $3. and $7. each team should have between five and eight members. such as a system development project team. United Data Services (the IT function for United Telephone System) saved $4. it is important to limit the time so there is not a major impact on member’s daily responsibilities. Build. It is also an excellent way to address problems that affect more than one work group or department. If teams are trained in effective problem-solving and meeting-leading skills.1 6-23 . such as planning. these teams already exist. Because these types of problems are typically more complex. Process improvement and associated cost savings will soar. (This time commitment includes both meeting time. If natural work group teams elect to address problems that impact beyond their team. The measurement system would be required to track and report these instances. If everyone participates. They are also usually aware of improvements that could be made to improve the effectiveness of their work group.2. and least confusing method. Regardless of the method.Define.75 million annualized in the first year (60 teams). • Interdepartmental Teams This method of organizing teams across departmental boundaries promotes interdepartmental teamwork and contributes to breaking barriers (and problems) that exist within organizations. 1.

Analyze results 7. After the process has been selected and the improvement team formed. Leaders of improvement teams are usually process owners.2. Compare results 8. customers of. Organizational improvement team members are usually volunteers. The customer's quality characteristics are defined and the current state of satisfying these.1 . and cost of quality.1 Identify and Understand the Process 1. Select Process and Team The process to be improved may be selected by the improvement team or assigned by management.Guide to the CASQ CBOK in the first four months (150 teams). 2. The customer's requirements are defined using operational definitions to assure complete understanding between those providing the product or service and the customer. defects. if subject experts are required and none volunteer. In most instances. the process are determined. Plan how to test proposed improvement 6. The quantitative tools used in this step can include the process flow. Describe current process 3. management will assign them. determined. 1. Change process or redo steps 4-8 6. If not already on the team. These are impressive figures.4. Select process and team 2. the customer referred to is the internal customer.4. rework. and suppliers to. the eight-step process below is recommended. Pareto charts. customer and supplier representatives should be added when practical. and members are process users and may be cross-functional.2 Process Improvement Process For organizations that do not have a process improvement process (sometimes called quality or continuous improvement). This same idea is applied to the suppliers in the process. Brainstorm for improvement 5. The first three steps focus on process identification and understanding while steps 4 through 8 focus on the improvement aspect.2. Describe the Process Two simultaneous actions are initiated in this step: defining customer-supplier relationships and determining actual process flow. however. 6. Often the improvement team begins by analyzing data on customer complaints. The process owner's expectations are defined for suppliers 6-24 Version 6. Assess process for control and capability 4. and run charts. but achievable with minimal time commitments.2.2.

and Improve Work Processes of inputs to the process. Data that needs to be collected and analyzed can be identified in this step. Separate cause-andeffect diagrams are constructed for each effect. Next categorize the causes. A discussion of the ideal way to do the process is initiated. If the measurement is counting. cause-and-effect diagrams. and scatter diagrams.Define. or services Cycle time Timeliness Number of schedules missed Engineering changes per document Downtime because of maintenance. Scatter diagrams are then used to establish the relationship (correlation) between each of the major causes of variation and the process output. Examples of process output indicators are listed below: • • • • • • • • Amount of rework Yield or productivity Errors (defects) in products.2. reports. Assess the Process To assure that decisions are being made using precise and accurate data. Implement. parts shortage. 3. flowcharts. the team is also building a flowchart of the current process (how it is done at this point in time) if one was not completed in Step 1. the measurement system used to assess the process must be evaluated. Build. care should be taken to assure that everyone is counting the same things in the same manner. the input and output should be measured and baselines established. The customer's required quality characteristics are the effects. Pareto charts. While the customer-supplier relationships are being defined. do it the same? Is there variation between projects? The flowchart is used to identify "work around" sources of variation. Both input and output of the process are monitored. and other causes of poor quality or productivity. and recycle. The flowchart and discussions with the customer determine process measurement points. which monitor process health. The customer and supplier must then agree on the specifications to be met and how quality will be measured. rework. or other factors Version 6. if more than one does the process. Questions the team asks include: Does a person do the process the same every time? Does each person. Tools used in this step include measurement and data collection methods. Once it has been determined that the measurement system accurately describes the data of interest. A brainstorming session should be used to develop a cause-and-effect diagram identifying all possible causes of process input variation.1 6-25 .

Pareto charts. causeand-effect diagrams. run charts. The cause-and-effect diagram developed in the previous step should be reviewed and updated. If the control limits are outside the specification limits. If points are found outside the control limits.Guide to the CASQ CBOK • • Overtime Absenteeism or sick leave Process inputs. the process is capable of satisfying the customer. 6. Process inputs and outputs can be baselined using Pareto charts. the process is stable and. Brainstorm for Improvement The team should review all relevant information. the process is not capable and common cause variation must be reduced or the process retargeted. The tools available for this step are data collection methods. Another assessment of the process determines whether it is capable of meeting the customer's expectations (refer also to Skill Category 8).2. Both process location and variation may be a problem. in a state of statistical control. All statistical tools should be used in this step. such as the process flowchart. It may help to visualize the ideal process. More data is then gathered to confirm the elimination of the special causes. scatter diagrams. how. Once special causes of variation have been eliminated. determined in Step 2. and why of the improvement should be documented so everyone agrees to the plan. histograms. and control charts. Improvement ideas must be prioritized and a theory for improvement developed. measurement capability. run charts. Control mechanisms may need to be installed in higher-level processes to eliminate them in the future. who will collect the data.2 Improve the Process 4.4.1 . and how the data will be 6-26 Version 6. The improvement team may have to proceed to Steps 4 through 8 to find and eliminate the special causes of variation. A brainstorming session may be held to generate ideas for reducing input variation. 5. or both. Plan How to Test Proposed Improvement(s) In this step a plan for testing the proposed improvement developed in the preceding step is created. If the control limits are inside the specification limits. and control charts.2. how the data is to be collected. The control limits of the process are compared to the customer's specification limits. Next.2. represent possible sources of variation that must be quantified. process capability. therefore. This change in process location and reduction in variation is accomplished by using Steps 4 through 8 of the continuous improvement strategy. the process is assessed for statistical control by constructing a control chart(s) for each important process input. This process improvement plan specifies what data is to be collected. and advanced statistical techniques. The what. although brainstorming is used the most. control charts. these are due to special causes of variation and must be investigated (see Skill Category 4).

The specific statistical tools to be used for analysis are defined. Build. Should the team continue to improve this process by further reducing variation due to common causes or start working on another process that is not meeting customer requirements? The answer will probably be based on an economic analysis. and Improve Work Processes analyzed after collection. Change Process or Redo Steps 4-8 If the results are not as expected. a new method for improvement must be developed by reverting back to Step 4. it is not a failure . If the decision is made to improve a different process. or control charts are generally used in this step.1 6-27 . 6. Care is taken to document lessons learned and accomplishments. 7. histograms. measurement capability. Version 6. If it is economically desirable to continue to reduce the process variation. the change recommended in the process improvement plan is implemented. can they be explained? When results are not as expected. A set of before and after Pareto charts. Has the process been improved? Do the test results agree with scientific or engineering theory? If the results are not as expected. the team develops another improvement method and repeats Steps 4 through 8. Analyze Results The plan developed in the prior step is implemented.something is still learned about the process and its variation. and are practical and cost-effective. scatter diagrams. The process improvement plan is a formal document that is approved by the improvement team. 8. The new information will be used to develop a new theory. Compare Results This step uses the same statistical tools as in Step 3 to compare the test results with that predicted in Step 4. The implementation should be monitored to assure that the gains made in reducing variation are maintained. A decision is now required concerning the next improvement effort.Define. process capability. The forms should be tested before being put into use. If the results are as expected. and easy to understand. and then analyzed using the statistical tools noted in the process improvement plan. and advanced statistical techniques. run charts. The plan is reviewed at improvement team meetings to track progress on process improvement. flowcharts. the process begins again at Step 1 by defining a team.2. control charts. The tools usually used in this step are data collection methods. Attention is given to designing data collection forms that are simple. Implement. The team then returns to Step 2 to update the process documentation. histograms. The required data is collected using the previously tested forms. then based on what was learned. run charts. concise.

Guide to the CASQ CBOK This page intentionally left blank.2.1 . 6-28 Version 6.

The purpose of this section is to describe those basic testing concepts.2.1.1 7-1 . During development. product construction at the end of development/acquisition and throughout product change and operation. Process workbenches are discussed in Skill Category 6. The quality practitioner should also be familiar with verification and validation techniques. the framework for developing testing tactics. change control and configuration management. This skill category will address the various types of controls and when they are best used in the process. the quality control process is frequently called verification and at the conclusion of development. To understand the testing methodology.1 The Testers’ Workbench The testers’ workbench is the process used to verify and validate the system structurally and functionally. it is necessary to understand the workbench concept. product acquisition.Skill Category 7 Quality Control Practices Q uality control practices should occur during product development. Testing Concepts Verification and Validation Methods Software Change Control Defect Management page 7-1 page 7-12 page 7-18 page 7-20 7.1 Testing Concepts Many testers fail to do testing effectively and efficiently because they do not know the basic concepts of testing. 7. Version 6. it is called validation.

the product is sent back for rework. module.g. Upon completion. If the check process finds a problem.1 Integration Testing These tests are conducted on tasks that involve more than one application or database.2.1 . the product is sent back for rework. module. program code) to the tester. • Work is checked to ensure the product meets the specifications and standards. a procedure is followed.2. the validated system requirements result in a tested system based on the specification developed or purchased. If the check finds no problems. documentation.2 Test Stages There are four main testing stages in a structured software development process.Guide to the CASQ CBOK A testers’ workbench is one part of the software development life cycle. which is comprised of many workbenches. The programmer takes the following steps: • Give input products (e. or on related programs.2. the test results). 7. For example. They are: Unit Testing These tests demonstrate that a single program. and produce a product or interim deliverable (e.2 System Testing These tests simulate operation of the entire system and confirm that it runs correctly. modules. hardware. and a product or interim deliverable (a program.1.1.1. Tested units are ready for testing with other system components such as other software units. follow a procedure. and that the procedure was followed. observing the result when pressing a function key to complete an action. Two examples of the workbench concept are given below. 7-2 Version 6. • Check work to ensure test results meet test specifications and standards and that the test procedure was followed. Each integrated portion of the system is then ready for testing with other parts of the system. A project team uses the workbench to guide them through a unit test of computer code. If the check finds no problems. 7.. to validate that multiple parts of the system interact according to the system design. • The programmers’ workbench for one of the steps to build a system is: • Input (program specifications) is given to the producer (programmer). If the check finds a problem. release the product (test results) to the next workbench. or users. or units of code.. • Work (coding and debugging) is performed. • 7. or unit) is produced. the product is released to the next workbench. or unit of code function as designed. • Perform work (execute unit tests).g.

the oversight of user acceptance testing. successful development teams will have a peer perform unit testing on a program or module.3 Independent Testing The primary responsibility of individuals accountable for testing activities is to ensure that quality is measured accurately. The roles and reporting structure of test resources differ across. organizations. less beneficially. The existence of a Tester or someone in the organization devoted to test activities is a form of independence. Often.3 User Acceptance Testing This real-world test is the most important to the business. the test resources will have a reporting structure independent from the group designing or developing the application in order to assure that the quality of the application is given as much consideration as the project budget and timeline. Key Testers may also join the team at this stage on large projects to assist with test planning activities. they are usually responsible for system testing. Additional Testers.1. Other testers join later to assist with the creation of test cases and scripts. the same benefits can be achieved by having an independent person plan and coordinate the integration testing. or. The Test Manager should join the team by the beginning of the requirements definition stage. or other users interact with the system to ensure that it will function as desired regardless of the system requirements.1 7-3 .Quality Control Practices 7. Where an independent test team exists. Ideally. Internal staff. Key Testers. 7. in the loosest definition. that it is documented. The manager is also responsible for: • • • • • Planning and estimating tests Designing the test strategy Ensuring tests are created and executed in a timely and productive manner Reviewing analysis and design artifacts Chairing the test readiness review Version 6. including users who will participate in test execution. knowing that quality is being measured is enough to cause improvements in the applications being developed. and it cannot be conducted in isolation.2. The Test Manager ensures that testing is performed.1. they may be Testers who report to the project manager. and providing an unbiased assessment of the quality of an application. An independent test team is usually comprised of a Test Manager or team leader. and additional Testers. The result is a tested system based on user needs. and that testing techniques are established and developed. These resources may be business or systems analysts assigned to perform testing activities. Often. The team may also support or participate in other phases of testing as well as executing special test types such as performance and load testing. vendor. customers. Once a portion of the application is ready for integration testing. The benefits of independent testing can be seen even in the unit testing stage. usually join the test team right before system testing is scheduled to begin. and within.2.

Guide to the CASQ CBOK • Managing the test effort • Overseeing acceptance tests Testers are usually responsible for: • Developing test cases and procedures • Planning, capturing, and conditioning test data • Reviewing analysis and design artifacts • Executing tests • Utilizing automated test tools for regression testing • Preparing test documentation • Tracking and reporting defects Other Testers primarily focus on test execution, defect reporting, and regression testing. They may be junior members of the test team, users, marketing or product representatives, or others. The test team should be represented in all key requirements and design meetings, including: JAD or requirements definition sessions, risk analysis sessions, prototype review sessions, etc. They should also participate in all inspection or walkthrough reviews for requirements and design artifacts.

7.1.4 Static versus Dynamic Testing
Static testing is another name for in-process reviewing. It means that the test is being performed without executing the code. Static testing occurs throughout the development life cycle; however, a large part of it takes place during the requirements and design phases in the form of walk- through inspections, and system reviews. Other examples of static testing include code analyzers or writing analyzers. Dynamic testing (also known as program testing) implies that the code is being executed on a machine.

7.1.5 Verification versus Validation
Verification ensures that the system (software, hardware, documentation, and personnel) complies with an organization’s standards and processes, relying on review of non-executable methods. Validation physically ensures that the system operates according to plan by executing the system functions through a series of tests that can be observed and evaluated. Verification answers the question, “Did we build the right system?” while validation addresses, “Did we build the system right?” Keep in mind that verification and validation techniques can be applied to every element of the computerized system. You’ll find these techniques in publications dealing with the design and implementation of user manuals and training courses, as well as in industry publications.


Version 6.2.1

Quality Control Practices Computer System Verification and Validation Examples Verification requires several types of reviews, including requirements reviews, design reviews, code walkthroughs, code inspections, and test reviews. The system user should be involved in these reviews to find defects before they are built into the system. In the case of purchased systems, user input is needed to assure that the supplier makes the appropriate tests to eliminate defects. Table 71 shows examples of verification. The list is not exhaustive, but it does show who performs the task and what the deliverables are. For purchased systems, the term “developers” applies to the supplier’s development staff. Table 7-1 Computer System Verification Examples
Verification Example Requirements Reviews Performed By Developers, Users Explanation The study and discussion of the computer system requirements to ensure they meet stated user needs and are feasible. The study and discussion of the computer system design to ensure it will support the system requirements. Deliverable Reviewed statement of requirements Ready to be translated into system design System design Ready to be translated into computer programs Hardware configurations Documentation Training Code Walkthroughs Developers An informal analysis of the program source code to find defects and verify coding techniques. A formal analysis of the program source code to find defects as defined by meeting computer system design specifications. Usually performed by a team composed of developers and subject matter experts. Computer software ready for testing or more detailed inspections by the developer. Computer software ready for testing by the developer.

Design Reviews


Code Inspections


Validation is accomplished simply by executing a real-life function (if you wanted to check to see if your mechanic had fixed the starter on your car, you’d try to start the car). Examples of validation are shown in Table 7-2. As in the table above, the list is not exhaustive. Determining when to perform verification and validation relates to the development, acquisition, and maintenance of software. For software testing, this relationship is especially critical because: • The corrections will probably be made using the same process for developing the software. If the software was developed internally using a waterfall methodology, that methodology will probably be followed in making the corrections; on the other hand, if the software was purchased or contracted, the supplier will likely make the correction. You’ll need to prepare tests for either eventuality.

Version 6.2.1


Guide to the CASQ CBOK • Testers can probably use the same test plans and test data prepared for testing the original software. If testers prepared effective test plans and created extensive test data, those plans and test data can probably be used in the testing effort, thereby reducing the time and cost of testing.

Table 7-2 Computer System Validation Examples
Validation Example Unit Testing Performed By Developers Explanation The testing of a single program, module, or unit of code. Usually performed by the developer of the unit. Validates that the software performs as designed. The testing of related programs, modules, or units of code. Validates that multiple parts of the system interact according to the system design. The testing of an entire computer system. This kind of testing can include functional and structural testing, such as stress testing. Validates the system requirements. The testing of a computer system or parts of a computer system to make sure it will work in the system regardless of what the system requirements indicate. Deliverable Software unit ready for testing with other system components, such as other software units, hardware, documentation, or users. Portions of the system ready for testing with other portions of the system.

Integrated Testing


System Testing

Developers, Users

A tested computer system, based on what was specified to be developed or purchased.

User Acceptance Testing


A tested computer system, based on user needs.

7.1.6 The Life Cycle Testing Concept Example
Life cycle testing involves continuous testing of the system during the developmental process. At predetermined points, the results of the development process are inspected to determine the correctness of the implementation. These inspections identify defects at the earliest possible point. Life cycle testing cannot occur until a formalized SDLC has been incorporated. Life cycle testing is dependent upon the completion of predetermined deliverables at specified points in the developmental life cycle. If information services personnel have the discretion to determine the order in which deliverables are developed, the life cycle test process becomes ineffective. This is due to variability in the process, which normally increases cost. The life cycle testing concept can best be accomplished by the formation of a test team. The team is comprised of members of the project who may be both implementing and testing the system. When members of the team are testing the system, they must use a formal testing methodology to clearly


Version 6.2.1

Quality Control Practices distinguish the implementation mode from the test mode. They also must follow a structured methodology when approaching testing, the same as when approaching system development. Without a specific structured test methodology, the test team concept is ineffective because team members would follow the same methodology for testing as they used for developing the system. Experience shows people are blind to their own mistakes, so the effectiveness of the test team is dependent upon developing the system under one methodology and testing it under another. The life cycle testing concept is illustrated in Figure 7-1. This illustration shows that when the project starts, both the system development process and system test process begins. The team that is developing the system begins the systems development process and the team that is conducting the system test begins planning the system test process. Both teams start at the same point using the same information. The systems development team has the responsibility to define and document the requirements for developmental purposes. The test team will likewise use those same requirements, but for the purpose of testing the system. At appropriate points during the developmental process, the test team will test the developmental process in an attempt to uncover defects. The test team should use the structured testing techniques outlined in this guide as a basis of evaluating the system development process deliverables.

Figure 7-1 The “V” Concept of Software Testing During the system test process, an appropriate set of test transactions should be developed, to be completed at the same time as the completion of the application system. When the application meets the acceptance criteria, it can be integrated into the operating environment. During this

Version 6.2.1


Guide to the CASQ CBOK process, the systems development team and the systems test team work closely together to ensure that the application is properly integrated into the production environment. At that point, the teams again split to ensure the correctness of changes made during the maintenance phase. The maintenance team will make whatever changes and enhancements are necessary to the application system, and the test team will continue the test process to ensure that those enhancements are properly implemented and integrated into the production environment. In the V-testing concept, your project’s “Do” and “Check” procedures slowly converge from start to finish (see Figure 7-1), which indicates that as the “Do” team attempts to implement a solution, the “Check” team concurrently develops a process to minimize or eliminate the risk. If the two groups work closely together, the high level of risk at a project’s inception will decrease to an acceptable level by the project’s conclusion.

7.1.7 Stress versus Volume versus Performance
Many testers define stress and volume testing as testing the system constraints. A stricter definition is that stress testing tests the built-in constraints of the system, such as internal table size; and volume testing tests the system’s ability in an operating environment to process very large amounts of data. Performance testing tests the systems ability to meet performance standards, such as a maximum three-second response to a user’s request.

7.1.8 Test Objectives
A test objective (goal) is a statement of what the test team or tester is expected to accomplish during a specific testing activity. Test objectives, are usually defined during requirements analysis, and guide the development of test cases, test scripts, and test data. Test objectives enhance communication both within and outside the project team by defining the scope of the testing effort, and enabling the test manager and project manager to gauge testing progress and success. Each test objective should contain a statement of purpose and a high-level description of the expected results stated in measurable terms. Completion criteria for test objectives define the success measure for the tests. Test objectives can be easily derived using the system requirements documentation, the test strategy, results of the risk assessment, and the test team assignments. Test objectives are not simply a restatement of the system’s requirements, but the actual way in which the system will be tested in order to assure that the system objective has been met. If requirements are lacking or poorly written, then the test team must have a defined method for uncovering and defining test objectives. Techniques to consider include brainstorming, relating test objectives to the system outputs, developing use cases or relating test objectives to events or system inputs. The users and project team must prioritize the test objectives. Usually the highest priority is assigned to objectives related to high priority or high-risk requirements defined for the project. In cases where test time is cut short, test cases supporting the highest priority objectives would be executed first.


Version 6.2.1

The main goal is to identify defects within the stage or phase of the project where they originate.1.1 7-9 .2 Semiformal Review (or Walkthrough) This review is facilitated by the producer of the material being reviewed (e. or reviewing the objectives with the system users. when it takes place.2. and results are not formally reported. they are less expensive to correct. at the end of a phase.g. In addition to learning about a specific product or project.1 Informal Review This review is generally a one-on-one meeting between the producer of a work product and a peer or co-worker. Possible solutions for uncovered defects are typically not discussed during the review. and testing is often less than 30% efficient.1.1. Version 6. this is referred to as “stage containment. The participants are led through the material in one of two formats: the presentation is made without interruptions and comments are given at the end. and how it is conducted. rather than in later test stages. Reviews are performed during the development process.1 Review Formats There are three review formats as follows: 7. the advantage is obvious.9. reviews provide training in. or comments are made throughout. In addition. Reviews are an efficient method of educating a large number of people on a specific product or project in a relatively short period of time. and is initiated as a request for input regarding a particular artifact or problem. documentation or code).1. team members are exposed to a variety of approaches to technical issues (a cross-pollination effect). Semiformal reviews should occur multiple times during a phase for segments or “packages” of work.” As reviews are generally greater than 65% efficient in finding defects.Quality Control Practices As a final step. Semiformal reviews (see Review Formats below) are especially good for this. These reviews occur on an as needed basis throughout each phase of a project. This might entail using a checklist or worksheet to ensure that the process to set test objectives was followed. the issues raised are captured and published in a report distributed to the participants. The timing and the purpose of a review determine what type of review takes place.. and are often held for just that purpose. no preparation time. Another advantage of holding reviews is not readily measurable.1. the test team should perform quality control on this activity. Finally. 7. and at the end of the project. In either case. standards. and enforce the use of. There is no agenda. since defects identified in the review process are found earlier in the life cycle.9 Reviews and Inspections Reviews are conducted to utilize the variety of perspectives and talents brought together in a team. 7. as nonconformance to standards is considered a defect and reported as such.

Defects found are tracked through resolution.1 . vendors. etc. phase-end reviews are more general in nature.3 Checkpoint Reviews These are facilitated reviews held at predetermined points in the development process. A recorder assists the moderator by capturing issues and action items. Phase-end reviews are held at the end of each phase.1. if system performance was identified as a critical requirement.1. programmers.2. In contrast to the checkpoint reviews. For example.2 In-Process Reviews In-Process reviews are used to examine a product during a specific time period of its life cycle. with the goal of identifying defects as work progresses. analysts. Full participation by all members of the review team is required. designed. Regardless of the format. implemented. semiformal or formal review format. Defects found are tracked through resolution. a designated performance expert would look specifically at whether performance requirements were being met. and coding phases to ensure there were no performance issues before proceeding to the next phase.9. design. usually by way of the existing defect-tracking system.9. and material is distributed to participants before the review so they will be familiar with the topic and arrive prepared. These reviews may use an informal.3 Formal Review (or Inspection) This review is facilitated by a knowledgeable individual called a moderator. such as during the design activity. They are usually limited to a segment of a project. rather than at the close of a phase or even later. in a formal review format. and publishing them in a formal report with distribution to participants and management. Formal reviews may be held at any time.4 Phase-End Reviews Phase-end reviews (also called Decision-Point or Gate reviews) look at the product for the main purpose of determining whether to continue with planned activities. who is not the producer or a team member of the product under review. and tested. auditors. Checkpoint reviews focus on ensuring that critical success factors are being adequately addressed during system development. not the producer 2. therefore. 7. 7-10 Version 6.1. not corrected during the session 3. and could include customer representatives.1. 7. The product is reviewed. The meeting is planned in advance. The participants are subject matter experts on the specific factors to be reviewed against. The objective is to evaluate a system as it is being specified. and apply to both development and test products. when they are more costly to correct.9. All members of the review team are responsible for the results of the review 7.1. usually through a defect-tracking system. the quality of a formal review is directly dependent on the preparation of the participants.9. Defects and issues are identified. Although there may be more. three rules apply to all reviews: 1. three checkpoint reviews might be set up at the end of the requirements.Guide to the CASQ CBOK 7. Instead of walking team members through a general checklist (as would be done in a phase-end review). which focus on critical success factors.

9. These reviews focus on questions such as: “Is the quality what was expected?” “Did the process work?” “Would buying a tool have improved the process?” or “Would automation have sped up the process?” Post-implementation reviews are of value only if some use is made of the findings. The review determines the readiness of the application or project for system and acceptance testing.5 Post-Implementation Reviews Post-implementation reviews (also known as "postmortems") are conducted in a formal format up to six months after implementation is complete.1. each analysis or design “package” or segment of the application may be in a different phase of the project simultaneously. coding officially begins at the close of this review.2 Critical Design Review This review baselines the Detailed Design Specification (the “build to” document). 7. It is important to note that although the completion of a phase-end review signals the formal beginning of the next phase.1.9. Both methods require preparation and study by the team members. subsequent phases may have already been started. Project status.1.1. and scheduling and coordination by the team moderator. Both techniques are based on a reading of the product (e. The team includes the developer. in order to audit the process based on actual results. This procedure requires a team. The difference between inspection and walkthrough lies in the conduct of the meeting. Documentation Plan. or code) in a formal meeting environment with specific rules for evaluation.1 Software Requirements Review This review is aimed at verifying and approving the documented software requirements for the purpose of establishing a baseline and identifying analysis packages.g.1. Software Test Plan. usually directed by a moderator. They are held to assess the success of the overall process after release.4.9. requirements. In fact. 7..9. Careful analysis and planning are critical to ensure that the iterations are sequenced appropriately to minimize the risk of a defect found in one iteration causing excessive rework in previous iterations. Training Plan and Configuration Management Plan derived from the requirements are also verified and approved.1 7-11 .4.4. Normally. Version 6. Test cases are also reviewed and approved.Quality Control Practices the most common phase-end reviews are listed below.9. 7. The Development Plan. specifications. in iterative development methodologies. 7. but the remaining members and the moderator should not be directly involved in the development effort.6 Inspections Inspections are formal manual techniques that are a natural evolution of desk checking. risks. The quality assurance practitioner draws significant insight into the processes used and their behaviors.2. and non-technical issues are also reviewed. and to identify any opportunities for process improvement.3 Test Readiness Review This review is performed when the appropriate application components are near completion. 7.

Finally. At the problem definition stage. such as correctness techniques. such as the requirements review. The V&V plan is revised as necessary by updating tasks or modifying the scope and intensity of existing V&V tasks. of course. inspections can be used to determine if the requirements satisfy the testability and adequacy measures as applicable to this stage in the development. At key project milestones.2 Verification and Validation Methods Verification and validation represents both static testing (verification) and dynamic testing (validation). The developer finds many errors just by the simple act of reading aloud. Guidance for developing the test criteria can be found elsewhere. the software.Guide to the CASQ CBOK Inspection involves a step-by-step reading of the product. and plans the V&V tasks to address the change.1 . Each module should be analyzed separately and as integrated parts of the finished software. the V&V manager consolidates the V&V results to establish supporting evidence regarding whether to proceed to the next set of software development activities. Inspections should be performed at the preliminary and detailed design stages.1 Management of Verification and Validation Management of software development verification and validation (V&V) activities begins at the start of the project. and coordinates the results with the project team. Together they comprise the test activities. Each proposed change must also be assessed to determine whether any new hazards or risks are introduced in. Whenever necessary. The V&V manager assesses each proposed change to the system and software. or test readiness review. A reexamination of the problem definition and requirements may also be required. it must 7-12 Version 6. revises the Software V&V Plan as necessary based upon updated project schedules and development status. the inspection procedures should be performed on the code produced during the construction stage. design review. Design inspections will be performed for each module and module interface. 7. If formal requirements are developed. This activity continuously reviews the V&V effort. 7. These criteria include checks for historically common errors. are determined because of the discussion with team members and by applying the test criteria. identifies the software requirements that are affected by the change. Adequacy and testability of the module interfaces are very important.2. Any changes that result from these analyses will cause at least a partial repetition of the verification at both stages and between the stages. with each step checked against a predetermined list of criteria. may be applied to ensure adherence with the quality factors. Others. The methods available for verification and validation are briefly described. The developer is usually required to narrate the reading product. or eliminated from.2. and is performed for all software life cycle processes and activities. formal methods.

7. ready to be translated into software. Output from this review is a statement of requirements ready to be translated into system design.2 Requirements Reviews These reviews examine system requirements to ensure they are feasible and that they meet the stated needs of the user. process specifications. They also verify software relationships.2.2. When done. semi-structured reviews of the program source code against specifications and standards to find defects and verify coding techniques.2.2.2. Output from this review is a preliminary statement of high-level market requirements that becomes input to the requirements definition process (where the detailed technical requirements are produced).2. Design reviews yield a system design. The minimum tasks performed by V&V management include: • • • • • Create the Software V&V Plan Conduct Management Review of V&V Support Management and Technical Reviews Interface with Organizational and Supporting Processes Creation of V&V 7.2 Verification Techniques Verification is the process of confirming that interim deliverables have been developed according to their inputs. and standards. the structural limits of how much load (e. 7. 7.2. Verification techniques are listed below.3 Design Reviews These structural tests include study and discussion of the system design to ensure it will support the system requirements. documentation and training. 7..2.Quality Control Practices also be determined whether a V&V task needs to be repeated as a result of changes in the application or work products.2.2. for example.1 Feasibility Reviews Tests for this structural element verify the logic flow of a unit of software (e. hardware configurations. Version 6. verifying that the software could conceivably perform after the solution is implemented the way the developers expect)..4 Code Walkthroughs These are informal. transactions or number of concurrent users) a system can handle.g.g.1 7-13 . the computer software is ready for testing or more detailed code inspections by the developer.

Requirements must be traced throughout the rest of the software development life cycle to ensure they are delivered in the final product. highly structured session to review the program source code against clearly defined criteria (System Design Specifications. Validation ensures that the system operates according to plan by executing the system functions through a series of tests that can be observed and evaluated for compliance with expected results. Completion of the inspection results in computer software ready for testing by the developer.2. White-box testing consists of testing paths.1 White-Box White-box testing (logic driven) assumes that the path of logic in a unit or program is known. The level of traceability also enables project teams to track the status of each requirement throughout the development and test process. Multiple white-box testing techniques are listed below.2.5 Code Inspections or Structured Walkthroughs These test techniques use a formal.2. Table 7-3 illustrates how various techniques can be used throughout the standard test stages.2.2. as too many techniques can lead to an unmanageable number of test cases.2.3.Guide to the CASQ CBOK 7. product standards) to find defects. Table 7-3 Validation Techniques Used in Test Stages Techniques Test Stages Unit Test String/Integration Test System Test Acceptance Test White-box X X X X X X X X X Black-box Incremental Thread Regression X X X X 7. Within an IT environment.6 Requirements Tracing At each stage of the life cycle (beginning with requirements or stakeholder needs) this review is used to verify that inputs to that stage are correctly translated and represented in the resulting deliverables. 7. and test plans and code. Each technique is described below. branch by branch. the end product is typically executable code. 7. class and sequence diagrams. but should be limited.3 Validation Techniques Validation assures that the end product (system) meets requirements and expectations under defined operating conditions. This is accomplished by tracing the functional and non-functional requirements into analysis and design models. These techniques can be combined as appropriate for the application.1 .2. 7-14 Version 6. to produce predictable results. Statement Coverage Execute all statements at least once.

2.2 Black-Box In black-box testing (data or condition driven). For example. white-box or program-based testing produces a higher defect yield than the other dynamic techniques when planned and executed correctly.3. this technique determines whether combinations of inputs and operations produce expected results. Execute all possible combinations of condition outcomes in each decision. the initial conditions and input data are critical for black-box test cases.2.3. Equivalence partitioning is a technique for testing equivalence classes rather than undertaking exhaustive testing of each value of the larger class. Three successful techniques for managing the amount of input data required include: 7.1 7-15 .999 and $15. a program which edits credit limits within a given range (at least $10.Quality Control Practices Decision Coverage Condition Coverage Decision/Condition Coverage Execute each decision direction at least once.999 and $10.1 Equivalence Partitioning An equivalence class is a subset of data that represents a larger class.2 Boundary Analysis This technique consists of developing test cases and data that focus on the input and output boundaries of a given function.3 Error Guessing This is based on the theory that test cases can be developed from the intuition and experience of the tester.000 but less than $15. 2001. 7.2.000 and $15. Multiple Condition Coverage When evaluating the paybacks received from various test techniques.3.000 but not as great as $15.000 or greater (invalid) 7. Version 6.2.000 (valid) $15.2.000) would have three equivalence classes: • • • Less than $10. Specifically. As a result. Execute each decision with all possible outcomes at least once.2.2. For example.001) 7.000) Upper boundary plus or minus one ($14. Invoke each point of entry at least once.2. 2000 or February 29. the focus is on evaluating the function of a program or application against its currently approved specifications. In the credit limit example.3.000 (invalid) Equal to $10. treating all iterations as twoway conditions exercising the loop zero times and once.001) Boundaries ($10. boundary analysis would test the: • • • Low boundary plus or minus one ($9. in a test where one of the inputs is the date. a tester may try February 29.

This technique assures that the change will not cause adverse effects on parts of the application or system that were not supposed to change.3.3. There are pros and cons associated with each of these methods.2. reduction. demonstrates key functional capabilities by testing a string of units that accomplish a specific function in the application. Unit and bottom-up incrementally test the application server components. or repair has been made.1 . Modules are added in ascending hierarchical order. Bottom-up testing requires the development of driver modules.2 Bottom-Up This method of testing begins testing from the bottom of the hierarchy and works up to the top.2. An example of an effective strategy for a simple two-tier client/server application could include: 1. 7.3. which provide the test input. For example.Guide to the CASQ CBOK 7. 4.3.3. Thread test a valid business transaction through the integrated client. Unit and incrementally test the GUI or client components. 7. Modules are added in descending hierarchical order.1 Top-Down This method of testing begins testing from the top of the module hierarchy and works down to the bottom using interim stubs to simulate lower interfacing modules or programs. It involves adding unit-tested programs to a given module or component one by one. 3.3 Incremental Incremental testing is a disciplined method of testing the interfaces between unit-tested programs and between system components. and testing each resultant combination. Output from bottom-up testing is also often easier to examine. 7-16 Version 6. these techniques are extremely critical.2. which is often used during early integration testing.3. regression testing should be conducted during all stages of testing after a functional change.2. 7. call the module or program being tested. threading through the integrated components. improvement.3. Drivers tend to be less difficult to create than stubs.4 Thread This test technique. Test the network. and display test output. as it always comes from the module directly above the module under test. and network. server. There are two types of incremental testing: 7.2.2. although bottom-up testing is generally considered easier to use.5 Regression There are always risks associated with introducing change to an application. When testing client/server applications. 2. Thread testing and incremental testing are usually used together. units can undergo incremental testing until enough units are integrated and a single business function can be performed. and can serve multiple purposes. To reduce this risk.

The test manager’s objective is to maximize the benefits of the regression test while minimizing the time and effort required for executing the test. input validation.Quality Control Practices Regression testing can be a very expensive undertaking. and still assures that no new defects were introduced. and still helps assure the project team and users that no new defects were introduced. Structural testing includes path testing.3.2. and similar techniques. This is a significant timesaving over executing a full regression test. A regional regression test executes a subset of the full set of application test cases. Functional testing is considered black-box testing because no knowledge of the internal logic of the system is used to develop test cases. and performance testing are considered structural. and then executing a set of test cases for the receiving application to assure that it was not adversely affected by the changes. it is possible to use them to identify system components adjacent to the changed components. code coverage testing and analysis. both in terms of time and money. System testing.5. Functional testing addresses the overall behavior of the program by testing transaction flows. Unit testing. and define the appropriate set of test cases to be executed. regression testing. called the “downstream” application. This is the full set of test cases defined for the application.3 Full Regression Testing This retests the entire application after a change has been made. 7.2 Regional Regression Testing This retests modules connected to the program or component that have been changed. Both methods Version 6. 7.2. nested loop testing. and user acceptance testing are types of functional testing.2. The types of regression tests include: 7. A full regression test is usually executed when multiple changes have been made to critical components of the application.5.4 Structural and Functional Testing Structural testing is considered white-box testing because knowledge of the internal logic of the system is used to develop test cases. string or integration testing. If accurate system models or system documentation are available. and functional completeness. The test manager must choose which type of regression test minimizes the impact to the project schedule when changes are made. load testing. which involves passing data from the changed application to the downstream application.1 7-17 . testers perform structural and functional tests that can be applied to every element of a computerized system. stress testing. As part of verifying and validating the project team’s solution.1 Unit Regression Testing This retests a single program or component after a change has been made. a determination must be made whether regression testing should be conducted with the integrated application. 7.3.3. the developer should always execute unit regression testing when a change is made.5.2. Testers from both project teams cooperate to execute this integrated test.2. At a minimum. When an application feeds data to another application. logic testing.

2. • Disadvantages Its tests do not ensure that user requirements have been met. 7. such as accepting bar code input. To effectively test systems. Its tests may not mimic real-world situations. both methods are needed. Makes no system structure assumptions.4. Change control concerns should be identified so that proper control mechanisms can be established to deal with the concerns. The primary objective of configuration management (or change control) is to get the right change installed at the right time. which are listed below: 7.1 Software Configuration Management The dynamic nature of most business activities causes software or system changes. Both are described in this section. 7.3 Software Change Control Controlling software changes requires both a configuration management process and a change control process. Changes require well-formulated and well-documented procedures to prevent the manipulation of programs for unauthorized purposes.2 Functional Testing • Advantages Simulates actual system usage.4.2. Each method has its pros and cons. • Disadvantages Potential of missing logical errors in software.3. a functional test case might be taken from the documentation description of how to perform a certain function. 7.1 Structural Testing • Advantages The logic of the software’s structure can be tested. 7-18 Version 6. A structural test case might be taken from a technical documentation manual. Parts of the software will be tested which might have been forgotten if only functional testing was performed.1 . For example.2.Guide to the CASQ CBOK together validate the entire system. Possibility of redundant testing.

If new files have been established. not the operations group. as changes to either of these have the potential for impacting the project.3. the backup data required for recovery purposes may also have to be changed. In addition. commonly called the CCB or Configuration Control Board. the job control language associated with those programs and other documentation procedures involved in making the system operational after a problem occurs. it would greatly increase the difficulty of controlling versions and of maintaining up-to-date documentation. If this is a normal function. • The nature of the proposed change should be explained in writing. standards. Each time an application is changed. As a result. should have a unique version number. written proposals provide a history of changes in a particular system. as large libraries can negatively impact operations performance. Major changes should be approved by the systems-planning steering committee. Developers should make the program changes..2. then those people should be notified that a change has occurred. • • • • • • 7. and guidelines should also be identified with version numbers and kept under version control to ensure the project team is working with the latest. • Version 6. documentation. procedures. Modifying an application system may also require modifying the recovery procedures. If the operators were authorized to make minor changes. it may be overlooked. Project documentation such as requirements specifications. or if new operating procedures or priorities have been designed. Backup data includes the new program versions. etc. and formally approved by a responsible individual. Changes should be incorporated through new versions of the program. Any change should be supported by adequate systems documentation. There should be a process for moving versions in and out of production on prescribed dates. database. Documenting the proposed change clears up any initial misunderstandings that may arise when only verbal requests are made. approved documents. in the same manner as for new systems. design documents.Quality Control Practices Some key points regarding changes include: • Each release of software. test plans. Minor changes may only require the joint approval of the IT manager and senior personnel in the user department. Since this step occurs outside the normal change procedures. They should address when to add to the library and when prior versions should be deleted. people should be assigned to review output immediately following changes. Other environmental considerations to keep under version control are the operating system and hardware. Procedures should exist for maintaining the production and source libraries. Testing will not uncover all of the problems. Care should be taken to regularly review libraries for obsolete programs.1 7-19 . they must be incorporated into the recovery procedures.2 Change Control Procedures Several procedures are necessary to maintain control over program changes.

Defect information should be used to improve the process. • 7. and resources should be based on an assessment of the risk and the degree to which the expected impact of a risk can be reduced (see Skill Category 8).2. • • • • 7. To manage defects properly requires a process that prevents. As imperfect or flawed processes cause most defects.1 .2 Defect Reporting Recording the defects identified at each stage of the test process is an integral part of a successful life cycle testing approach. priorities. a defect is a deviation from specifications. the goals are to find the defect as quickly as possible and to minimize the impact of the defect.4. The defect management process.4 Defect Management A defect is a variance from expectations. should be risk driven. a defect is 7-20 Version 6. and improves processes to reduce future defect occurrences. like the entire software development process. The QA analyst should look for trends and perform a root-cause analysis to identify special and common cause problems. From the producer’s viewpoint.1 Defect Management Process The general principles of a Defect Management Process are as follows: • The primary goal is to prevent defects. the documentation system should be updated with all change sheets or change registers and printouts.Guide to the CASQ CBOK • Someone independent of the person who designed and made the change should be responsible for testing the final revised program. resolves. Finally. 7. discovers. Operations should accept only properly approved changes. The information captured is used in multiple ways throughout the project.e. From the Customer’s viewpoint. The results should be recorded on program change registers and sent to the IT manager for approval. Defect measurement should be integrated into the development process.. i. Where this is not possible or practical.4. It should not be done after the fact by people unrelated to the project or system. the capture and analysis of the information should be automated. or extra. As much as possible. wrong. tracks. The purpose of this activity is to create a complete record of the discrepancies identified during testing. Information on defects should be captured at the source as a natural by-product of doing the job. and forms the basis for quality measurement. processes may need to be altered to prevent defects. whether missing. A defect can be defined in one of two ways. strategies.

This tool could be as simple as a white board or a table created and maintained in a word processor. While the defect itself may not be a big deal.Quality Control Practices anything that causes customer dissatisfaction. Defects are recorded for four major purposes: • To ensure the defect is corrected • To report status of the application • To gather statistics used to develop defect expectations in future applications • To improve the software development process Most project teams use some type of tool to support the defect tracking process. Based on the research team findings. 7. In large projects. Even minor defects. risk.4. which determines the order in which defects should be fixed. represent an opportunity to learn how to improve the process and prevent potentially major failures. and graphing capabilities. 7. no different from critical defects. or one of the more robust tools available today on the market. etc. Seemingly unimportant defects are. etc. NASA emphasizes the point that any defect represents a weakness in the process. For example a “severity one” defect may be defined as one that causes data corruption.2. from a process perspective. The priority assigned to a defect is usually more subjective as it may be based on input from users regarding which defects are most important.4. the test team should assign the severity of a defect objectively. Severity levels should be defined at the start of the project so that they are consistently assigned and understood by the team. a system crash. Tools marketed for this purpose usually come with a number of customizable fields for tracking project specific data in addition to the basics. it may also be necessary to assign a priority to the defect. which should have caught the defect earlier Version 6. It is critical that defects identified at each stage of the life cycle be tracked to resolution. the fact that there was a defect is a big deal. security violations.1 7-21 .3 Severity versus Priority Based on predefined severity descriptions. this activity should include the following: • • Go back to the process that originated the defect to understand what caused the defect Go back to the verification and validation process. This foresight can help test teams avoid the common disagreements with development teams about the criticality of a defect.4 Using Defects for Process Improvement Using defects to improve processes is not done by many organizations today. resources available. It is only the developer’s good luck that prevents a defect from causing a major failure. whether in the requirements or not. therefore. but it offers one of the greatest areas of payback. They also provide advanced features such as standard and ad-hoc reporting. e-mail notification to developers or testers when a problem is assigned to them.

This human factor dimension alone.1 . can have a very large impact on the effectiveness of the review process. This aggressiveness should be mandatory on life-critical systems. NASA takes an additional step of asking the question: “If this defect could have gotten this far into the process before it was captured. 7-22 Version 6. these steps make everyone involved in these activities take them more seriously.2. what other defects may be present that have not been discovered?” Thus. not only is the process strengthened to prevent defects. according to some of the people the research team interviewed. it is strengthened to find defects which have been created but not yet discovered.Guide to the CASQ CBOK Not only can valuable insight be gained as to how to strengthen the review process.

1 8-1 . This skill category addresses measurement concepts. Version 6. Measurement can be used to gauge the status. effectiveness and efficiency of processes. Measurement data is most reliable when it is generated as a by-product of producing a product or service. and presented to management in a timely and easy-to-use manner. and as a tool for management to use in their decision-making processes. risk management. This section provides those basic measurement concepts.2. the ways measurement can be used and how to implement an effective measurement program.Skill Category 8 Metrics and Measurement A properly established measurement system is used to help achieve missions. one needs to know the basic concepts of measurement. the use of measurement in a software development environment. goals. customer satisfaction. variation. visions. product quality. The QA analyst must ensure that quantitative data is valued and reliable.1 Measurement Concepts To effectively measure. Measurement Concepts Measurement in Software Variation and Process Capability Risk Management Implementing a Measurement Program page 8-1 page 8-7 page 8-11 page 8-17 page 8-21 8. and objectives. process capability.

it is more important to know how effective a person is in 8-2 Version 6. even qualified observers may determine different values for a given measure. With subjective measurement. Standard units of measure are the base on which all measurement exists. Objective measurement is more reliable than subjective measurement. Lines of code may mean LOC written. but as a general rule.1. such as how easy a system is to use. Measurement cannot be used effectively until the standard units of measure have been defined. Additionally. Measurement programs typically have between five and fifty standard units. The reliability of subjective measurement can be improved through the use of guidelines. feelings and opinions. etc. or the skill level needed to execute the system. An objective measurement should result in identical values for a given measure. For example. It is a person's perception of a product or activity. Since metrics are combinations of measures they can add more value in understanding or evaluating a process than plain measures. the more valuable it is.1 Standard Units of Measure A measure is a single quantitative attribute of an entity. for example. 8. or non-compound LOC written. organizations may use weighting factors.1. For example. Examples of measures are lines of code (LOC). talking about lines of code does not make sense until the measure LOC has been defined. subjective measurement is considered more important.1 . one verb would be weighted as more complete than other verbs in the same programming language. For example.2 Metrics A metric is a derived (calculated or composite) unit of measurement that cannot be directly observed. work effort. since their subjective judgment is involved in arriving at the measured value. or months. The more difficult something is to measure. the measure LOC refers to the “number” of lines and work effort refers to the “number” of hours.3 Objective and Subjective Measurement Objective measurement uses hard data that can be obtained by counting. stacking. 8. or completed deliverables. Examples include number of defects. Examples of metrics are mean time to failure and actual effort compared to estimated effort. when measured by two or more qualified observers. timing. It is the basic building block for a measurement program. If a line of code contained a compound statement (such as a nested IF statement two levels deep) it could be counted as one or two lines of code. measures should be expressed in numbers. days. or number of defects. executable LOC written. hours worked. Since quantitative measures can be compared. Subjective data is normally observed or perceived.1. but is created by combining or relating two or more measures.2. A metric normalizes data so that comparison is possible.Guide to the CASQ CBOK 8. weighing. and includes personal attitudes. which define the characteristics that make the measurement result one value or another.

than knowing they got to work on time (objective measurement).2. it is subjective. Data Type Nominal Ordinal Interval Ratio Possible Operations = ≠ < > + / Description of Data Categories Rankings Differences Absolute Zero 8. For example. Version 6. depending on the rules for classification. the type of information involved must be considered. Statisticians recognize four types of measured data.1. but differences or ratios between values are not meaningful.4.4. programmer experience level may be measured as low.1 Nominal Data This data can be categorized.2 Ordinal Data This data can be ranked.1. Nominal data can be objective or subjective. For ordinal data to be used in an objective measurement the criteria for placement in the various categories must be well defined. medium. The classification of software as user-friendly is a subjective product measure. Usually the data is used in a process model. should obtain the same measure value for a given program.1 8-3 . The reliability of the measure could be improved by providing customers with a guideline that describes how having or not having a particular attribute affects the scale. or high. Nominal data cannot be subjected to arithmetic operations of any type. operating system. Following are a few other examples of objective and subjective measures: • The size of a software program measured in LOC is an objective product measure. used in other calculations. For a scale of 1-5. customers of the software would likely rate the product differently. a program can be classified as database software.1. or is subjected to statistical analyses. Level of programmer experience is a subjective process measure.Metrics and Measurement performing a job (subjective measurement). Development time is an objective process measure. etc." The only possible operation is to determine whether something is the same type as something else.4 Types of Measurement Data Before measurement data is collected and used. 8. It should be collected for a specific purpose. Any informed person. otherwise. and the values cannot be ranked in any "natural order. For example. • • • 8. which are summarized in Table 8-1 and described below. Table 8-1 Types of Measured Data Operations for a data type also apply to all data types appearing below it. working from the same definition of LOC.

and mode. 3.1. If the values are to be used in mathematical equations designed to represent a model of the software process.4. 5.1 . being relatively insensitive to intuitively small changes in the process or product 8-4 Version 6. 8. not just describing them. ordinal. J.2. This is facilitated by measures and resulting metrics that are: • • • • • Simple and precisely definable.Guide to the CASQ CBOK 8. Many proposed measurements use values from an interval. a program with a complexity value of 6 is four units more complex than a program with a complexity of 2. 2. Interval data has no absolute zero. 8. The median is “3” because there are three values less and three values higher than 3.5 Measures of Central Tendency The measures of central tendency are the mean. and ratios of values are not necessarily meaningful.1. 2. A program of 2.6 Attributes of Good Measurement Ideally models should be developed that are capable of predicting process or product parameters. so it is clear how they can be evaluated Objective Easily obtainable at reasonable cost Valid. For example.4 Ratio Data This data has an absolute zero and meaningful ratios can be calculated.000 lines can be considered twice as large as a program of 1. It is important to understand the measurement scale associated with a given measure or metric. measurements associated with a ratio scale are preferred. T. but it is probably not meaningful to say that the first program is three times as complex as the second.3 Interval Data This data can be ranked and can exhibit meaningful differences between values. 8.000 lines. if a population of numbers are: 1. Measuring program size by LOC is an example. The mean is the average of the items in the population. The mode is “2” because that is the item with the most occurrences.1. McCabe’s complexity metric is an example of an interval scale. median. since the ratio scale allows mathematical operations to be meaningfully applied. the median is the item at which half the items in the population are below this item and half the items are above this item. For example. and the mode represents which items are repeated most frequently. and 11: • • • The mean is “4” because 1 + 2 + 2 + 3 + 4 + 5 + 11 = 28 and 28 ÷ 7 = 4. 4.4. or nominal scale. measuring what they are intended to measure Robust.1.

they provide an overall snapshot of the car's performance. and operating quality. Version 6. but overtime or time spent on the project by those outside the project team is not included.e. Indicators evaluated alone can lead to faulty conclusions and decision-making.6. A measure can be reliable.1.1. Measurement dashboards help ensure that all critical performance areas are analyzed in relation to other areas. but the use of an automated code analyzer would result in the same answer each time it is run against an unchanged program. 8.4 Timeliness This test refers to whether the information can be reported in sufficient time to impact the decisions needed to manage effectively.1. For example. but invalid.6. Arrayed together. If actual project work effort is intended to quantify the total time spent on a software development project.1. 8.7.1 Reliability This test refers to the consistency of measurement.6.3 Ease of Use and Simplicity These two tests are functions of how easy it is to capture and use the measurement data. Various key indicators are presented in comparison to their own desired target value (i.1 Measurement Dashboards Measurement dashboards (also called key indicators) are used to monitor progress and initiate change. Sometimes measures are unreliable because of the measurement technique.7 Using Quantitative Data to Manage an IT Function An integral part of an IT function is quantitative management.Metrics and Measurement Before being approved for use. speed. which contains two aspects: measurement dashboards and statistical process control. 8.6. 8. 8. An unreliable measure cannot be valid. health.1.2 Validity This test indicates the degree to which a measure actually measures what it was intended to measure.6.5 Calibration This test indicates the modification of a measurement so it becomes more valid. modifying a customer survey to better reflect the true opinions of the customer. for example.2. oil temperature). the same results should be obtained. 8. human error could make counting LOC unreliable. 8.1. A measurement dashboard is analogous to the dashboard on a car. If taken by two people.1 8-5 .1.. the measure is invalid for its intended purpose. measures and metrics should be subjected to the six tests described below.

The concept of “key” means the metric is considered important by the user in managing job responsibilities. Line managers use process or tactical dashboards to manage the process. vision.1. Using dashboards is known as “management by fact." Figure 8-1 depicts strategic and tactical dashboards. Normally the key indicator is created from many different measures. the Dow Jones Average is created from the combining of stock prices from approximately 40 different stocks. For example. 8.8 Key Indicators Key indicators are the metrics used by management to help them fulfill their management responsibilities. A dashboard is comprised of the total number of key indicators used by a single manager. Figure 8-1 Strategic and Tactical Measurement Dashboards 8.1 . and as a basis for continuous process improvement. 8-6 Version 6.2 Statistical Process Control Statistical process control is used to ensure that the process behaves in a consistent manner.1.7. or goals. Line managers use the principles of statistical process control to assess consistency of products and services.Guide to the CASQ CBOK Measurable results defined by an organization can be organized into dashboards for different levels of management. Senior management uses strategic dashboards to manage the function or organization and track to mission. Managers decide what key metrics they need.2.

with different levels of programming languages. These metrics and models are used to estimate and predict product costs and schedules. and to measure productivity and product quality. and cost per LOC suggest that assembly language programmers are more productive than high-level language programmers (higher LOC per month and lower $ per LOC). leading to improved results. complexity. have differences in underlying definitions and counting techniques making it almost impossible to compare quoted results. Again. Similarly. applications. Both the software product and the process by which it is developed can be measured. the productivity metrics LOC per month. Furthermore. it is not universally agreed what they mean. even though the total programming cost is usually lower for high-level languages. As for the proposed process models. especially with different environments. J. many measurements are done subjectively.1 8-7 . For example. Many software measures and metrics have been defined and tested but used only in limited environments. Generally. The software metrics are studied and developed for use in modeling the software development process. The above considerations make it difficult to interpret and compare quoted measurement results. using these measurements may obscure overall productivity and quality Version 6. Information gained from the measurements and models can be used in the management and control of the development process. such as LOC or T. Additionally. Metrics involving LOC values across different program languages can lead to incorrect conclusions and thereby conceal the real significance of the data. such as LOC. or development methodologies. such as product type or level of programming expertise. and statistical analysis of empirical data. There is no single process model that can be applied with a reasonable degree of success to a variety of environments.2 Measurement in Software The use of measurement in the software life cycle requires the development and use of software metrics.2. commonly accepted way of measuring software products and services. A large number of measures and metrics exist.Metrics and Measurement 8. expert judgment. few have a significant theoretical basis. Most are based upon a combination of intuition. The software product should be viewed as an abstract object that evolves from an initial statement of need to a finished software system. defects per LOC and cost per defect values have been used as quality or productivity indicators. which are standardized through the use of defined units of measure. including source and object code and the various forms of documentation produced during development. Even simple measures. Some studies have attempted to correlate the measurements with a number of software properties. including size. languages. Even with those widely studied. This measurement program enables management to control and manage software throughout its entire life. but only a few have had widespread use or acceptance. significant recalibration is required for each new environment in order to produce useful results. reliability (error rates). There is no clearly defined. They are difficult to evaluate because of the potentially large number of factors involved and the problems associated with assessing or quantifying individual factors. the various models often use a wide variety of basic parameter sets. McCabe's cyclomatic complexity. and maintainability.

productivity). inquiries (4). however. the size of the final program’s source or object code. 8-8 Version 6. Albrecht proposed a metric for software size and the effort required for development that can be determined early in the development process. Each FP contributor can be adjusted within a range of ±35% for a specific project complexity. Defect density and McCabe's complexity have been found to be reasonably good predictors of other characteristics. and master files. These were chosen because of their wide use or because they represent a particularly interesting point of view.Guide to the CASQ CBOK improvements by systematically yielding lower defect per LOC and cost per defect values for lower-level languages. Function points have the advantage of being measurable during the design phase of the development process or possibly earlier.2. inquiries. comment lines.1 Size LOC is the most common way of quantifying software size.1. this cannot be done until the coding process is complete. Despite these problems. LOC is a useful predictor of program complexity.1. Additional experience with current models and a better understanding of underlying measurements and their application to the software process will improve the results.2. The most common definition counts any line that is not a blank or a comment. and maintainability.1.1. Experience with measurement and models shows that measurement information available earlier in the development cycle can be of greater value in controlling the process and results.2. multiple statements per line.1 Product Measurement A product can be measured at any stage of its development. and programmer performance (debugging. multiple lines per statement. total effort. although there are many different definitions. Numerous studies have attempted to validate these relationships. non-executable statements. There are many useful products for measurement and modeling available on the market today. such as defect counts. outputs (5). or the number of pages of documentation produced for the installed system can be measured.1 Lines of Code This is probably the most widely used measure for program size. 8. outputs. the requirements.2. J. and master files (10). The following examples show various ways of measuring a product.1. The differences involve treatment of blank lines. total development effort. Thus. 8. regardless of the number of statements per line. even though total defects and costs are actually higher. the complexity of the software design. by totaling the number of external user inputs. Most of the initial work in measuring products has dealt with the characteristics of source code. For a software product. and the question of how to count reused lines of code.1 . a number of papers have dealt with the size or complexity of the software design. 8. 8. and then applying the following weights: inputs (4). This approach computes the total function points (FP) value for the project. applying software metrics and models in limited environments can help improve software quality and productivity. In theory.2 Function Points A.2.

performance. Since useful definitions are difficult to devise.1. • • • • Number of design changes Number of errors detected by code inspections Number of errors detected in program tests Number of code changes required Version 6.Metrics and Measurement 8. Two examples of complexity metrics are: 8. draw its control flow graph.2. as measured by defect counts. Skill Category 1 lists the commonly accepted quality attributes for an information system.2.2. 8.1. as computed from defect data. most efforts to find any single way to measure overall software quality have been less than successful. as measured by various other metrics. For example.1. as v(G)=e-n+2. The cyclomatic complexity of such a graph can be computed by a simple formula from graph theory. While software quality can theoretically be measured at every phase of the software development cycle. and reliability. such as correctness. or result.2.1 8-9 . A knot is defined as a necessary crossing of directional lines in the graph.1 Cyclomatic Complexity -. These alternative measures depend on both the program and the outcome. and software maintainability. portability.1 Correctness The number of defects in a software product should be readily derivable from the product itself. where e is the number of edges.2. and n is the number of nodes in the graph. since there is no easy and effective procedure to count the number of defects in the program. where each node corresponds to a block of sequential code and each edge corresponds to a branch or decision point in the program. 8.2 Knots Calculate program knots by drawing the program control flow graph with a node for every statement or block of sequential statements.3 Quality There is a long list of quality characteristics for software. Although much work has been done in this area.2. increased performance or speed of processing (desirable) may result in lowered efficiency (undesirable). 8. However. of some phase of the development cycle. including complexity metrics.v(G) Given any computer program.2.1. the characteristics often overlap and conflict with one another. Three areas that have received considerable attention are: program correctness. maintainability. software reliability. G.2 Complexity More metrics have been proposed for measuring program complexity than for any other program characteristic.2. there is still less direction or definition than for measuring software size or complexity. the four alternative measures listed below have been proposed. efficiency. The same phenomenon can be observed by drawing transfer-of-control lines from statement to statement in a program listing.3.1.

and v(G) to predict the psychological complexity of software maintenance tasks. A carefully detailed experiment indicated that software complexity metrics could effectively explain or predict the maintainability of software in a distributed computer system.3 Maintainability Efforts have been made to define ways to measure or predict the maintainability of a software product. However. Assuming such predictions could be made accurately. this data can be used to model and compute software reliability metrics. complexity metrics could then be profitably used to reduce the cost of software maintenance. service level agreements. investigated the ability of Halstead's effort metric. 8. or the rate at which software errors will occur.1. type of methodology used.4 Customer Perception of Product Quality Determining the customer's perception of quality involves measuring how the customer views the quality of the IT product.2.2 Reliability It would be useful to know the probability of a software failure.2. in addition to the ones listed above. such as overall development time.2. or the mean time to failure (MTTF) and mean time between failures (MTBF). An early study by Bill Curtis. although this information is inherent in the software product. Again. and the number of help desk calls per LOC can indicate the effectiveness of a system design methodology. actual costs Time spent fixing errors Wait time Number of contract modifications • 8-10 Version 6.1. it can only be estimated from data collected on software defects as a function of time.Guide to the CASQ CBOK 8. actual costs Budgeted costs vs. the number of failures per month can indicate the effectiveness of computer operations. • Accumulating product measures into a metric so that meaningful information about the process can be provided.3. loyalty. 8. These metrics attempt to indicate and predict the probability of failure during a particular time interval. and others.3.2 Process Measurement A process can be measured by either of the following: Attributes of the process.2. some others to consider include: • • • • • • Number of deliverables completed on time Estimated costs vs. and recommendations to others. E. such as using customer surveys.1. 8. It can be measured in a number of ways. If certain assumptions are made. function points per person-month or LOC per person-month can measure productivity (which is product per resources).1 . or the average level of experience of the development staff. There is no standardized list of software process metrics currently available.2. For example.

1 The Measurement Program A measurement program is defined as the entire set of activities that occur around quantitative data. and to differentiate between the two types.. 8. the same software development process may be followed to produce two different applications. measurement): 1.2. typically they need to be dealt with individually. W. and to prevent problems. proposals won Percentage of time spent performing value-added tasks 8. Management may need to adapt the process for Version 6. As a result.1 8-11 . Since the key to quality is process consistency. schedule. variation (the lack of consistency) must be understood before any process can be improved. The natural changes occurring in organizations are moving systems and processes toward increasing variation. their use of measurement changes with the maturation of the management approaches. and project status. Statistical tools are the only methods available to objectively quantify variation. and they are discussed in Skill Category 4. Quantitative measurement occurs at all levels of IT maturity. He was a strong proponent of the use of statistics that took into account common and special causes of variation.3 Variation and Process Capability Dr. management relies on the quantitative data produced from the processes to determine whether or not the requirements are complete.3.Metrics and Measurement • • Number of proposals submitted vs. It can be as simple as measuring whether a system is completed on time or completed within budget. each time a process is executed it normally produces a different product or service from what was built by the same process at another time. Immature organizations typically measure for budget. When work processes are optimized. In addition. It is generally not cost-effective or practical to deal with special causes in the day-today work processes. Edwards Deming's quality principles call for statistical evidence of quality. For example. it is important for the QA analyst to understand the difference between common and special causes. There are four major uses of quantitative data (i. A process is a series of tasks performed to produce deliverables or products. Special causes are those that must be controlled outside the process. Common causes are those that can be controlled by improving the work processes. As organizations mature. IT processes usually combine a skilled analyst with the tasks defined in the process. or it can be extensive and complex. and management relies on project teams to determine when requirements are done.e. Control charts are the tools used to monitor variation. Manage and control the process.

Manage the risks Risk is the opportunity for something to go wrong . 8-12 Version 6. Quantitative data gathered during process execution can identify process weaknesses.for example. Manage and control the product Quality is an attribute of a product. and. Improve the process The most effective method for improving quality and productivity is to improve the processes. know the potential consequences if the risk occurs. and understand the probability of success based upon different management actions. 3. newly purchased software will not work as stated. 2. therefore. and needs to know that when performed.2. and that the delivered product is what the customer expects and needs. or workers assigned to a project do not possess the skills needed to successfully complete it. know the probability of the risk occurring.Guide to the CASQ CBOK each product or service built.1 . Control requires assuring that the specified requirements are implemented. opportunities for improvement. Management needs to understand each risk. 4. The same database of quantitative data is employed for these four uses. the process will produce the desired product or service. but different measures and metrics may be utilized. Table 8-2 illustrates how the four uses of measurement can be achieved. Quality level must be controlled from the start of the process through the conclusion of the process. projects will be delivered late. Improved processes have a multiplier effect because everyone that uses the improved process gains from the improvement.

Mean time to failure .3. As a stable process is predictable. weeks) of activity completed . Processes containing only common causes of variation are considered stable.How much have we made? .Earned Value .Technical performance . In a computer operation.2. One researcher1 provides the following thoughts on common causes of variation: "Process inputs and conditions that regularly contribute to the variability of process outputs.1 Common Causes of Variation All processes contain some inherent variation.Mean time to repair .How cost-efficient is the process? . abnormal terminations cause variation.Time to complete Probability of exceeding constraints or not meeting requirements . and errors in operating or job control instructions.% of each activity completed Labor hours that differentiate requirements. or common causes of variation.Boxes . implementation and test Calendar times (months.1 8-13 .Metrics and Measurement Table 8-2 Achieving the Four Uses of Measurement Use Questions Answered Measurement Category Examples of Measures/Metrics Used . design.3.Units of output .2 Common and Special Causes of Variation 8.Measures specified by customers and management . The control chart in Skill Category 4 depicts a stable process. A process is defined as stable when its mean and standard deviation remain constant over time. The amount of variation in a process is quantified with summary statistics (the mean and standard deviation).What is the current performance? What are the risks? Performance Improve the Process Manage the Risks Time and effort Risks 8.” Version 6. A stable process is said to be in a state of statistical control.Unit costs .How much is left to make? Size Manage and Control the Process How much progress have we made? Status How much effort has been expended? When will the product be completed? Effort Schedule Manage and Control the Product How good is the product? Quality How effectively does the product perform? .Lines of code (LOC) .2. Typical common causes of abnormal terminations include invalid data.Procedures .Amount of scheduled work that is done .Number of defects found . no available disk space. future process values can be predicted within the control limits with a certain amount of belief.

” 1. Brian Joiner summarized special causes of variation as follows: "Process inputs and conditions that sporadically contribute to the variability of process outputs.2.” “Stratify and desegregate your observations to compare performance of sub-processes.3. special causes might include operator strikes. A state of statistical control is established when all special causes of variation have been eliminated (the operator strike ends.1 . and staff from various functions." From an address given at a Deming User's Group Conference in Cincinnati. They occur because of special or unique circumstances. If special causes of variation exist. or earthquakes.” “Investigate cause-and-effect relations.’ random-looking appearance. the process may be unpredictable. and therefore unstable." 8.” “Identify and rank categories of problems by Pareto analysis. OH. Joiner. 8-14 Version 6. In the IT example of abnormal terminations in a computer operation. "Stable and Unstable Processes.Guide to the CASQ CBOK “Common causes contribute to output variability because they themselves vary.2.” “Each common cause typically contributes a small portion to the total variation in process outputs. Brian. Illustrated in Skill Category 4 is a process that is unstable because it contains a special cause of variation in addition to the common causes.” “Because common causes are ‘regular contributors. Appropriate and Inappropriate Managerial Action. running experiments (one factor and multi-factor)." Joiner outlined this strategy for reducing common causes of variation: "Talk to lots of people including local employees.’ the ‘process’ or ‘system’ variability is defined in terms of them.2 Special Causes of Variation Special causes of variation are not present in a process. citywide power outages.” “Special causes contribute to output variability because they themselves vary. citywide power returns or business returns to normal operations after an earthquake). other managers.” “The aggregate variability due to common causes has a ‘non-systematic.” “Improve the measurement processes if measuring contributes too much to the observed variation.

2.' due to some specific circumstances. Or. Tampering is any adjustment made to a process in response to common 1. The concept of statistical control makes it possible to determine which problems are in a process due to common causes of variation and which are external to the process due to special causes of variation. W.” “Instead. It has been widely recognized that at least 85% of problems in any organization are system problems and the responsibility of management to solve. Cambridge. if results are good. Deming.” “Immediately search for the cause when the control chart gives a signal that a special cause has occurred. Improvements to address the common causes of variation usually require process or system changes." Joiner then presented this strategy for eliminating special causes of variation: "Work to get very timely data so that special causes are signaled quickly . Management working on the process is responsible for leading the effort to reduce common causes of variation.” “Do not make fundamental changes in the process. 1986.” “The variability due to one or more special causes can be identified by the use of control charts. the `process' or `system' variability is defined without them.Metrics and Measurement “Each special cause may contribute a `small' or `large' amount to the total variation in process outputs. Some sources quote 94%1. MA. and use statistical methods when making improvements to processes.” Because special causes are `sporadic contributors.3 Variation and Process Improvement Consistency in all processes from conception through delivery of a product or service is the cornerstone of quality. process variation due to common causes is expected and is not a reason for adjusting or changing the process. Employees using the process have the lead responsibility for reducing special causes of is bringing it back to its typical operation. seek ways to change some higher-level systems to prevent that special cause from recurring. Edwards.use early warning indicators throughout your operation. Version 6. MIT Press. Find out what was different on that occasion from other occasions. Managers must change the way they manage.3." 8. Bringing a process into a state of statistical control is not improving the process . By definition. retrain that lesson. One of the challenges in implementing quality management is to get process users thinking in terms of sources of variation. Out of the Crisis. Reducing variation due to common causes is process improvement and the real essence of continuous process improvement.1 8-15 .

or both. However. If the variation due to common causes results in a process operating outside of the customer specifications. yielding a process that is both stable and capable.2. in an effort to compensate for this variation . Deming defines tampering as "Action taken on a stable system in response to variation within statistical control. LCL and UCL represent lower and upper control limits. variation due to special causes must be removed to create a stable process. so a value could be within the control limits but outside the specification limits.the results of which will inevitably increase the variation and will increase cost from here on out. making the process non-capable. the process is called "noncapable.Guide to the CASQ CBOK cause variation. Figure 8-2 Making a Process Capable 8-16 Version 6. In the first picture.3. Figure 8-2 illustrates the transition of a process from non-capable to capable. a stable process may not be an acceptable process. retargeting the process. which have moved within the specification limits. the control limits are outside the specification limits. In the last picture the modified process results in different control limits.1 .4 Process Capability As previously stated. In this figure. 8." The process must be improved by reducing variation due to common causes." Management that does not understand variation continually asks for explanations or corrective action when confronted with variation due to common causes. LSL and USL represent lower and upper specification limits.

during the life cycle. replacing a team member. Cost such as sensitivity to technical risk.2 Time-Based Considering a software development life cycle. etc.4. overhead. Supportability or Environment such as people. etc. sensitivity to cost. Version 6. material availability. unproven technology. In contrast. each of which must be considered separately when determining how to manage the risk. equipment. and managing risk in order to eliminate or minimize any potential negative effect associated with risk. requirement changes.Metrics and Measurement 8.4. or changing a project's scope.2. 8. measuring.4 Risk Management Risk management involves the activities of defining. Schedule such as degree of concurrency. regulatory changes. 8. etc.the likelihood • The impact or consequence of the event if it occurs – the penalty Risks can be categorized as one of the following: Technical such as complexity.4. undergoing reorganization. the impact (cost) from a risky event occurring is low at the beginning (since not much time and effort have been invested) and higher at the end (as there is more to lose). etc. • The event that could occur – the risk • The probability that the event will occur. Examples include. Programmatic or Performance such as safety. 8. whereas at the end of the project the probability is very low. estimating errors.2 Characterizing Risk Risk has five distinguishing characteristics: 8. the probability of risk occurring at the beginning of the project is very high (due to the unknowns).2. number of critical path items.2.4.1 Defining Risk Risk is the possibility that an unfavorable event will occur. maintainability.1 8-17 . etc. reliability. prioritizing. Risk has three components. It may be predictable or unpredictable. skills.1 Situational Changes in a situation can result in new risks.

1 .4.this process answers the question "Which risks do we care about?" Risk Prioritization . consider the risk of spending $1 for a 50/50 chance to win $5. • • • Risk Identification Risk Quantification Risk Response Development • Risk Response Control This discussion of risk management addresses six processes. and user involvement. Value-Based Risk may be affected by personal.2. completing a project on schedule may be dependent on the time of year and nationalities or religious beliefs of the work team.000. and the magnitude of the risk typically makes a difference.4 Magnitude Dependent The relationship of probability and impact are not linear. the risk of spending $1.000 vs. The Project Management Institute's Project Management Body of Knowledge (PMBOK) defines the following four processes to address risk management. and respond to a risk. analyze. For example. If one deliverable takes longer to create than expected. many tasks and deliverables are intertwined. Risk Identification Risk Identification – this process answers the question "What are the risks?" Risk Quantification Risk Analysis . the probability of loss is all the same (50%) yet the opportunity cost of losing is much greater. analyzing.3 Interdependent Within a project.Guide to the CASQ CBOK 8. For example. 8. 8. Projects being developed in international locations where multiple cultures are involved may have a higher risk than those done in one location with a similar work force.4. other items depending on that deliverable may be affected.3 Managing Risk Risk management is the process used to identify.2. Identifying. and the result could be a domino effect.4. which have the following mapping to the PMBOK processes. corporate or cultural values. The PMBOK also notes that different application areas may use different names for these four processes.000 for a 50/50 chance of winning $5. the risk of spending $100. vs.000 for a 50/50 chance of winning $500. and prioritizing risks require knowledge of the business functions.2.this process answers the question "How are the risks prioritized?" Risk Response Development 8-18 Version 6. In this example.

and optimizing the organization's mix of projects in order to obtain a balance.4. documents the risk results and repeats the cycle of identification.this process answers the question "What should be done about the risk?" Risk Response Control Risk Resolution – this process executes the plan that was developed in the prior step.Metrics and Measurement Risk Response Planning .4 Software Risk Management Within many software development organizations risk management remains ad hoc and incomplete. 8.2. Risk Monitoring – this process evaluates the action taken.1 8-19 . or those perceived to be risky. they tend to be used only for large projects. quantification and response. Incorporating risk management into the software development life cycle includes planning at the following levels: • Long-Term or High-Level This level includes both long range planning. Where risk management processes exist. Version 6.

The customer.2. not added on to the end of the planning activities. smaller projects may require less effort. This may be done by line management and reviewed by the QA analyst.5 Risks of Integrating New Technology One of the major challenges facing an IT organization is to effectively integrate new technology.4. Assuring that the Controls are Adequate to Reduce the Risk The QA analyst needs to assess whether the controls proposed for the new technology are adequate to reduce the risk to an acceptable level. Although the QA analyst will probably not perform the actual task. The QA analyst has three roles in integrating new technology: • Determining the Risks Each new technology poses new risks. management. most risk management activity occurs close to milestones. Large projects should follow a formal risk management process. the QA analyst needs to ensure that a risk analysis for the new technology is undertaken and effectively performed. Risk management should be implemented as a series of integral tasks that are inserted into the normal project planning process. schedule and performance issues. and selection and project portfolio management Short-Term or Low-Level This level includes development. integration and testing strategies.1 . It is a part of a project manager's job that should be explicitly performed. Risks should not be the sole responsibility of the project manager. Assuring that Existing Processes are Appropriately Modified to Incorporate the Use of the New Technology • • 8-20 Version 6. • There are several components to consider when incorporating risk management into software project management: • In a traditional waterfall methodology. These risks need to be identified and prioritized and. development team. quantified. Ultimately the project manager must decide based on the project's cost. • • • 8. This integration needs to be done without compromising quality. In a spiral development model the risk management activity falls in the explicit risk management portion. if possible.Guide to the CASQ CBOK • Medium-Term or Medium-Level This level deals with project management strategies. either quantitatively or qualitatively. independent auditing process. project evaluation. Risk management is not a separate. and others should be involved with determining project risks.

Projects are often characterized by: • Schedule and cost estimates that are grossly inaccurate • Poor quality software • A productivity rate that is increasing more slowly than the demand for software • Customer dissatisfaction Addressing these problems requires more accurate schedule and cost estimates. This may be done by the workers responsible for the work processes. and what drives those results. and higher productivity that can be achieved through improved software management. 8.1 The Need for Measurement Current IT management is often ineffective because the IT function is extremely complex. The results are what management wants. objectives. and customer satisfaction..5 Implementing a Measurement Program The key to a good measurement program is knowing what results are wanted. Thus. Consistent measurements provide data for the following: • • • • • • • Expressing requirements. measure. and has few well-defined. 8. but at least needs to be assessed or reviewed by the QA analyst. Improvement of the management process depends upon improved ability to identify.5. cost. information is obtained that helps control schedule. planning. quality. the effect wanted) with the contributors or causes that will enable that effect to be achieved. productivity.2. This section explains how an effective measurement program is implemented. Measurement is an algorithm connecting the desired result (i. and the contributors are attributes of the processes that will be used to achieve those results. and control essential parameters of the IT processes. and acceptance criteria in a quantitative manner Monitoring progress of a project and/or product Making trade offs in the allocation of resources Evaluating process and product quality Anticipating problems Predicting deadlines of current project Estimating future projects of a similar nature Version 6. Then metrics need to be developed to measure those results and drivers.1 8-21 .e. By measuring processes and products. and control are nearly impossible to achieve. accurate and effective estimating. reliable process or product measures to guide and evaluate results. better-quality products.Metrics and Measurement Work processes that will utilize new technologies normally need to be modified to incorporate those technologies into the step-by-step work procedures.

idea generators. or division VP sponsors an organizational commitment to a measurement program. general manager. As a senior manager. and are assisted by a change agent. Establish tangible objectives and meaningful measurement program activities. needed resources. Unless a very limited program is contemplated. A champion is an advocate for the program by virtue of his/her technical credibility and influence. the sponsor has the authority to ensure understanding at all levels in the organization. Idea generators contribute new ideas about the measurement program. the site manager. and can provide informed realities of it. and information gatekeepers should be identified. These people implement the ideas to form a workable measurement program and ensure developers accept the program. During this step. authority. and organizations responsible for implementing the program. 2. Idea exploiters implement the new ideas in the form of pragmatic programs. or project level. The project or organization selected for the implementation of the measurement program should have the resources. Perceive the need for a measurement program and make a commitment to it. idea exploiters. and assign organizational responsibility. Change agents are most effective when they are members of working groups that will benefit from the measurement program. Information gatekeepers are experts in measurement. They have the operational knowledge needed to schedule. and report the accomplishments of the measurement program. The change agent guides the planning of the program. 8-22 Version 6.2 Prerequisites Implementing a measurement program requires four prerequisite steps: 1. Seeing the need for better management information (as discussed in the prior section). 8. tasks.2. middle.Guide to the CASQ CBOK Results indicate that implementation and application of a measurement program can help achieve better management results. Champions are managers at the senior. 3. control. both in the short run (for a given project) and in the long run (improving productivity on future projects). The planning takes the sponsor's goals for more effective information and defines expected results. The lack of timely and usable quantitative information to solve major project problems becomes apparent at the senior management level. A change agent is a management leader empowered by the sponsor and champions to plan and implement the program. and responsibility to make the program happen. responsibility for implementing the program should not be given to a single individual. including the creation of program objectives and the design of program activities.1 . monitor. program.5. Identify a champion and change agent.

estimating algorithms. For each of the arguments against measurement that might be raised. Lower levels of management may need more detailed. for all projects. over all levels of management. if practiced in a systematic manner. and experience data need to be tailored to an organization's unique environment and processes. The measurement program’s sponsor must clearly inform all levels of management of his/her interest in the measurement program and motivate their cooperation. may not be available in time to solve the problem. responsibilities. and interfaces with other organizations. take it away. they produce estimates that are not meaningful or reliable in that environment. Facilitate management buy-in at all levels for the measurement program. the prime contractor may want to see it. The ability to measure exists if and when it is needed. ensures that information is available at all times. Data in many forms is typically scattered throughout an organization. when needed for problem solving and decision-making. otherwise. All levels of management need measurement data in a meaningful form. Actual experience with measurement suggests that recurring costs of 2 . At that point.3% of direct project costs are adequate for data collection and analysis and for project tracking and monitoring. by tailoring the implementation so that most of their needs are met. authority. System measurement. but they only do it when a problem is apparent. and our budgeting and planning is sufficient. The customer has access to all customer contract data. Many organizations have the ability to measure their performance. Also important is to work with affected managers to obtain their buy-in. This small price buys real help in meeting project goals. must have their parameter values calibrated to reflect the organization's unique environment. if it exists at all. including management data not collected as a part of the • • • • Version 6.1 8-23 . Experience shows that controllability of system development projects decreases when budgets and the budgeting process bear little relation to the operating environment. risk reduction. estimates.2.Metrics and Measurement 4. or use it to harm your organization. while useful. If data is collected. available. and can require access to a central measurement database. there is a counter argument as to its importance. • Measurement has a high cost. and in increasing project control through better budgeting. To be good enough. metrics. too much investment is required and the return is too low. and incremental process improvement. appropriate information. quantitative technical information. All the data exists to support special studies for the senior management. They need to know the implementation team's goals. but all levels need the information that the measurement function can provide. problem anticipation. or accessible on a timely basis. Industry standard estimating algorithms. but the data may not be organized. Our estimates are based on standard industry methods.

it cannot be successfully managed for long without information. Reliable information about a business requires measurements. After a contract is satisfactorily completed. • There is no use for a measurement program in the organization. The measurement database will contain information from past projects as well as ongoing projects. it is unlikely that the old data will be requested. 8-24 Version 6.Guide to the CASQ CBOK contract.1 . The bottom line is that if a business cannot be measured. it should be kept under a reasonable level of security. Because this database will prove vital to the management of the business.2.

Skill Category 9 Internal Control and Security T wo key issues for quality assurance are internal control and security. Interest in internal control has been highlighted by the passage of the Sarbanes-Oxley Act. Principles and Concepts of Internal Control Environmental or General Controls Transaction Processing Controls The Quality Professionals Responsibility for Internal Control and Security Building Internal Controls Building Adequate Security page 9-2 page 9-5 page 9-6 page 9-9 page 9-9 page 9-12 Version 6. For Securities and Exchange Commission (SEC)-regulated corporations. was passed in response to the numerous accounting scandals such as Enron and WorldCom.2. While much of the act relates to financial controls. there is a major section relating to internal controls. Interest in security has been highlighted by publicized penetrations of security and the increased importance of information systems and the data contained by those systems. Because misleading attestation statements is a criminal offense. The Sarbanes-Oxley Act. sometimes referred to as SOX.1 9-1 . top corporate executives take internal control as a very important topic. both the CEO and the CFO must personally attest to the adequacy of their organization’s system of internal control.

some very specific. • Exposure – The amount of loss that might occur if an undesirable event occurs.” The following four key terms are used extensively in internal control and security: • Risk – The probability that an undesirable event will occur. • Control – Anything that will reduce the impact of risk. American Institute of Certified Public Accountants. which oversees the Sarbanes-Oxley Act. The COSO report defines internal control as: "…A process. designed to provide reasonable assurance regarding the achievement of objectives in the following categories: • • • Effectiveness and efficiency of operations Reliability of financial reporting Compliance with applicable laws and regulations. effected by an organization’s Board of Directors. This group is called the Committee of Sponsoring Organizations and is frequently referred to by the acronym COSO.1 . but others take a much broader view of internal control. • Threat – A specific event that might cause an undesirable event to occur. management and other personnel. A threat that might cause that risk to turn into a loss might be an improper electrical connection or children playing with matches. Some of those definitions focus more on financial controls. This is because the Act requires organizations to have a “framework for internal control” and the SEC. The five members of the group are: Financial Executives International. which is the risk of fire. Controls that would minimize the loss associated 9-2 Version 6.2. In the 1990’s five major accounting organizations developed a framework for internal control. only recognizes the COSO Internal Control Framework.1 Principles and Concepts of Internal Control There are many definitions of internal control. and the Institute of Management Accountants. The Institute of Internal Auditors.1 Internal Control and Security Vocabulary and Concepts There is no one generally accepted definition of internal control.Guide to the CASQ CBOK 9. Let’s look at an example of these terms using a homeowner’s insurance policy and focus on one risk. American Accounting Association. it is important to have a clear definition of internal control. However. 9. Many have developed definitions. Note that security is part of a system of internal control. some broad. The COSO Internal Control Framework has been widely accepted after the passage of the Sarbanes-Oxley Act. The exposure associated with a risk of fire would be the value of your home. Most of those definitions were developed by accountants.1.

is responsible for an organization’s internal control system. All management personnel play important roles and are accountable for controlling their units’ activities. Monitor and evaluate the effectiveness of the organization’s risk management system. limiting unusual transactions such as transferring the monies to an overseas account. or the amount of money that the bank allows to be transferred electronically. The exposure is the amount of money in the account. 9.Internal Control and Security with this risk would include such things as fire extinguishers. we might look at the risk of someone penetrating a banking system and improperly transferring funds to the perpetrators personal account. objective assurance and consulting activity designed to add value and improve an organization’s operations. the professional association representing internal auditors worldwide. they are not responsible for the effectiveness of.1. The chief executive officer is ultimately responsible for the internal control system.1. which allow the perpetrator to penetrate the banking system. and a control which limits who can transfer money from the account. and governance processes.1 Internal Control Responsibilities Everyone in an organization has some responsibility for internal control. The Institute of Internal Auditors. It helps an organization accomplish its objectives by bringing a systematic. which was penetrated. specify that internal auditors should: • • Assist the organization by identifying and evaluating significant exposures to risk and contributing to the improvement of risk management and control systems.1.” International Standards for the Professional Practice of Internal Auditing established by the Institute of Internal Auditors.2. but they do not have primary responsibility for establishing or maintaining it. control. such as lawyers and external auditors. disciplined approach to evaluate and improve the effectiveness of risk management. 9. Management. the organization’s internal control system. contribute to the achievement of the organization’s objectives and provide information useful in improving internal control. Controls can include passwords limiting access.1. Internal auditors contribute to the ongoing evaluation of the internal control system. Version 6. defines internal auditing as: “… an independent. limiting the amount that can be transferred at any one time. However.1 9-3 . Financial and accounting officers are central to the way management exercises control. fire alarms and non-combustible material used in construction.2 The Internal Auditor’s Internal Control Responsibilities Internal auditors directly examine internal controls and recommend improvements. The risk is the loss of funds in the account. In looking at information technology. The threat is inadequate security systems. however. A number of other parties. sprinkler systems. nor are they a part of. The Board of Directors and its audit committee provide important oversight to the internal control system.

Guide to the CASQ CBOK • Evaluate risk exposures relating to the organization’s governance.1 . is the responsibility of the CEO. Similarly. It should be recognized that the internal audit function does not – as some people believe – have primary responsibility for establishing or maintaining the internal control system. consideration and response. state that internal auditors should be independent of the activities they audit. That. and internal auditor authority to follow up on findings and recommendations. They possess. test the timeliness of shipments to customers. Internal auditors are objective when not placed in a position of subordinating their judgment on audit matters to that of others. The internal auditors play an important role in evaluating the effectiveness of control systems and thus contribute to the ongoing effectiveness of those systems. • 9-4 Version 6. Organizational position and authority involve such matters as a reporting line to an individual who has sufficient authority to ensure appropriate audit coverage. as noted. The Institute of Internal Auditors standards also set forth the internal auditors’ responsibility for the roles they may be assigned. The primary protection for this objectivity is appropriate internal audit staff assignments. and contracts Assist the organization in maintaining effective controls by evaluating their effectiveness and efficiency and by promoting continuous improvement. along with key managers with designated responsibilities. the internal audit function is heavily involved with controls over operations. In some entities. In other entities.2. Those standards. they should not be assigned to audit activities with which they were involved in connection with prior operating assignments. All activities within an organization are potentially within the scope of the internal auditors’ responsibility. These assignments should be made to avoid potential and actual conflicts of interest and bias. among other things. regulations. the internal audit function may focus primarily on compliance or financial reporting-related activities. or evaluate the efficiency of the plant layout. internal auditor access to the Board or audit committee. operations. such independence through their position and authority within the organization and through recognition of their objectivity. internal auditors may periodically monitor production quality. selection and dismissal of the director of internal auditing only with Board of Directors or audit committee concurrence. For example. or should possess. and information systems regarding the: Reliability and integrity of financial and operational information Effectiveness and efficiency of operations Safeguarding of assets Compliance with laws. Staff assignments should be rotated periodically and internal auditors should not assume operating responsibilities.

management would define a control objective of “All products shipped should be invoiced”.2 Environmental or General Controls Environmental controls are the means which management uses to manage the organization. They include such things as: Organizational policies Organizational structure in place to perform work Method of hiring.2. they recognize that there will be a risk that products will be shipped but not invoiced. Therefore. The formula for risk is as follows: Risk = Frequency x Probable Loss To calculate the loss due to risk. such as a system development methodology for building software systems. 9. training. To address risk such as this. supervising and evaluating personnel Processes provided to personnel to perform their day-to-day work activities. There is a risk that products shipped will not be invoiced. Let’s look at a simple example. Auditors state that without strong environmental controls the transaction processing controls may not be effective. if passwords needed to access computer systems are not adequately • • • • Version 6. one must first determine: • The frequency with which an unfavorable event will occur. and • The probable loss associated with that unfavorable occurrence. the sole purpose of control is to reduce risk.1. If we were to assume that an average of two products will be shipped per day and not be invoiced and the average billing per invoice is $500. Management has chosen to use a positive concept in addressing risk. For example. management has chosen to define control objectives rather than risks. if there is no risk.000 per day.1 9-5 . then the risk associated with not invoicing shipments is $1. and the second is the transaction processing controls within an individual business application. 9. there is no need for control. The first is environmental controls (sometimes called general controls). rather than a negative concept.Internal Control and Security 9. In our shipped but not billed risk example.2 Risk versus Control From an academic perspective.1. They would then implement controls to accomplish that positive control objective.3 Environmental versus Transaction Processing Controls It is important for the quality professional to know that internal control systems have two components. In other words.

it is difficult to see how the totality of control over the processing of a transaction is controlled. 9. these two are designed and implemented as one system. Risks are the probability that some unfavorable event may occur during processing. However. Controls are the totality of means used to minimize those business risks. and corrective and provides examples of those types of controls. and corrective controls correct incorrect processing. When one visualizes a single system. There are two systems in every business application.1 Preventive. those edits are part of the system that controls the processing of business transactions.Guide to the CASQ CBOK protected the password system will not work. detect. Adding to the difficulty is that the system documentation is not divided into the system that processes transactions and the system that controls the processing of transactions. Since the potential for errors is always assumed to exist. Individuals will either protect. For example. if one looks at edits of input data by themselves. The first is the system that processes business transactions. and the second is the system that controls the processing of business transactions. Because these two systems are designed as a single system. For example. That is. or not protect. most software quality analysts do not conceptualize the two systems. the monitoring of the use of passwords that exist. the objectives of transaction processing controls will be summarized in five positive statements: 9-6 Version 6. Preventive controls will stop incorrect processing from occurring.3.1 . From the perspective of the system designer. The objectives of transaction processing controls are to prevent. and management’s actions regarding individual workers failure to protect passwords.2. 9.3 Transaction Processing Controls The object of a system of internal control in a business application is to minimize business risks. This risk occurs throughout the system and not just during the editing of data. or correct incorrect processing. Also provided is a detailed process to follow when building controls within an information system. one has difficulty in visualizing the total system of internal control. edits that determine the validity of input are included in the part of the system in which transactions are entered. there is a risk that invalid transactions will be processed. detective. Detective and Corrective Controls This section describes three different categories of transaction processing controls. preventive. When the system of internal controls is viewed it must address all of the risks of invalid processing from the point that a transaction is entered into the system to the point that the output deliverable is used for business purposes. their password based on environmental controls such as the attention management pays to password protection. detective controls identify incorrect processing.

Preventive controls are located throughout the entire IT system. However. Preventive controls include standards. One question that may be raised is. pre-numbered forms. The following preventive controls will be discussed in this chapter: • • • • • • • Source-data authorization Data input Source-data preparation Turn-around documents Pre-numbered forms Input validation Computer updating of files • Version 6. consistency of operations. For these reasons transaction processing controls have been classified according to whether they prevent.” Some input controls depend on access to master files and so must be timed to coincide with file availability. representative of these control types. • Assure that the application can continue to function. authorization. • Assure that transaction processing is correct and appropriate to the circumstances. By including the tests in programs performing such operations. • Assure that processing results are utilized for the intended benefits. Preferably. documentation.1 Preventive Controls Preventive controls act as a guide to help things happen as they should. “At what point in the processing flow is it most desirable to exercise computer data edits?” The answer to this question is simply. A single control can also fulfill multiple control objectives. The controls listed in the next sections are not meant to be exhaustive but.3. passwords. • Assure that transaction data is complete and accurate. in order to uncover problems early and avoid unnecessary computer processing. the input validation tests are included in programs to perform data-conversion operations such as transferring data files from one application to another. the controls may be employed without significantly increasing the computer run time. rather. “As soon as possible.2. 9. Many of these controls are executed prior to the data entering the computer programs. This type of control is most desirable because it stops problems from occurring. these tests should be performed in a separate edit run at the beginning of the computer processing. Normally. forms design. It is more economical and better for human relations to prevent a problem from occurring than to detect and correct the problem after it has occurred. or correct causes of exposure. etc.1 9-7 . In most instances controls can be related to multiple exposures. Computer application systems designers should put their control emphasis on preventive controls. detect.1. training.Internal Control and Security Assure that all authorized transactions are completely processed once and only once. many input validation tests may be performed independently of the master files. segregation of duties.

Therefore detective control should be applied to corrective controls. The following detective controls will be discussed here: • • • • • Data transmission Control register Control totals Documentation and testing Output Checks 9. batch serial numbers. and audit trails. etc. backup and recovery. and so forth. error statistics. Such a transaction may be correct. Examples of detective controls are batch control documents. discrepancy reports.1. labeling.3. Detective controls will not prevent problems from occurring. but rather will point out a problem once it has occurred.1.Guide to the CASQ CBOK • Controls over processing 9. Detective controls should bring potential problems to the attention of individuals so that action can be taken. It should be noted that the corrective process itself is subject to error. These costs need to be evaluated as no control should cost more than the potential errors it is established to detect. 9.3 Corrective Controls Corrective controls assist individuals in the investigation and correction of causes of risk exposures that have been detected. The following two corrective controls will be discussed in greater detail: error detection and resubmission.1 . Corrective action is often a difficult and timeconsuming process. if controls are poorly designed or excessive. 9-8 Version 6.2. or it may be a systems error. however.3. One example of a detective control is a listing of all paychecks for individuals who worked over 80 hours in a week. Also.3. clearing accounts. they become burdensome and may not be used. or correct. These controls primarily collect evidence that can be utilized in determining why a particular problem has occurred. Many major problems have occurred in organizations because corrective action was not taken on detected problems.2 Detective Controls Detective controls alert individuals involved in a process so that they are aware of a problem. Many system improvements are initiated by individuals taking corrective actions on problems.2 Cost versus Benefit of Controls In information systems there is a cost associated with each control. Examples of corrective controls are audit trails. or even fraud. it is important because it is the prime means of isolating system problems. The failure to use controls is a key element leading to major risk exposures. prevent.

The quality professional should support the importance of environmental controls in creating an environment conducive to effective internal control and security. eliminating. Detective controls usually require some moderate operating expense.4 The Quality Professionals Responsibility for Internal Control and Security The quality professional is the organization’s advocate for quality. 9. prior to installing any control. The quality professional should not be responsible for building or assessing the adequacy of internal control and security systems. This is a prime function of the auditor. 9. Version 6. corrective controls are almost always quite expensive. The quality professional can also evaluate the effectiveness and efficiency of those work processes. The assessment normally uses a combination of qualitative and quantitative methods. first.1 Perform Risk Assessment Building controls starts with risk assessment because reduction in risk is the requirement for a control. The role of the quality professional involves identifying opportunities to improve quality and facilitating solutions. Because internal control and security are important responsibilities of management. the impact of that event. As the result of such a review an auditor may recommend adding. As noted above.Internal Control and Security Preventive controls are generally the lowest in cost. The control models emphasize the importance of the control environment.5. Risk assessment allows an organization to consider the extent to which potential events might have an impact on the achievement of objectives. the likelihood of an event occurring and second. Controls need to be reviewed continually.5 Building Internal Controls The system of internal control is designed to minimize risk. 9. On the other hand.2. but can review the control environmental practices in place in the IT organization. Management should assess events from two perspectives. the quality professional can assist in building work processes for building and assessing internal control and security. Normally quality assurance does not establish the control environment. or modifying system controls.1 9-9 . The following section on control models will emphasize the importance of a strong control environment. This section will focus on building transaction processing control in software systems. However. The auditor should determine if controls are effective. a cost/benefit analysis should be made. the quality professional should be involved in those areas.

and degree of regulation over its activities.2. Manual application control procedures are also required within IT. as well as the control of the data flow within a computer system. File retention and security procedures may be required and specified for individual computer applications. including organization size. the IT input/ output control section frequently balances and reconciles input to output. 9-10 Version 6. that shape the organization’s risk profile and influence the methodology it uses to assess risks. In risk assessment. For example. This entails examining factors. management considers the mix of potential future events relevant to the organization and its activities.Guide to the CASQ CBOK The positive and negative impacts of potential events should be examined. and submitted to IT. individually or by category. Manual procedures in user areas are developed to ensure that the transactions processed by IT are correctly prepared. Potentially negative events are assessed on both an inherent and residual basis.2 Model for Building Transaction Processing Controls System controls for computer applications involve automated and manual procedures. Automated procedures may include data entry performed in user areas.5. complexity of operations. Figure 9-1 shows the six steps of a transaction flow through a computer application system. authorized.1 . Such controls are unique to the requirements of the application and complement management controls that govern input/output controls and the media library. across the organization. Transaction flow is used as a basis for classifying transaction processing controls because it provides a common framework for users and system development personnel to improve computer application system controls. 9.

5. including message accountability. 9.2.2 Transaction Entry Transaction entry controls govern the data entry via remote terminal or batch.3 Transaction Communications Transaction communications controls govern the accuracy and completeness of data communications. data protection hardware and software.2.1 9-11 . Each step is described below.5. transaction or batch proofing and balancing.2. Version 6. error identification and reporting. 9. security and privacy.5. and error identification and reporting.Internal Control and Security Figure 9-1 Model for Building Transaction Processing Controls The two shaded areas in the figure indicate the steps which require the greatest involvement of the user organization. data validation. approval.1 Transaction Origination Transaction originations controls govern the origination.2. and processing of source documents and the preparation of data processing input transactions and associated error detection and correction procedures. 9. and error correction and reentry.

Guide to the CASQ CBOK 9. such as protecting passwords.5. This is typically accomplished via security awareness training.1 . data security and privacy. the opinion can be based on that documentation. backup. recovery. However. including the appropriateness of machine-generated transactions. quality assurance needs to build a security baseline. 9. 9.5 Database Storage and Retrieval Transaction database storage and retrieval controls govern the accuracy and completeness of database storage. It is the point at which software systems are easiest to penetrate. and output data retention. error handling.6 Building Adequate Security Security is heavily dependent on management establishing a strong control environment that encourages compliance to security practices. Good security practices. and second is the security technical controls. will not be effective unless employees are diligent complying with password protection practices. and retention. 9-12 Version 6. Understanding the vulnerabilities helps in designing security for information systems. Quality assurance should focus on the security management controls. Security can be divided into two parts. control over negotiable documents (within data processing and user areas). security experts are needed to identify.2. install and monitor the technical controls such as anti-virus software.1 Where Vulnerabilities in Security Occur Vulnerability is a weakness in an information system. 9. 9. validation against master files. and error identification and reporting. As a general rule. Normally.6 Transaction Output Transaction output controls govern the manual balancing and reconciling of input and output (within the input/output control section and at user locations). prior to building the baseline the team needs to understand the vulnerabilities that allow security penetrations.6. if risks are significant.5. To build good security management controls. If the quality assurance analysts and/or the individual developing the adequacy opinion can match the risks with controls. controls should be strong. The next section identifies some of the more widely used security practices. First is the security management controls.4 Transaction Processing Transaction processing controls govern the accuracy and completeness of transaction processing. distribution of data processing output.

6. Nine primary functional locations are listed. Programming Offices 6. Media 9.Internal Control and Security This section describes vulnerabilities that exist in the functional attributes of an information system.6. Input/Output Data 2. Physical Access 3. Test Processes 5. Online Data Preparation and Report Generation Version 6. Data and Report Preparation Facilities 2. 9. and distinguishes accidental from intentional losses. listed in order of historic frequency of abuse: 1. Non-IT Areas 4. Online Terminal Systems 5. Computer Operations 3.1 9-13 .1. described.2 IT Areas Where Security is Penetrated Data and report preparation areas and computer operations facilities with the highest concentration of manual functions are areas most vulnerable to having security penetrated.1. and ranked according to vulnerability: 1.2. identifies the location of those vulnerabilities. Computer Programs 6. Impersonation 8. IT Operations 4.1 Functional Vulnerabilities The primary functional vulnerabilities result from weak or nonexistent controls in the following eight categories. Operating System Access and Integrity 7.

but proof of this is usually required before searching elsewhere for the cause. Solutions are well known and usually well applied relative to the degree of motivation and costeffectiveness of controls. employees do not understand the significant difference between accidental loss from errors and intentionally caused losses. Finally.2 Establishing a Security Baseline A baseline is a snapshot of the organization’s security program at a certain time.Guide to the CASQ CBOK 7. one that is much more challenging and that requires adequate safeguards and controls not yet fully developed or realized.1 . let alone adequately applied. They anticipate.6. Online Operations 9. In many computer centers. experience. The next most common area of suspicion is users or the source of data generation because. The problem is rarely a hardware error. employees and managers tend to blame the computer hardware first (to absolve themselves from blame and to pass the problem along to the vendor to solve).1. This presents a different kind of vulnerability. that the same controls used in similar ways also have an effect on people engaged in intentional acts that result in losses. and access capability to solve the problem or reach a goal. The errors are usually data errors. and users arguing over who should start looking for the cause of a loss. computer program errors (bugs). In fact. The baseline is designed to answer two questions: • • What are we doing about computer security? How effective is our computer security program? 9-14 Version 6. and damage to equipment or supplies. again. when all other targets of blame have been exonerated. some reported intentional loss is due to perpetrators discovering and making use of errors that result in their favor. and replacement/repair of equipment/supplies.6. When loss occurs. however. since the beginning of automated data processing. Central Processors 9.2. programmers.3 Accidental versus Intentional Losses Errors generally occur during labor-intensive detailed work and errors lead to vulnerabilities. Blame is usually next placed on the computer programming staff. Organizations using computers have been fighting accidental loss for 40 years. Such errors often require rerunning of jobs. the IT department can blame another organization. error correction. Digital Media Storage Facilities 8. 9. They frequently fail to understand that they are dealing with an intelligent enemy who is using every skill. It is not uncommon to see informal meetings between computer operators. The thought that the loss was intentional is remote because they generally assume they function in a benign environment. IT employees suspect their own work. Nevertheless. it is often difficult to distinguish between accidental loss and intentional loss. maintenance engineers.

or against. Even though some of the facts are based on attitude.1 9-15 . it reduces computer security discussions from opinion to fact. if a security software package is acquired. It reports the status of the current program and provides a basic standard against which improvements can be measured. In some instances. In other instances.1 Creating Baselines The establishment of a security baseline need not be time-consuming. A determination must be made about what specific pieces of information would be helpful in analyzing the current security program and in building a more effective computer security program • From whom will the information be collected? Determining the source of information may be a more difficult task than determining what information should be collected. as opposed to people’s opinion and prejudices. the uncertainty about the last expenditure may eliminate the probability of a new improvement. The desired precision should be both reasonable and economical. and ignore the information that is difficult to collect.2. In many instances.6. bias for. but there is no way to determine whether the environment has been improved. When the next computer security request is made. they are a statistical base of data on which analyses and discussion can be focused. management will wonder whether that expenditure was worthwhile. In many instances. it must be carefully weighed against the benefit of having highly precise information. There is a tendency to want highly precise information. the source will be current data collection mechanisms (if used by the organization). First. • The precision of the information collected.3 Using Baselines The baseline is the starting point to a better security program. The process itself should measure both factual information about the program and the attitudes of the people involved in the program.6. If people are being asked to identify past costs. the same decisions would be made regardless of whether the precision was within plus or minus 1 percent. high precision is unreasonable. and if the cost is large.2. individuals will be asked to provide information that has not previously been recorded. but in many instances it is not necessary. 9. a security program should be removed from the process. The three key aspects of collecting computer security baseline information are as follows: • What to collect. the needed information may be already available. For example. Version 6. The baseline study serves two primary objectives. as much as possible. The objective is to collect what is easy to collect. The baseline helps answer the question of whether the expenditure was worthwhile. or within plus or minus 50 percent. 9.Internal Control and Security Baseline information should be collected by an independent assessment team.

procedures. and availability of information in today’s highly networked systems environment without ensuring that all the people involved in using and managing IT: • • • Understand their roles and responsibilities related to the organizational mission Understand the organization’s IT security policy.6. A robust and enterprise-wide awareness and training program is paramount to ensure that people understand their IT security responsibilities. 3. • As cited in audit reports. some general information security principles should be kept in mind. Create a security awareness policy. The strategy is presented in a life cycle approach. The “people factor” – not technology – is the key to providing an adequate and appropriate level of security. and practices Have at least adequate knowledge of the various management. ranging from designing. This practice provides a strategy for building and maintaining a comprehensive awareness and training program. the process that follows is an all inclusive process of the best security awareness training program. more and better attention must be paid to this “component of security”.2. develop a training plan. developing. and conference presentations. This example includes these three steps: 1. operational.1 . and technical controls required and available to protect the IT resources for which they are responsible Fulfill their security responsibilities. and get organizational buy-in for the funding of awareness and training program efforts. and how to properly use them to protect the IT resources entrusted to them.Guide to the CASQ CBOK 9. Develop the strategy that will be used to implement that policy. organizational policies. and implementing an awareness and training program. it is generally agreed by the IT security professional community that people are the weakest link in attempts to secure systems and networks. through post-implementation evaluation of the program. If people are the key. The security awareness program includes guidance on how IT security professionals can identify awareness and training needs. as follows: 9-16 Version 6.5 Security Practices When addressing security issues.6. but are also a weak link. Assign the roles for security and awareness to the appropriate individuals. 2. 9. periodicals. integrity. as part of an organization’s IT security program. While there is no one best way to develop a security awareness program.4 Security Awareness Training IT organizations cannot protect the confidentiality.

Internal Control and Security • • • • • • • • Simplicity Fail Safe Complete Mediation Open Design Separation of Privilege Psychological Acceptability Layered Defense Compromise Recording Version 6.1 9-17 .2.

9-18 Version 6.Guide to the CASQ CBOK This page intentionally left blank.1 .2.

COTS and Contracting Quality O rganizations can assign software development work responsibilities to outside organizations by purchasing software or contracting services. Quality of software remains an internal IT responsibility regardless of who builds the software.2. Quality and Outside Software Selecting COTS Software Selecting Software Developed by Outside Organizations Contracting for Software Developed by Outside Organizations Operating for Software Developed by Outside Organizations page 10-1 page 10-5 page 10-6 page 10-7 page 10-8 10.Skill Category 10 Outsourcing. The quality professionals need to assure that those quality responsibilities are fulfilled through appropriate processes for acquiring purchased software and contracting for software services. but they cannot assign the responsibility for quality.1Quality and Outside Software There is a trend in the software industry for organizations to move from in-house developed software to commercial off-the-shelf (COTS) software and software developed by contractors. Software developed by contractors who are not part of the organization is referred to as Version 6.1 10-1 .

and thus it is difficult to oversee the development process. 10-2 Version 6. versus performing an assessment which requires that the software be tested in a dynamic mode before selection. Differences or challenges faced with COTS software include: • • • • • • Task or items missing Software fails to perform Extra features Does not meet business needs Does not meet operational needs Does not meet people needs 10. nor have control over the many day-to-day decisions that are made in developing software. the contractor cannot take workers off one project and assign them to another project. As the COTS software becomes larger and more expensive. Loss of control over reallocation of resources If work needs to be done to correct problems and/or speed up development. as well as differences specific to COTS and contractor developed software. • 10. The contracting organization cannot direct the employees of the other organization.2. There are some common differences between software developed in-house and that developed by any outside organization.Guide to the CASQ CBOK outsourcing organizations.1.1 Evaluation versus Assessment Many organizations select COTS software based on evaluation which is a static analysis of the documentation and benefits of the software. For smaller. Quality professionals should be familiar with these differences as they impact their quality responsibilities.1.1. Two differences between software developed in-house and software developed by an outside organization are: • Relinquishment of control The software is developed by individuals who are not employees of the organization. Contractors working in another country are referred to as offshore software developers. less expensive software packages the software is normally “shrink wrapped” and is purchased as-is.1 Purchased COTS software COTS software is normally developed prior to an organization selecting that software for its use. the purchasing organization may be able to specify modifications to the software.1 .

1.1. Customer’s standards may not be met Unless the contract specifies the operational standards and documentation standards the delivered product may be more complex to use than desired by the purchasing orgainzation. Non-testable requirements and criteria If the requirements or contractual criteria are in measurable and testable terms then the delivered result may not meet the intent of the purchasing orgainzation.2. COTS and Contracting Quality 10.1 10-3 .1 Additional differences if the contract is with an offshore organization Experience has shown that over 50% of the software developed by offshore organizations fails to meet the expectations of the purchasing organization. the differences associated with having the software developed offshore negate the economic advantages in many cases.2. These offshore differences are: Version 6. Thus when the software is delivered it may not be as easy to use or as reliable as desired by the purchasing orgainzation.Outsourcing. Missing requirements Unless detailed analysis and contractual specifications work is complete the purchasing organization may realize during the development of the software that requirements are missing and thus the cost of the contract could escalate significantly. Since many of the decisions to have software developed offshore are economic decisions. Overlooked changes in standards in technology If changes in standards that the purchasing organization must meet. • • • • • 10.2 Outsourced Software The differences in contracted software developed by an outsourcer include: • Quality factors may not be specified There are many factors such as reliability and ease of use which are frequently not included as part of the contractual criteria. or the introduction of new desirable technology is incorporated into the contract there may be significant cost to modify the software for those new standards in technology. Training and deployment may be difficult If software is developed by another organization there may be inadequate knowledge in the contracted organization to provide the appropriate training for staff and to ensure that deployment is effective and efficient.

Guide to the CASQ CBOK • Cultural differences There may be a significant difference in the culture and values between the purchasing organization and the offshore organization. Communication barriers The language of the offshore organization may be different or difficult to comprehend which causes difficulty in communicating the needs and desires of the purchasing orgainzation. Loss of employee morale and support Employees who would like to have developed the software may resent the software being developed offshore and thus make it difficult for the offshore developed software to be successful. Root cause of the purchasing organization IT organization not addressed Frequently, offshoring is chosen because there are problems in the purchasing organization that executives do not want to address. For example, the problems might include a lack of training for the employees or other, perhaps better, options for software development were not explored.

The above discussions are not meant to be an exhaustive list of the differences between in-house developed software and software developed by outside organizations. The objective is so the quality assurance professional recognizes some potential root causes of software quality. If those differences are not adequately addressed in the contract, and with employees of the organization, the probability of the contracted or offshore-developed software not meeting expectations increases. Quality Professionals Responsibility for Outside Software While the software may be developed by an outside organization, the responsibility for quality of that software cannot be contracted. The purchasing organization is still responsible for the quality of the organization. There must be a process to monitor the development and validate the correct functioning of the software when it is developed by an outside organization. The quality professional is the individual who should accept the quality responsibility for software developed by an outside organization. This may mean that the quality professional needs to visit periodically or during the entire developmental period of the software to ensure the quality. Many of the same practices used to assure quality of in-house developed software are applicable to software developed by outside organizations. For example, conducting reviews at specific checkpoints should occur on contracted software. Acceptance testing should be conducted on all software regardless of how developed. The quality professional’s specific responsibility for software developed by outside organizations is to assure that the process for selecting COTS software and contracting with an outside organization for software are adequate.


Version 6.2.1

Outsourcing, COTS and Contracting Quality One of the major responsibilities of the quality assurance activity is to oversee the development and deployment of work processes, and then to assure that those work processes are continuously improved. Without a process for selecting COTS software and a process for contracting for software those processes would be subject to great variability. One contract may work well, one acquisition of COTS software may work well, while other acquisitions may result in a failure. The quality professional needs to look at these two processes in the same way that they view the SEI CMMI® Capability Maturity Model. If contracting is done at a Level 1 maturity there will be great variability and thus many disappointments in the delivered product and services. On the other hand, as those processes move to a Level 5 maturity, the probability of getting exactly what is wanted from COTS software and contracted software is very high. This skill category contains a prototype process for selecting COTS software and a prototype process for contracting for software with outside organizations. The two processes incorporate many of the best practices for acquiring software developed by outside organizations.

10.2Selecting COTS Software
There is no generally accepted best practice for acquiring COTS software. However, there are many practices in place by organizations that have a process for selecting COTS software. The process proposed includes many of those best practices. It is important for the quality professional to first understand that a process is needed for acquiring COTS software, and second understanding the key components of that process. Thus, the purpose of presenting a process for selecting COTS software is to facilitate the quality professional understanding the type of criteria that should be included in a COTS software selection process. The following seven-step process includes those activities which many organizations follow in assuring that the COTS software selected is appropriate for the business needs. Each of the processes is discussed below: • • • • • • • Assure Completeness of Needs Requirements Define Critical Success Factor Determine Compatibility with Hardware, Operating System, and other COTS Software Assure the Software can be Integrated into Your Business System Work Flow Demonstrate the Software in Operation Evaluate People Fit Acceptance Test the Software Process

Version 6.2.1


Guide to the CASQ CBOK

10.3Selecting Software Developed by Outside Organizations
Normally contracting for software is a more significant activity than acquiring COTS software. COTS software has been developed and the acquisition challenge is to assure that the developed software will meet the organization’s needs. On the other hand the contracted software may not be developed or may be only partially developed and thus it incorporates many of the aspects of inhouse developed software except the actual implementation of the requirements/criteria. There is no one single process generally accepted for contracting for software. Many large organizations have a purchasing section which specifies many of the components of contracting for software. In addition, large organization’s legal departments may also be involved in writing and approving contracts for software development. If an off-shore organization develops the software, even more attention should be given to the contract because of cultural and communication differences. Off-shore contracts may involve the laws of the country in which the software is developed. The following contracting life cycle incorporates many of the best practices used by organizations in selecting, contracting, and operating software developed by an outside organization. This process commences when the need for software developed by an outside organization is defined, and continues throughout the operation of the contracted software.

10.3.1 Contracting Life Cycle
In order to participate in the various aspects of contracting for software development, it is necessary to establish an acquisition life cycle for contracted software. This life cycle contains the following three activities: • Selecting an Outside Organization Selecting a contractor is similar to systems design. It is a time of studying alternative contractors, costs, schedules, detailed implementation design specifications, and the specification of all the deliverables, such as documentation. The selection of an outside organization may involve the following individuals in addition to the quality control reviewer: • • • • • Systems analysts User personnel Internal auditor Purchasing agent

• Legal counsel Contract Negotiations


Version 6.2.1

Outsourcing, COTS and Contracting Quality In some organizations, the purchasing agent conducts all the negotiations with the contractor; the other parties are involved only in the needs specification. In other organizations, there is no purchasing agent, so the data processing department deals directly with contractors for the acquisition of application systems. • Operations and Maintenance The maintenance and operations of purchased applications may be subject to contractual constraints. For example, some contracts limit the frequency with which an application can be run without paying extra charges, and limit an organization’s ability to duplicate the application system in order to run it in a second location. Some purchased applications can be maintained by in-house personnel, but others do not come with source code and thus are not maintainable by in-house personnel. There may also be problems in connecting a purchased application from one contractor with purchased or rented software from another contractor. It is important that software from multiple contractors be evaluated as to its capability to work in the same operating environment. The contract lists the obligations assumed by both parties. The contractor may be obligated to meet contractual requirements for such things as updating the application system to be usable with new versions of operating systems, etc. The purchasing organization may have obligations for protecting the application system from compromise, paying for extensive use of the application, etc. This provision should be monitored and enforced during the life of the contract. Provisions which are violated and not enforced may be unenforceable at a later point in time due to the implied agreement by one party not to enforce a provision of the contract. Therefore, it is important that contracts be reviewed regularly to determine that all the provisions of the contract are being enforced.

10.4Contracting for Software Developed by Outside Organizations What Contracts Should Contain Contracts are legal documents. To fully understand the impact of provisions being included in, or excluded from, the contract may require legal training. However, the following information should be included in all contracts: • What is done. The contract should specify the deliverables to be obtained as a result of execution of the contract. Deliverables should be specified in sufficient detail so that it can be determined whether or not the desired product has been received. Who does it.

Version 6.2.1


Guide to the CASQ CBOK The obligations of both contractual parties should be spelled out in detail in the contract. • When it is done. The dates on which the contractual obligations need to be filled should be specified in the contract. How it is done. The contract should specify the methods by which the deliverables are to be prepared if that is important in achieving the contractual needs. For example, the organization may not want certain types of source instructions used in developing an application system because they plan to perform the maintenance with the in-house personnel. Where it is done. The location where the deliverables are to be prepared, delivered, operated, and maintained should be specified as part of the contract. Penalties for nonperformance. If the contractual agreements are not followed, the penalties should be specified in the contract. For example, if the contractor is late in delivering the work products, the contract may specify a penalty of x dollars per day.

10.5Operating for Software Developed by Outside Organizations
Three activities occur once the software is ready for delivery. These are acceptance testing, operation and maintenance of the software, and contractual relations.

10.5.1 Acceptance Testing
The acceptance testing phase is necessary to verify that the contractual requirements have been met. During this phase, the customer’s personnel verify that the delivered products are what the organization needs. This requires examination and testing of the deliverables. The contractor has a responsibility to test the application to determine that it meets contractual requirements. However, the contractor will be testing against what the contractor believes to be the contractual requirements. This may or may not be what the customer needs. The customer tests against what is actually wanted. In some cases, there will be a difference between what the user wants and what the contractor delivers. If the contract has specified the deliverables in sufficient detail, the customer will have


Version 6.2.1

If differences can be uncovered during the development phase.2. and other application systems. However.1. correction is not costly and will usually be made by the contractor. The customer is always in a better position to negotiate changes prior to contract signing than after contract signing. Preparation for operation Any supporting activities should be completed in time for acceptance testing. This may involve the ordering of forms. or it may be done by the contractor. etc.Outsourcing. For application systems. COTS and Contracting Quality adequate recourse for correction. developing procedures to use the application. Ideally. • On time The delivered products will be in the hands of the user on the date specified on the contract. the acceptance test phase occurs throughout the development process if the application is being developed for the customer. This training may be provided by the organization itself. The acceptance test criteria for training courses and manuals may be review and examination. problems uncovered after the application system has been developed can be costly. making changes to existing operating systems. if the contract is loosely worded. However. the contractor may be asked to put on a training class in order to determine the adequacy of the material.5. When can it begin • • • • Version 6. and may result in resistance by the contractor making the change. The specific concerns during this phase are: • Meets specifications All of the deliverables provided by the contractor meet the user requirements as specified in the contract.1 Acceptance Testing Concerns The primary concern of the user during acceptance testing is that the deliverables meet the user requirements. This is ideal from a user perspective because they know exactly what they are getting. If the application systems have been developed prior to contract negotiations. the user can perform acceptance testing prior to signing the contract. the customer may be obligated to pay all or part of additional costs necessary to meet the customer’s specifications. User training The users of the application should be trained prior to the acceptance test phase. or what modifications are needed prior to contract signing. 10.1 10-9 . The customer should have identified these needs during the feasibility phase. For example. Adequate test data The customer should prepare sufficient test data so that the deliverables can be adequately tested. this may be test transactions to verify the processing performed by the application. and worked on meeting those requirements while the contractor was preparing the application system.

10. and when they do occur the organization neglects to go back to the contract to determine the obligation of the contractor for these problems. Frequently organizations overlook contractual agreements after the application system has gone operational.1. Thus. The relationship should not be considered fixed at the point in time the contract was signed but. special needs and interests of the customer are normally handled even if they are above and beyond the contractual negotiations.5. The contractor is anxious to sell more applications and service to the customer. rather. or produce it at an equivalent cost or time span. It encompasses continued service and maintenance. a continual evolving relationship in which both the interest of the contractor and the customer are protected. The customer hopefully has received a valuable product from the contractor.2 Operation and Maintenance of the Software The contractual arrangements determine the ongoing relationship between the contractor and the customer. • Conversion to production The procedures and steps necessary to put the application into production should be tested during the acceptance testing phase.3. The major concern during the operation and maintenance of a purchased or leased application system is that both parties to the agreement comply with the contractual requirements. These procedures should be tested just as vigorously as the contractor’s application system. However. Reviewers should evaluate negotiations over time to determine that the contractor fulfills their part of the contract. In most instances. Time and effort must be expended to keep that a viable and healthy relationship.1 Contractual Relation Concerns The concerns that arise in maintaining a relationship of harmony and good will include: 10-10 Version 6. 10.1 . There are also instances where the customer is obligated to meet ongoing contractual obligations and compliance to those obligations should be verified. the customer either could not produce this product.3 Contractual Relations The relationship between the contractor and the customer is an ongoing relationship. the ongoing relationship may only involve guarantee and warranty of the product.Guide to the CASQ CBOK Acceptance testing should occur on the date which was specified in the contract. schedule production. This is because problems may not occur initially. This relationship may continue as long as the customer continues to use the application system.1. etc. Therefore. These are normally customer obligations to prepare files. Contracts should be periodically reviewed to verify that the contractual requirements are being met.5. 10.2. it is normally within the best interest of the customer to gain more products of a similar nature.5. These are normally performed in an effort to continually improve the relationship in hopes of ongoing business.1.

Limits on cost increases The cost specified in the contract should include provisions for ongoing costs. the contractual termination procedures should be performed in accordance with the contract requirements. For example.g.1 10-11 . if the customer wants to extend the contract. Compensation for error If the contractor agrees to compensate for problems due to contractor causes. For example. Need met It is to the benefit of both parties to have the customer satisfied with the application system.. COTS and Contracting Quality • • • Contractor obligations met The contractor should meet their requirements as specified in the contract. • • • • • Version 6. Customer obligations met The customer should meet their requirements as specified in the contract.2. In an inflationary economy. the penalties should be specified in the contract. there will be ongoing maintenance required to meet the continually evolving needs of the customer. that extension may involve a renegotiation of the terms of the contract. The renegotiation process should be conducted in accordance with the contractual specifications. it may be advantageous to have limits placed on cost increases. When new requirements are needed.Outsourcing. Exercising options (e. The methods of doing this should be specified in the contract. Renegotiation Many contracts contain provisions to renegotiate in the event of some specified circumstances. Even when the initial deliverables meet the customer’s need. if service is provided at an hourly rate. added features) Many contracts contain options for additional features or work. it should first be determined if they can be obtained by exercising some of the options already available in the contract. and those requirements should form the basis for both parties specifying and delivering new contractual obligations. Returns on termination If the contract is terminated. the increases in that rate might be specified in the contract.

Guide to the CASQ CBOK This page intentionally left blank. 10-12 Version 6.2.1 .

1994 Edition. The source documents are listed below. using available standards. and Testing of Computer Software. Guideline for Software Documentation Management.g. as much as possible.. Guideline For Lifecycle Validation. IEEE Standards Collection. (NIST). Software Engineering. FDA Technical Report. functional. American National Standard for Information Systems. Verification. Dictionary for Information Systems. qualification. published by the Institute of Electrical and Electronic Engineers Inc. The New IEEE Standard Dictionary of Electrical and Electronics Terms. Those related terms are located sequentially to assist the user in finding all defined terms in these domains. The source of such definitions appears immediately following the term or phrase in parenthesis. American National Standards Institute. Verification.g. and Testing of Computer Software. 1981. Software Development Activities. 1991. May 1987. 1983. e.Appendix A Vocabulary The organization of this document is primarily alphabetical. The terms are defined. FDA Guide to Inspection of Computerized Systems in Drug Processing. 1983. National Bureau of Standards [NBS] Special Publication 500-75 Validation. Federal Information Processing Standards [FIPS] Publication 105. Acronyms are expanded at the beginning of each alphabetical section and defined with the full term or phrase. 1984. Federal Information Processing Standards [FIPS] Publication 101. 1001992.2 A-1 . IEEE Std. Four modifications are the grouping of terms and phrases in the domains of specifications. terms and phrases. testing. and validation. Acronyms are grouped at the beginning of each alphabetical section. FDA Guideline on General Principles of Process Validation. Version 6. e. and are followed by words. July 1987. functional testing is defined under testing.

Application of the Medical Device GMP to Computerized Devices and Manufacturing Processes. Software Testing Techniques.Guide to the CASQ CBOK Reviewer Guidance for Computer Controlled Medical Devices Undergoing 510(k) Review. May 1992. Office of Device Evaluation. MIL-STD-882C. The Computer Glossary. 1994. FDA. Myers. Preproduction Quality Assurance Planning.Procedure for Failure Mode and Effects Analysis [FMEA]. HHS Publication FDA 90-4236. G. International Standard 812. Deluxe Second Edition. Inc.. Military Standard System Safety Program Requirements. 1979. August 1991. McGrawHill. Information Processing. McGraw-Hill Inc. 1993.. International Standard 1025. Pressman. Fifth Edition. Freedman. International Electrotechnical Commission. Sixth Edition. Webster's New Universal Unabridged Dictionary. 1984. Inc. Analysis Techniques for System Reliability .. McGraw-Hill Electronics Dictionary. The Art of Software Testing. McGraw-Hill Dictionary of Scientific & Technical Terms. Science Research Associates. 1994. A. Fourth Edition.. 1990. Fifth Edition. 1979. CDRH.2 . International Electrotechnical Commission. Software Engineering. Third Edition. Second Edition.. Van Nostrand Reinhold. FDA recommendations... M. A Practitioner's Approach. Additional general references used in developing some definitions are: Bohl. Beizer. A-2 Version 6. R. 1992.. Wiley Interscience. B. 19JAN1993. Fault Tree Analysis. McGraw-Hill Inc. American Management Association.

addressing exception. access. A software tool used to perform calculations or determine accuracy of computer manipulated program variables.. American National Standards Institute. (IEEE) (1) A qualitative assessment of correctness or freedom from error. and sizing requirements. ANSI. address. (IEEE) (1) A finite set of well-defined rules for the solution of a problem in a finite number of steps. ALU. (IEEE) Software maintenance performed to make a computer program usable in a changed environment. actuator. or group of characters. accuracy study processor. It is a function of precision and bias. appropriate. or group of characters which identifies a given device or a storage location which may contain a piece of data or a program step. precision. (ISO) The time interval between the instant at which a call for data is initiated and the instant at which the delivery of the data is completed. accuracy. Version 6. (ANSI) To obtain the use of a resource. and usually other characters such as punctuation marks. See: servomechanism. calibration. (2) To refer to a device or storage location by an identifying number. (IEEE) An exception that occurs when a program calculates an address outside the bounds of the storage available to it. a stepper motor which acts on an electrical signal received from a computer instructing it to turn its shaft a certain number of degrees or a certain number of rotations. digits. abstraction. and meet all accuracy. ASCII. Pertaining to a character set that contains letters. information hiding.g. adaptive maintenance. alphanumeric. (IEEE) A software V&V task to ensure that the algorithms selected are correct. See: mishap. access time. (2) Any sequence of operations for performing a specific task. Contrast with precision.Vocabulary -AADC. accident. e. A peripheral [output] device which translates electrical signals into mechanical actions. algorithm analysis. (CDRH) (3) The measure of an instrument's capability to approach a true or absolute value. The separation of the logical properties of data or function from its implementation in a computer program. American Standard Code for Information Interchange. analog-to-digital converter. perfective maintenance. See: bias. software engineering. and stable. arithmetic logic unit. character. (1) A number. (2) A quantitative measure of the magnitude of error. algorithm. Contrast with corrective maintenance. See: encapsulation. timing. character.2 A-3 .

See: analog. A seven bit code adopted as a standard to represent specific data characters in computer systems. digital. American Standard Code for Information Interchange. (1) To separate into elemental parts or basic principles so as to determine the nature of the whole. cartridges and magnetic tapes. See: component. member body to ISO and IEC. analog. architectural design.2 .S. voluntary national standards for nearly all industries. Information technology standards pertain to programming languages. Provides 128 possible characters. (2) The result of the process in (1). application software. See: application software. e. architecture. An organization that coordinates the development of U. A-4 Version 6. the first 32 of which are used for printing and transmission control. (IEEE) A device that operates with variables represented by continuously measured quantities such as pressures. (ISO) An historical copy of a database saved at a significant point in time for use in recovery or restoration of the database. Since common storage is an 8-bit byte [256 possible characters] and ASCII uses only 128. (IEEE) (1) The process of defining a collection of hardware and software components and their interfaces to establish the framework for the development of a computer system. and voltages. telecommunications and physical properties of diskettes. system software. New York. Contrast with support software. It is the U. electronic data interchange. exception. and the separation of the problem into smaller related units for further detailed study. for example. Pertaining to data [signals] in the form of continuously variable [wave form] physical quantities. analysis.Guide to the CASQ CBOK American National Standards Institute. temperatures. Contrast with digital. anomaly. subprogram.g.Y. (IEEE) The organizational structure of a system or component. and to facilitate interchange of data between various machines and systems. See: bug. pressure. See: extended ASCII. software for navigation. voltage. 10036.S. routine. resistances.. the extra bit is used to hold a parity bit or create special symbols. application program. defect. Contrast with DAC [digital-to-analog converter]. archive. (IEEE) A lasting collection of computer system data or other records that are in long term storage. module. See: software engineering. N. payroll. temperature. rotations. analog-to-digital converter. error. (2) A course of reasoning showing that a certain result is a consequence of assumed premises. (IEEE) Anything observed in the documentation or operation of software that deviates from expectations based on previously verified software products or reference documents. 11 West 42nd Street. See: functional design. analog device. archival database. fault. Input related devices which translate an input device's [sensor] analog signals to the corresponding digital signals needed by the computer. (IEEE) Software designed to fill specific needs of a user. rotation. resistance. (3) (ANSI) The methodical investigation of a problem. or process control.

for historical or legal purposes. arithmetic logic unit. (IEEE) A computer program that translates programs [source code files] written in assembly language into their machine language equivalents [object code files]. Tools that test the validity of assertions as the program is executing or tools that perform formal verification of assertions have this feature. cross-compiler. e.2 A-5 . (IEEE) A low level programming language. (ISO) A file that is part of a collection of files set aside for later research or verification. See: cross-assembler.g. arithmetic overflow. (ISO) That portion of a numeric word that expresses the result of an arithmetic operation. and usually results in a one-to-one translation of program instructions [mnemonics] into machine instructions. (1) (IEEE) An independent examination of a work product or set of work products to assess compliance with specifications. A timing independent method of electrical transfer of data in which the sending and receiving units are synchronized on each character. table. assembly language. (ISO) In an arithmetic operation.. asynchronous transmission. See: instrumentation. contractual agreements. See: underflow. See: low-level language. Contrast with compiler. allows symbolic naming of operations and addresses. or vector. The [high speed] circuits within the CPU which are responsible for performing the arithmetic and logical operations of a computer. interpreter. Version 6. or other criteria. See: assembly language. assertion checking. or small block of characters. (NIST) Checking of user. or for backup. a result whose absolute value is too small to be represented within the range of the numeration system in use. An assertion is a logical expression that specifies a condition or relation among program variables.. by which the length of the word exceeds the word length of the space provided for the representation of the number. assertion. audit. (NIST) Pertaining to an actual configuration of software code resulting from a software development project. timing independent. array. overflow exception. (NIST) A logical expression specifying a program state that must exist or a set of conditions that program variables must satisfy at a particular point during program execution. See: assembling. usually by the use of start and stop signals. assembly code. arithmetic underflow. underflow exception. Contrast with synchronous transmission.Vocabulary archive file. that corresponds closely to the instruction set of a given computer. See: overflow. assertion.e. assembler. Occurring without a regular time relationship. i.embedded statements that assert relationships between elements of a program. so that each element of the set is individually addressable. for security purposes. asynchronous. standards. (NIST) Translating a program expressed in an assembly language into object code. assemble. assembling. testing. (IEEE) An n-dimensional ordered set of data items identified by a single name and one or more indices. as built. a matrix.

proceeds to completion without additional input or user interaction. and to recommend any necessary changes. and examination of the sequence of environments and activities surrounding or leading to each event in the path of a transaction from its inception to output of final results. e. (2) (ANSI) To conduct an independent review and examination of system records and activities in order to test the adequacy and effectiveness of data security and data integrity procedures.g. parallel port and serial port. disks and tapes. baseline. -BBIOS.. a 10 MHz band in the 100 to 110 MHz range. screen. An acronym for Beginners All-purpose Symbolic Instruction Code. once started.Guide to the CASQ CBOK See: functional configuration audit. audit trail. and a job. band. (IEEE) Pertaining to a system or mode of operation in which inputs are collected and processed all at one time. (NIST) A specification or product that has been formally reviewed and agreed upon. e. auxiliary storage. It is expressed in cycles per second [Hz]. A-6 Version 6. a high-level programming language intended to facilitate learning to program in an interactive environment.e. software audit. batch. (1) (ISO) Data in the form of a logical path linking a sequence of events. used to trace the transactions that have affected the contents of a record. i. to ensure compliance with established policy and operational procedures. real time. Firmware that activates peripheral devices in a PC. A band can be identified by the difference between its lower and upper limits. It loads the operating system and passes control to it. and for internal services such as time and date. as well as by its actual lower and upper limits. (ISO) A code representing characters by sets of parallel bars of varying thickness and separation that are read optically by transverse scanning. Storage device other than main memory [RAM]. It accepts requests from the device drivers in the operating system as well from application programs. disk. physical configuration audit. and also is often stated in bits or bytes per second. See: computer system audit. BASIC. bar code. bandwidth. basic input/output system. communications line or bus. (2) A chronological record of system activities that is sufficient to enable the reconstruction.2 . Includes routines for the keyboard. bandwidth. rather than being processed as they arrive. Range of frequencies used for transmitting a signal. bits per second.g. on-line. It also contains autostart functions that test the system on startup and prepare the computer for operation.. interactive. See: band. that serves as the basis for further development. Contrast with conversational. and that can be changed only through formal change control procedures. The transmission capacity of a computer channel. reviews. bps. basic input/output system.

Contrast with graph. it may be a physical record. See: accuracy. words or characters in a block. words. However. baud. A measure of the speed of data transfer in a communications system. (ISO) The process. It's the switching speed. A trojan horse which attacks a computer system upon the occurrence of a specific logical event [logic bomb]. the occurrence of a specific time-related logical event [time Version 6. The base two number system. Syn: grouping factor. or segment parts of the program for other purposes. blocking factor. of transferring one or more blocks of data. or characters. Contrast with real time processing. Permissible digits are "0" and "1". It may be thought of as a switch which is either on or off. 300 baud is equal to 300 bps. or number of transitions [voltage or frequency change] made per second. functional. It may be in one of two states. A measure of how closely the mean value in a series of replicate measurements approaches the true value. binary. bit. At low speeds bauds are equal to bits per seconds. (ISO) The part of the error control procedure that is used for determining that a block of data is structured according to given rules. in which the principal parts are represented by suitably annotated geometrical figures to show both the basic functions of the parts and the functional relationships between them..2 A-7 . and the units are separated by interblock gaps. bomb. Execution of programs serially with no interactive processing. bits per second. delineate the applicability of labels. a subdivision of a program that serves to group related statements. block length. one baud can be made to represent more than one bit per second. (NIST) A diagram of a system. black-box testing. A contraction of the term binary digit. a block may be a sequence of statements. block transfer. (4) In programming languages. The bit is the basic unit of digital data. benchmark. computer words. instrument or computer. (3) A group of bits or digits that are transmitted as a unit and that may be encoded for error-control purposes. Bits are usually combined into computer words of various sizes. (1) (ISO) The number of records. In FORTRAN. initiated by a single action. words. bias. delimit routines. such as the byte. specify storage allocation.g. block. usually specified in units such as records. logic 1 or logic 0. An exact or detailed plan or outline. (2) (ANSI) A measure of the size of a block. The signalling rate of a line. block diagram. e. in COBOL. A standard against which measurements or comparisons can be made.Vocabulary batch processing. calibration. See: testing. (ISO) The number of records in a block. (2) A collection of contiguous records that are recorded as a unit. (ISO) (1) A string of records. block check. or characters that for technical or logical purposes are treated as a unity. precision. The number is computed by dividing the size of the block by the size of each record contained therein. blueprint.

(2) To cause a computer system to reach a known beginning state. true or false. (IEEE) A control flow diagram consisting of a rectangle that is subdivided to show sequential steps. branch. (NBS) A selection technique in which test data are chosen to lie along "boundaries" of the input domain [or output range] classes. boot. and 2) a warm boot does not clear all memory. structure chart. (2) A value which lies at. and DIVIDE are the primary operations of arithmetic. boundary value. boundary value analysis. (IEEE) A short computer program that is permanently resident or easily loaded into a computer and whose execution brings a larger program. or output value specified for a system or component. A warm boot means restarting the computer while it is powered-up. or is hidden in electronic mail or data and is triggered when read in a certain way [letter bomb]. Important differences between the two procedures are. See: trojan horse. See: block diagram. branch.Guide to the CASQ CBOK bomb]. a nineteenth century mathematician. See: testing.. Boolean algebra is the study of operations carried out on variables that can have only one of two possible values. branch analysis. See: testing. box diagram. A-8 Version 6. data structures. (1) (IEEE) To initialize a computer system by clearing memory and reloading the operating system. This technique is often called stress testing. rather than execute the next instruction. typically performs this function which includes loading basic instructions which tell the computer how to load programs into memory and how to begin executing those programs. (1) (IEEE) A data value that corresponds to a minimum or maximum input. path coverage. OR. MULTIPLY.2 . statement coverage. In Pascal a boolean variable is a variable that can have one of two possible values. minimum. repetition. Pertaining to the principles of mathematical logic developed by George Boole. boolean. Contrast with condition coverage. Choices often include maximum. in firmware. i. branch coverage. flowchart. (Myers) A test case identification technique which produces enough test cases such that each decision has a true and a false outcome at least once. 1 (true) and 0 (false). virus. internal. graph. Contrast with path analysis. is performed during a cold boot while a warm boot does not normally perform such self-tests. bootstrap. multiple condition coverage. etc. A boot program. procedure parameters. 1) a power-up self-test. input-process-output chart. A distinction can be made between a warm boot and a cold boot. Syn: jump. Nassi-Shneiderman chart. and trivial values or parameters. Syn: decision coverage. worm. boundary value. (NBS) A test coverage criteria which requires that for each decision point each possible branch be executed at least once. if-then-else conditions. Syn: Chapin chart. SUBTRACT.e. program structure diagram. A cold boot means starting the system from a powered-down state. and NOT are the primary operations of Boolean Logic. into memory. and case conditions. An instruction which causes program execution to jump to a new point in the program sequence. AND. in which various portions of the hardware [such as memory] are tested for proper operation. or just inside or just outside a specified range of valid input and output values. bubble chart. such an operating system or its loader. As ADD.

or optical fibers. the CPU. input-process-output chart. usually eight. (IEEE) A data flow. A common pathway along which data and control signals travel between different hardware devices within a computer system. or amounts of data that can be handled by the devices or processes involved in the transfer or use of the data. structure chart. The bus is often divided into two channels. a bus architecture used in medical and industrial equipment due to its small size and rugged design [Originally 8 bits. VMEbus [Versa Module Eurocard Bus]. computer aided software engineering. Multibus I & II [advanced. flowchart. coaxial cable. computer aided design. fault.Vocabulary bubble chart. or other diagram in which entities are depicted with circles [bubbles] and relationships are represented by links drawn between the circles. -CCAD. a DEC 32 bit data bus with peak transfer rates of 100 MB/second. Common buses are: ISA [Industry Standard Architecture] the original IBM PC 16 bit AT bus. CD-ROM. memory and peripheral equipment are interconnected through the bus. bus architecture by Intel used in industrial. Ethernet is a common LAN architecture using a bus topology. CO-AX. complementary metal-oxide semiconductor. CAM. bug. commercial and military applications worldwide [VME64 is an expanded version that provides 64 bit data transfer and addressing]. et. military and aerospace applications]. See: anomaly. STD bus. CMOS. See: block diagram. 16 & 32 bit respectively. defect. time of occurrence of events. exception. complex instruction set computer. Version 6. CCITT. compact disc . TURBO Channel. CISC. computer aided manufacturing. A fault in a program which causes the program to perform in an unintended or unanticipated only memory. with extensions to 16 and 32 bits].2 A-9 . Consultive Committee for International Telephony and Telegraphy. and the other to transfer the data [data bus or I/O bus]. CASE. graph. a control channel to select where data is located [address bus]. bus. box diagram. data structure. coaxial cable.. error. a 32 bit bus from Motorola. all terminals and computers are connected to a common channel that is made of twisted wire pairs. NuBus. (B) When bus architecture is used in a network. EISA [Extended Industry Standard Architecture] the IBM PC 32 bit XT bus [which provides for bus mastering]. A device or storage area [memory] used to store data temporarily to compensate for differences in rates of data flow. (A) When bus architecture is used in a computer. A sequence of adjacent bits. used in industrial. operated on as a unit. buffer. a 32 bit bus architecture originally developed at MIT [A version is used in the Apple Macintosh computer]. MCA [MicroChannel Architecture] an IBM 32 bit bus.

functional. screen. (1) (NBS) Test data selection technique. a technical evaluation. retrieves data and programs from memory. The input and output domains are partitioned into classes and analysis is performed to determine which input classes cause which effect. bias. calibration. central processing unit. (IEEE) A diagram that identifies the modules in a system or computer program and shows which modules call one another. See: testing. It strives to combine the power of assembly language with the ease of a high-level language. central processing unit.Guide to the CASQ CBOK COTS. data flow diagram. that establishes the extent to which a particular computer system or network design and implementation meet a prespecified set of requirements. A technique for error detection to ensure that data or program files have been accurately copied or transferred. CP/M. The processes. The graph is actually a digital-logic circuit (a combinatorial logic network) using a simpler notation than standard electronics notation.2 . check summation. (2) (Myers) A systematic method of generating test cases representing combinations of conditions. measurement. change tracker. tier chart. cyclic redundancy [check] code. cause effect graph. An object-oriented high-level programming language. CRT. Basically. authorities for. See: configuration control. monitor. cause effect graphing. made as part of and in support of the accreditation process. and procedures to be used for all changes that are made to the computerized system and/or the system's data. Syn: call tree. certification. state diagram. See: accuracy. It receives and sends data through input-output channels. call graph. (Myers) A Boolean graph linking causes and effects. The CPU controls the entire computer. A software tool which documents all changes made to a program. CRC. CPU. and actuating equipment with regard to specified accuracy and precision requirements. data structure diagram. cathode ray tube. Ensuring continuous adequate performance of sensing. See: control flow diagram. Created for use in the development of computer operating systems software. The unit of a computer that includes the circuits controlling the interpretation of program instructions and their execution. C++. An output device. Note: The result is not necessarily the same as that shown in a structure chart. Syn: display. precision. a redundant check in which groups of digits. cathode ray tube. (ANSI) In computer systems. Change control is a vital subset of the Quality Assurance [QA] program within an establishment and should be clearly described in the establishment's SOPs. and conducts mathematical and logical functions of a program. configurable. A-10 Version 6. A minimal set of inputs is chosen which will cover the entire effect set. Contrast with structure chart. Control Program for Microcomputers. off-the-shelf software. A general purpose high-level programming language. change control. C.

code auditor. However. (IEEE) A sum obtained by adding the digits in a numeral. (ISO) A device that generates periodic. are summed. or group of numerals [a file]. or generation of interrupts. (IEEE) A meeting at which software code is presented to project personnel. or other interested parties for comment or approval. code walkthrough. In the world of microcomputers. as the client. statement by statement. client-server systems are inherently more complex than file server systems. the workstation is the client and the file server is the server. Contrast with cyclic redundancy check [CRC]. See: program. code review. or tool to verify compliance with software design documentation and programming standards. accurately spaced signals used for such purposes as timing. Two disparate programs must work in tandem. or significance. In a local area network [LAN]. and that sum checked against a previously computed sum to verify operation accuracy.. chip. the term client-server describes a networked system where front-end applications. managers. parity check. team. make service requests upon another networked system. A high-level programming language intended for use in the solution of problems in business data processing. This technique can also be applied to other software and configuration items.g. Contrast with code audit. code review. coaxial cable. position. See: check summation. See: checksum. See: static analysis. code inspection. Correctness and efficiency may also be evaluated. analyzing the code with respect to a checklist of historically common programming errors. (IEEE) An independent review of source code by a person. See: integrated circuit. enforces security. to analyze the Version 6. customers. code. users. A term used in a broad sense to describe the relationship between the receiver and the provider of a service. usually without regard to meaning.2 A-11 . while the state of program variables is manually monitored. code audit. code walkthrough. and provides applications with a consistent interface to data via a data dictionary. See: static analysis. (Myers/NBS) A manual [formal] testing [error detection] technique where the programmer reads source code. code review. Acronym for COmmon Business Oriented Language. and analyzing its compliance with coding standards. High-capacity cable used in communications and video transmissions. (Myers/NBS) A manual testing [error detection] technique where program [source code] logic [structure] is traced manually [mentally] by a group with a small set of test cases. code walkthrough. client-server. clock. Syn: Fagan Inspection. COBOL. Contrast with code audit. See: static analysis. code inspection. source code. A software tool which examines source code for adherence to coding and documentation conventions. code walkthrough. to a group who ask questions analyzing the program logic. a file. restricts access. Contrast with code inspection.Vocabulary e. usually without regard to overflow. checksum. and there are many more decisions to make about separating data and processing between the client workstations and the database server. Client-server relationships are defined primarily by software. regulation of the operations of a processor. The database server encapsulates database files and indexes. Provides a much higher bandwidth than twisted wire pair.

compile. Typical objects of comparison are similar versions of source code. See: assembler. (IEEE) (1) In software engineering. files. A-12 Version 6. data base files. job control statements. (2) The compiler takes the finished source code listing as input and outputs the machine code instructions that the computer must have to execute the program. (2) (IEEE) Information embedded within a computer program. control complexity and promote understandability of the source code. (IEEE) A software tool that compares two computer programs.Guide to the CASQ CBOK programmer's logic and assumptions. Contrast with assembling. programming standards. code review. comparator. compiling. (2) The transforming of logic and data from design specifications (design descriptions) into a programming language. (1) (ISO) In programming languages. object code. or 20. compiler. cross-assembler. See: static analysis. write-once read-many times disk. See: traceability analysis. comment. the process of expressing a computer program in a programming language. A type of integrated circuit widely used for processors and memories. A compact disk used for the permanent storage of text. compatibility. or test results. interpreter. This storage media is often used for archival purposes. See: implementation. formatting. and naming. compact disc . or sets of data to identify commonalities or differences. Written procedures describing coding [programming] style conventions specifying rules governing the use of individual constructs provided by the programming language. See: compilation. Syn: optical disk.000 pages of text. (NIST) The property that all necessary parts of the entity are included. and documentation requirements which prevent programming errors. that provides clarification to human readers but does not affect machine interpretation. It is a combination of transistors on a single chip connected to complementary digital circuits. complementary metal-oxide semiconductor. (NIST) Translating a program expressed in a problem-oriented language or a procedure oriented language into object code. coding. See: compilation. compilation. Capable of storing up to 680 MB of data. Completeness of a product is often used to express the fact that all requirements have been met by the product. code inspection. a language construct that allows [explanatory] text to be inserted into a program and that does not have any effect on the execution of the program.2 . See: compiler. interpret. Syn: development standards. or a set of data. equivalent to 250. cross-compiler.000 medium resolution images. Contrast with code audit. Digital data is represented very compactly by tiny holes that can be read by lasers attached to high resolution sensors. coding standards. (1) (IEEE) A computer program that translates programs expressed in a high-level language into their machine language equivalents. (ANSI) The capability of a functional unit to meet the requirements of a specified interface. graphic or sound only memory. completeness.

CAD systems are high speed workstations or personal computers using CAD software and input devices such as graphic tablets and scanners to model and simulate the use of proposed products. (2) A functional programmable unit that consists of one or more associated processing units and peripheral equipment. Traditional computer architecture that operates with large sets of possible instructions. computer program. coding. CAD software is available for generic design or specialized uses such as architectural. (ISO) The branch of science and technology that is concerned with methods and techniques relating to data processing performed by automatic means. CAD software may also be highly specialized for creating products such as printed circuits and integrated circuits. computer language. computer science. complexity. electrical. component. computer instruction set. Computers which operate with system software based on these instruction sets have been referred to as complex instruction set computers. and that can perform substantial computations. process control. or logic operations. programs. The use of computers to design products. computer aided design. (2) Pertaining to any of a set of structure based metrics that measure the attribute in (1). instruction sets expanded to include newer instructions which are complex in nature and require several to many execution cycles and. without human intervention during a run. (IEEE) (1) A functional unit that can perform substantial computations. Syn: machine instruction set. including numerous arithmetic operations. See: program. or logic operations. including numerous arithmetic operations. Version 6. therefore. See: unit. computer. testing and maintenance. which facilitate the accomplishment of software engineering methods and tasks such as project planning and estimation. computer aided software engineering. and mechanical design.e.2 A-13 . As computing technology evolved. without human intervention. (IEEE) (1) The degree to which a system or component has a design or implementation that is difficult to understand and verify. i. including the use of computers to communicate work instructions to automate machinery for the handling of the processing [numerical control. See: programming language. that is controlled by internally stored programs. program architecture and algorithm procedure. An automated system for the support of software development including an integrated tool set. (IEEE) A language designed to enable humans to communicate with computers.. Contrast with reduced instruction set computer [RISC]. Most computers are in this category. The automation of manufacturing systems and techniques. system and software requirements analysis. robotics.Vocabulary complex instruction set computer. more time to complete. material requirements planning] needed to produce a workpiece. including the IBM compatible microcomputers. design of data structure. computer aided manufacturing. (ANSI) A complete set of the operators of the instructions of a computer together with a description of the types of meanings that can be attributed to their operands. CAD output is a printed design or electronic output to CAM systems.

sometimes general purpose. computer word. Application software. software. procedures. configurable. (IEEE) The initial phase of a software development project. See: software audit. statement coverage. including arithmetic operations and logic operations. and each point of entry to a program or subroutine is invoked at least once. concept phase. performs userdesignated data manipulation. that uses common storage for all or part of a program and also for all or part of the data necessary for the execution of the program.g. consisting of one or more computers and associated peripheral input and output devices. computer system. written for a variety of industries or users in a manner that permits users to modify the program to meet their individual needs. (ISO) An examination of the procedures used in a computer system to evaluate their effectiveness and correctness and to recommend improvements. Security also pertains to personnel. multiple condition coverage. condition coverage. (Myers) A test coverage criteria requiring enough test cases such that each condition in a decision takes on all possible outcomes at least once. advance planning report. nature. (ANSI) a functional unit. Typically one to four bytes long.. Contrast with branch coverage. feasibility studies. path coverage. regulations. and documentation. A computer system may be a stand-alone unit or may consist of several interconnected units. destruction. A-14 Version 6. and implementation of changes to configuration items after formal establishment of their configuration identification. and operated on as a unit within a given computer. virus. computer system audit.2 . (2) In configuration management. and associated software. the functional and physical characteristics of hardware or software as set forth in technical documentation or achieved in a product. transmitted. addressed. or policies relevant to the project. See: bomb. personnel. computerized system. worm. decision coverage. depending on the make of computer. See: change control. consisting of the evaluation.g. approval or disapproval. and interconnections of its constituent parts. configuration. or disclosure. in which user needs are described and evaluated through documentation. See: functional configuration audit. peripheral devices. trojan horse. and that can execute programs that modify themselves during their execution. computer system security. system definition documentation. statement of needs. configuration audit. See: computer. data. configuration control. coordination. communications. and the physical protection of computer installations.Guide to the CASQ CBOK computer system. executes user-written or user-designated programs. project initiation memo. A sequence of bits or characters that is stored. modification. (IEEE) An element of configuration management. manuals and Standard Operating Procedures. use.. computerized system. (IEEE) (1) The arrangement of a computer system or component as defined by the number. (IEEE) The protection of computer hardware and software from accidental or malicious access. Includes hardware. See: computer. e. physical configuration audit. e. off-the-shelf software.

An operating system. software. Consultive Committee for International Telephony and Telegraphy. and to ensure that all interfaces have been considered for out-of-sequence and erroneous inputs. the design.Vocabulary configuration identification. and human reaction time. consistency. Contrast with variable. structure chart. software engineering. See: traceability. Version 6. consistency checker. change control. (ISO) In programming languages. (IEEE) (1) Evaluation of the safety of restrictions imposed on the selected design by the requirements and by real world restrictions. state diagram. constant. constraint analysis. control flow. Control Program for Microcomputers. A software tool used to test requirements in design specifications for both consistency and completeness. (ANSI) A bus carrying the signals that regulate system operations. standardization. and freedom from contradiction among the documents or parts of a system or component. flowchart. (IEEE) A discipline applying technical and administrative direction and surveillance to identify and document the functional and physical characteristics of a configuration item. and the target computer. the timing of a bus latch when using the longest safety-related timing to fetch data from the most remote circuit card. See: bus. Constraint analysis is designed to identify these limitations to ensure that the program operates within them. The impacts of the environment on this analysis can include such items as the location and relation of clocks to circuit cards. A registered trademark of Digital Research. control flow diagram. control flow analysis. control changes to those characteristics. input-process-output chart. interrupts going unsatisfied due to a data flood at an input. consisting of selecting the configuration items for a system and recording their functional and physical characteristics in technical documentation. See: International Telecommunications Union . (IEEE) An element of configuration management. configuration management. (IEEE) A diagram that depicts the set of all possible sequences in which operations may be performed during the execution of a system or program. an abstraction of all possible paths that an execution sequence may take through a program. See: configuration control. (IEEE) An aggregation of hardware. See: software element. (IEEE) A software V&V task to ensure that the proposed control flow is free of problems.Telecommunications Standards Section. such as design or code elements that are unreachable or incorrect. Types include box diagram. or both that is designated for configuration management and treated as a single entity in the configuration management process. record and report change processing and implementation status. See: call graph. (2) verification that the program operates within the constraints imposed upon it by requirements. and verifying compliance with specified requirements. A value that does not change during processing.2 A-15 . configuration item. Contrast with data flow diagram. control bus. (IEEE) The degree of uniformity.

(IEEE) An assembler that executes on one computer but generates object code for a different computer. Contrast with batch. Hardware that controls peripheral devices such as a disk or display screen. on-line. error. may have an adverse affect on the quality of the finished product and may result in a unacceptable health risk. and evaluate the adequacy of preliminary operation and support documents. (IEEE) The sudden and complete failure of a computer system or component. (IEEE) The degree to which software is free from faults in its specification. (QA) A function or an area in a manufacturing process or procedure. A-16 Version 6. coroutine. (IEEE) Analysis which identifies all software requirements that have safety implications. (IEEE) A compiler that executes on one computer but generates assembly code or object code for a different computer. (IEEE) A routine that begins execution at the point at which operation was last suspended.2 . testing. cross-compiler. See: interactive. See: preliminary design review. documentation and other items meet user needs and expectations. critical control point. and assigns a criticality level to each safety-critical requirement based upon the estimated risk. coverage analysis. Coverage analysis is useful when attempting to execute each statement. branch. fault. and personnel. Syn: severity. (IEEE) Maintenance performed to correct faults in hardware or software. It performs the physical data transfers between main memory and the peripheral device. to establish the compatibility among the configuration items and other items of equipment. and. documentation and other items meet specified requirements. failure. branch. facilities. Tools that capture this data and provide reports summarizing relevant information have this feature.Guide to the CASQ CBOK controller. software. evaluate preliminary test planning. design and coding. or loss of control over. criticality. criticality analysis. Contrast with subroutine. (IEEE) The degree of impact that a requirement. real time. critical design review. cross-assembler. See: testing. module. The degree to which software. path. (IEEE) Pertaining to a interactive system or mode of operation in which the interaction between the user and the system resembles a human dialog. perfective maintenance. whether specified or not. the failure of which. and that is not required to return control to the program or subprogram that called it. (NIST) Determining and assessing measures associated with the invocation of program structural elements to determine the adequacy of a test run. conversational. Contrast with adaptive maintenance. (IEEE) A review conducted to verify that the detailed design of one or more configuration items satisfy specified requirements. crash. as applicable. or iterative structure in a program. corrective maintenance. correctness. or other item has on the development or operation of a system. to assess the results of producibility analyses. testing. statement. review preliminary hardware product specifications. The degree to which software. to assess risk areas for each configuration item. system design review. path.

data flow diagram. (1) (McCabe) The number of independent paths through a program. (IEEE) An exception that occurs when a program attempts to use or access data incorrectly. data bus. digital-to-analog converter. See: bus. disk operating system. or instructions in a manner suitable for communication. cyclic redundancy [check] code. (ANSI) A movable. (IEEE) (1) A collection of the names of all data items used in a software system. Representations of facts. cyclomatic complexity. data exception. The calculations are chosen to optimize error detection. -DDAC. DOS. data bases. partitioning. data elements. (2) A set of definitions of data flows. length of data item. interpretation. visible mark used to indicate a position of interest on a display surface. (2) A named identifier of each of the entities and their attributes that are represented in a database. (2) (NBS) The cyclomatic complexity of a program is equivalent to the number of decision statements plus 1. (1) (ISO) A named unit of data that. direct memory access. (ISO) A violation of data integrity. together with relevant properties of those items. (IEEE) (1) Evaluation of the description and intended use of each data item in the software design to ensure the structure and intended use will not result in a hazard. The CRC is the result of a calculation on the set of transmitted bits by the transmitter which is appended to the data. and fault containment issues affecting safety. DFD. At the receiver the calculation is repeated and the results compared to the encoded value.Vocabulary cursor. data aliasing.g. concepts. e. data. data analysis..2 A-17 . DMA. Usually performed in conjunction with logic analysis. Data structures are assessed for data dependencies that circumvent isolation. parity check. data corruption. Syn: data contamination. A technique for error detection in data communications used to assure a program or data file has been accurately transferred. representation. and the control or mitigation of hazards. (ANSI) A bus used to communicate data internally and externally to and from a processing unit or a storage device. Contrast with check summation. or processing by humans or by automated means. etc. data element. files. and processes referred to in a leveled data flow diagram set. Version 6. data dictionary. in some contexts. is considered indivisible and in other contexts may consist of data items. (2) Evaluation of the data structure and usage in the code to ensure each is defined and used properly by the program.

Syn: data quality. and that each statement is executed at least A-18 Version 6. and fixing the error. and that the data flows are correct. incomplete. (ANSI) A collection of interrelated data. dead code. data storage. and processes performed on data as nodes. (IEEE) A diagram that depicts a set of data elements. Program code statements which can never execute during program operation. Contrast with data flow diagram. rules. (Myers) A test coverage criteria requiring enough test cases such that each decision has a true and false result at least once. database analysis.Guide to the CASQ CBOK data flow analysis. The data are stored so that they can be used by different programs without concern for the data structure or organization. data structure centered design. (IEEE) The degree to which a collection of data is complete. (2) The checking of data for correctness or compliance with applicable standards. See: archival database. or can be an artifact of previous versions or debugging efforts. designed to support specific data manipulation functions. decision coverage. database security. (IEEE) A physical or logical relationship among data elements. and the logical relationships among them. and accurate. data integrity. data flow diagram. (IEEE) A software V&V task to ensure that the database structure and access methods are compatible with the logical design. and conventions. Such code can result from poor coding style.2 . Dead code can be confusing. often with controlled redundancy. and logical flow of data as links between the nodes. data flow graph. (Myers) Determining the exact nature and location of a program error. or unreasonable. See: entity-relationship diagram. data validation. (ANSI) A named component of a data element. check key tests. Syn: file. data sink. (IEEE) A software V&V task to ensure that the input and output data and their formats are properly defined. A common approach is used to add new data and to modify and retrieve existing data. data sinks. Usually the smallest component. data structure diagram. The degree to which a database is protected from exposure to accidental or malicious alteration or destruction. A structured software design technique wherein the architecture of a system is derived from analysis of the structure of the data sets with which the system must deal. consistent. (IEEE) A diagram that depicts data sources. and is a potential source of erroneous software changes. database. data set. organized according to a schema to serve one or more applications. (1) (ISO) A process used to determine if data are inaccurate. their attributes. (IEEE) The equipment which accepts data signals after transmission. The process may include format checks. Syn: data flowchart. data structure. debugging. completeness checks. See: infeasible path. reasonableness checks and limit checks. data item. A collection of related records.

interfaces. design phase. interface descriptions and algorithms. Converting signals from a wave form [analog] to pulse form [digital]. (IEEE) A process or meeting during which a system. See: software design description. design level. system design review. components. exception. Syn: separator. hardware. subsystem. data structures. Contrast with specification. design review. (IEEE) A requirement that specifies or constrains the design of a system or system component. or software design is presented to project personnel. interfaces. detailed design. design. defect. demodulate. software components. path coverage. delimiter. system. See: architectural design. control logic. (IEEE) A table used to show sets of conditions and the actions resulting from them. error. managers. documented. (IEEE) The design decomposition of the software item. decision table. Version 6. A standard setting or state to be taken by the program if no alternate setting or state is initiated by the system or the user. See: failure analysis. preliminary design review. customers. (IEEE) The process of defining the architecture. demodulation. design description. (IEEE) The period of time in the software life cycle during which the designs for architecture. (ANSI) Pertaining to an attribute. default. and other characteristics of a system or component. requirements. or option that is assumed when none is explicitly specified. and data are created. default value. Typical contents include system or component architecture. A methodology for planning experiments so that data appropriate for [statistical] analysis will be collected. bug. A facet of reliability that relates to the degree of certainty that a system or component will operate correctly. value. users. e. the reverse of modulate. program or module. preliminary design. and verified to satisfy requirements..Vocabulary once. statement coverage. Syn: design document. defect analysis. Contrast with modulation. or other interested parties for comment or approval. dependability. multiple condition coverage.g. input/output formats. Syn: branch coverage. design requirement. Retrieve the information content from a modulated carrier wave. Types include critical design review.2 A-19 . fault. Contrast with modulate. (IEEE) A document that describes the design of a system or component. A value assigned automatically if one is not given by the user. See: anomaly. Contrast with condition coverage. (ANSI) A character used to indicate the beginning or the end of a character string. data flow. design of experiments.

a diagnostic manual. Disks can be hard [fixed] or flexible [removable] and different sizes. OS/2 from IBM. diagnostic. An operating system program. e. disk drive. Circular rotating magnetic storage hardware. Pertaining to data [signals] in the form of discrete [separate/pulse form] integral values. (IEEE) Pertaining to the detection and isolation of faults or failures. A person. See: testing.Guide to the CASQ CBOK design specification. (ANSI) A systematic approach to software creation that defines development phases and specifies the activities. System-7 from Apple. waterfall model. Performed when more than one software system is being integrated. spiral model. digital-to-analog converter. Output related devices which translate a computer's digital outputs to the corresponding analog signals needed by an output device such as an actuator. (IEEE) A graph in which direction is implied in the internode connections. inspection. Contrast with ADC [Analog-to-Digital Converter]. direct memory access. design standards. PC-DOS from IBM.. disk operating system. (IEEE) (1) The process of refining and expanding the preliminary design of a system or component to the extent that the design is sufficiently complete to be implemented. and completion criteria for each phase. design. review and walkthrough techniques to source code and other software documents usually by an individual [often by the person who generated them] and usually done informally. products. Syn: digraph. a diagnostic message. diskette. A floppy [flexible] disk. development methodology. that designs and/or builds and/or documents and/or configures the hardware and/or software of computerized systems. digital. MS-DOS from Microsoft Corp. The application of code audit. rapid prototyping. disk. Contrast with analog. Specialized circuitry or a dedicated microprocessor that transfers data from memory to memory without using the CPU. See: software development process. or group. desk checking. directed graph. (IEEE) Standards that describe the characteristics of a design or a design description of data or program components. detailed design. See: specification. verification procedures. DR-DOS from Digital Research. compatibility. Syn: coding standards. different software system analysis. developer.. A-20 Version 6.g.2 . development standards. (IEEE) Analysis of the allocation of software requirements to separate computer systems to reduce integration and interface errors related to safety. See: incremental development. (2) The result of the process in (1). Hardware used to read from or write to a disk or diskette. For example.

intended usage. ESD. that describe or specify the design or details. Contrast with static analysis.g. deleting. This means this IC does not necessarily have to be removed from the circuit in which it is mounted in order to erase and reprogram the memory. drift.2 A-21 . software. Contrast with test driver. and providing for activation of all device functions. Selection of the level may be based on project cost. The plan typically describes what documentation types are to be prepared. and user manuals. test plan. content. design. enhanced small device interface.Vocabulary documentation. dynamic analysis. (NIST) Modifying the content of the input by inserting. (ISO) The unwanted change of the value of an output signal of a device over a period of time when the values of all input signals to the device are kept constant. level of. specification. (NBS) Analysis that is performed by executing the program code. what their contents are to be. such as flowcharts. electrostatic discharge. (ANSI) The aids provided for the understanding of the structure and intended uses of an information system or its components. Chips which may be programmed and erased numerous times like an EPROM. and what are the available resources and external factors affecting the results. EEPROM. However an EEPROM is erased electrically. EPROM. test report. documentation. format. user's guide. when this is to be done and by whom. and quality. how it is to be done. specification. explain the capabilities. EMI. editing. Syn: device driver. -EEBCDIC. software design description. See: specification. erasable programmable read only memory. textual material. level of concern. Version 6. duplex transmission. (NIST) A description of required documentation indicating its scope. or data. electrically erasable programmable read only memory. documentation plan. or moving characters. in human readable form. A program that links a peripheral device or internal function to the operating system. numbers.. including computer listings and printouts. extent of effort. electrically erasable programmable read only memory. extended binary coded decimal interchange code. documentation. or provide operating instructions for using the software to obtain desired results from a software system. e. driver. (NIST) Technical data or information. or other factors. requirements. electromagnetic interference. ESDI. (NIST) A management document describing the approach to a documentation effort. (ISO) Data transmission in both directions at the same time. See: testing.

g. e. Low frequency electromagnetic waves that emanate from electromechanical devices.g. software used in an aircraft or rapid transit system. A device which has its own computing power dedicated to specific functions. plastic or other non-conductors and can be discharged by human skin. e. See: abstraction. entity relationship diagram. (IEEE) A device. device. from a nonconductive surface to an approaching conductive object that can damage or destroy semiconductors and other circuit components. computer program. (2) A person whose occupation requires the use of an information system but does not require any knowledge of computers or computer programming. SCSI. end user. (ANSI) (1) A person. integrated circuit. emulation. electronic media. embedded computer. contact. stand-alone computer. software engineering. magnetic disk. damage components and cause malfunctions. An electromagnetic disturbance caused by such radiating and transmitting sources as heavy duty motors and power lines can induce unwanted voltages in electronic circuits. Static electricity can build on paper. Hardware intended to store binary data. It can also be generated by scuffing shoes on a carpet or by brushing a non-conductor. See: firmware. magnetic tape. The movement of static electricity. The computer becomes an integral part of the device as opposed to devices which are controlled by an independent. e. See: data structure diagram. information hiding. sparks. (2) The conditions that affect the performance of a system or function. usually consisting of a microprocessor and firmware. encapsulation.Guide to the CASQ CBOK electromagnetic interference. MOSFETs and CMOS logic ICs are especially vulnerable because it causes internal local heating that melts or fractures the dielectric silicon oxide that insulates gates from other internal structures. See: user. emulator. (ANSI) (1) Everything that supports a system or the performance of a function. enhanced small device interface. It implies software that integrates operating system and application functions. or computer system that uses an information system for the purpose of data processing in information exchange. or system that accepts the same inputs and produces the same outputs as a given system. finger. Contrast with simulator. A standard interface for hard disks introduced in 1983 which provides for faster data transfer compared to ST-506.. program. IDE. embedded software. (IEEE) A model that accepts the same inputs and produces the same outputs as a given system. (IEEE) Software that is part of a larger system and performs some of the requirements of that system. To imitate one system with another. Contrast with ST-506.. A-22 Version 6. Such software does not provide an interface with the user. (IEEE) A software development technique that consists of isolating a system function or a set of data and the operations on those data within a module and providing precise specifications for the module. See: radiofrequency interference. environment. electrostatic discharge.g.g. Contrast with simulation. (IEEE) A diagram that depicts a set of real-world entities and the logical relationships among them.2 . e.

Types include addressing exception. and estimating the number of faults remaining in the program. Syn: code trace. EPROMs may be erased and reprogrammed because the electrical charge at the bit locations can be bled off [i. error guessing. or theoretically correct value or condition. specified. (IEEE) An event that causes suspension of normal program execution. There are two types of input equivalence classes. subroutine trace. error. overflow exception. A special type of event table. A table which lists events and the corresponding specified effect[s] of or reaction[s] to each event. (ISO) A discrepancy between a computed. the IC's window must be covered to prevent exposure to UV light until it is desired to reprogram the chip. reset to the default state] by exposure to ultraviolet light through the small quartz window on top of the IC. defect. either 1 or 0. See: spiral model. longitudinal redundancy. Types include addressing exception. symbolic trace. Chips which may be programmed by using a PROM programming device. data exception. data exception. Contrast with mutation analysis. Techniques used to identify errors in data transfers. or measured value or condition and the true. erasable programmable read only memory. exception. execution trace. See: special test data. exception conditions/responses table. control flow trace. parity check.e. overflow exception. special case. bug. The selection criterion is to pick values that seem likely to cause errors. (NBS) Test data selection technique. Before programming each bit is set to the same logical state. fault.Vocabulary equivalence class partitioning. testing. operation exception. See: debugging. to identify a minimal set of well selected test cases to represent these classes. An EPROM eraser is a device for exposing the IC's circuits to UV light of a specific wavelength for a certain amount of time. See: anomaly. Each bit location may be thought of as a small capacitor capable of storing an electrical charge. protection exception. See: check summation. operation exception. all bits whose states are to be changed from the default state. protection exception. Often takes the form of a list of code labels encountered as the program executes. bug. defect.2 A-23 . See: anomaly. (Myers) Partitioning the input domain of a program into a finite number of classes [sets]. observed. via an electrical current. cyclic redundancy check [CRC]. error analysis. After programming. event table. error detection. See: retrospective trace. (IEEE) The process of intentionally adding known faults to those already in a computer program for the purpose of monitoring the rate of detection and removal. (IEEE) A record of the sequence of instructions executed during the execution of a computer program. valid and invalid. fault. (IEEE) An event that causes suspension of normal program operation. functional. error. failure analysis. The logical state is established by charging. Version 6. exception. underflow exception. error seeding. variable trace. evolutionary development. underflow exception. See: testing. exception.

failure analysis. economy. Failure Modes and Effects Criticality Analysis. FMEA. exception. (IEC) A logical extension of FMEA which analyzes the severity of the consequences of failure. fail-safe. (IEEE) The inability of a system or component to perform its required functions within specified performance requirements. See: testing. Failure Modes and Effects Analysis. See: error seeding. Failure Modes and Effects Analysis. FMECA. The second half of the ACSII character set. FIPS. usually one which significantly affects system performance. crash. Failure Modes and Effects Criticality Analysis. 128 thru 255. fault. Fault Tree Analysis. or data definition in a computer program which causes the program to perform in an unintended or unanticipated manner. bug. safety or other required characteristics. See: code inspection. See: bug. An eight bit code used to represent specific data characters in some computers. It is non-standard ASCII. An incorrect step. exception. IBM mainframe computers. Federal Information Processing Standards. fault seeding. which have significant consequences affecting the system performance in the application considered. -FFDD. (NBS) Test data that is at the extreme or boundary of the domain of an input variable or which produces results at the boundary of an output domain. A-24 Version 6. extended binary coded decimal interchange code. (IEC) A method of reliability analysis intended to identify failures. defect. failure. at the basic component level. to identify and fix other similar errors. error. fault.Guide to the CASQ CBOK extended ASCII. floppy disk drive. file transfer protocol. The symbols are defined by IBM for the PC and by other vendors for proprietary use.2 . See: ASCII. boundary value. process. FTP. (IEC) The identification and analysis of conditions and factors which cause or contribute to the occurrence of a defined undesirable event. Fagan inspection. and to initiate corrective action to prevent future occurrences of this type of error. extremal test data. Contrast with debugging. Fault Tree Analysis. Determining the exact nature and location of a program error in order to fix the error..g. (IEEE) A system or component that automatically places itself in a safe operational mode in the event of a failure. See: anomaly. FTA. e.

flat file. changing. floppy disk drive. flow direction. Such software cannot be modified by the computer during processing. Any relationship between two flat files is logical. system. See: optical fiber. or analytical process by which a group of configuration items comprising a system is verified to have met specific contractual Version 6. See: block diagram. (4) A discrete location in a database that contains an unique piece of information. See: Kermit. for the definition. (IEEE) A variable that is set to a prescribed state. or deleting data. a data aggregate. file maintenance. often "true" or "false".g. Syn: data set. in stock control. See: TCP/IP. fiber optics. (2) TCP/IP protocol that is used to log onto the network. field. data. (1) Communications protocol that can transmit binary and ASCII data files without loss of data. (2) (ISO) A graphical representation in which symbols are used to represent such things as operations. formerly National Bureau of Standards. e. input-process-output chart.. flag. Standards published by U..g. See: embedded software. Analysis of the known or anticipated need for a product. A data file that does not physically interconnect with or point to other files. Communications systems that use optical fibers for transmission. A record is a component of a database.Vocabulary feasibility study. and copy files. bubble chart. Federal Information Processing Standards. or component to assess the degree to which the requirements. formal qualification review.. National Institute of Standards and Technology. file transfer protocol. flowchart or flow diagram.. based on the results of a process or the occurrence of a specified condition. and arrows are used to indicate the sequential flow from one to another.2 A-25 . or a link. These standards are intended to be binding only upon federal agencies. list directories. Zmodem. graph. and equipment. a file could consists of a set of invoices. a group of character positions used to enter or display wage rates on a screen. (1) (ISO) A set of related records treated as a unit. See: diskette. (2) (IEEE) A control flow diagram in which suitably annotated geometrical figures are used to represent operations.g. (IEEE) The combination of a hardware device. designs. Ymodem. box diagram. Department of Commerce. analysis. e. data. floppy disk. (ANSI) The activity of keeping a file up to date by adding. It can also translate between ASCII and EBCDIC. structure chart. or equipment. (3) The elementary unit of a record that may contain a data item. or solution of a problem. (2) Defined logical data that is part of a record. e. matching account numbers. a specified area used for a particular class of data. (1) (ISO) On a data medium or in storage. (2) The largest unit of storage structure that consists of a named collection of all occurrences in a database of records of a particular record type. or plans can be implemented. (IEEE) The test. A field is a component of a record. file. firmware.S. a pointer. See: disk.g. and computer instructions and data that reside as read only software on that device. disk drive. inspection. e. Syn: flow diagram. Xmodem. Syn: indicator. an IC.

test readiness review. graphic software specifications. See: physical configuration audit. directed graph. functional configuration audit. decision. gigabyte. states of data. that the item has achieved the performance and functional characteristics specified in the functional or allocated configuration identification. call graph. graphs which depict program structure. box diagram. functional analysis. the first widely used high-level programming language. and cause-effect relationships. structure chart. engineering.073. (IEEE) (1) The process of defining the working relationships among the components of a system. Contrast with blueprint. A-26 Version 6. See: block diagram. control. functional decomposition. See: architectural design. and that its operational and support documents are complete and satisfactory. Intended primarily for use in solving technical problems in mathematics.Guide to the CASQ CBOK performance requirements. the value of the dependent variable. requirements review. or its characteristic action. design review. bubble chart. See: kilobyte. flowchart. gigabyte. module interface. cause-effect graph. (IEEE) A diagram or other representation consisting of a finite set of nodes and internode connections called edges or arcs. See: duplex transmission. FORTRAN. function. diagrams. -GGB. precisely 230 or 1. transaction flowgraph. See: modular decomposition. (IEEE) Verifies that each safety-critical software requirement is covered and that an appropriate criticality level is assigned to each software element. and science.824 bytes. Approximately one billion bytes. namely. full duplex. exception conditions/responses necessary to establish design integrity. control flow diagram. event. functional requirement. (3) In data communication. input-process-output chart. Contrast with code review. graph. (IEEE) An audit conducted to verify that the development of a configuration item has been completed satisfactorily. a machine action such as carriage return or line feed. Documents such as charts.741. depends in a specified manner on the values of one or more independent variables. (2) A specific purpose of an entity. An acronym for FORmula TRANslator. transaction flow. megabyte. functional design. HIPO. (2) The result of the process in (1). data flow diagram. and tables including truth. state-transition. with not more than one value of the dependent variable corresponding to each permissible combination of values from the respective ranges of the independent variables. (1) (ISO) A mathematical entity whose value.2 . (IEEE) A requirement that specifies a function that a system or system component must be able to perform.

decimal 15 is 1111 in binary and F in hexadecimal. rules. 5. See: disk. D. 2. The base 16 number system.2 A-27 . 9. hierarchy of input-processing-output. hazard severity. can be translated into several different machine languages. A technique used to identify conceivable failures affecting system performance. e. Contrast with software. A unit of frequency equal to one cycle per second. hard drive. such as data transfer.. HIPO. 6. hierarchy of input-processing-output. software safety requirements analysis. Syn: hard disk drive. software safety code analysis. (ISO) Physical equipment. software safety design analysis. human safety or other required characteristics. hazard probability. 1. disk drive. handshake. and usually results in several machine instructions for each Version 6. hard disk drive. (DOD) The aggregate probability of occurrence of the individual events that create a specific hazard. hexadecimal. 8. Hz. See: modular decomposition. Hardware used to read from or write to a hard disk. & F. (DOD) A condition that is prerequisite to a mishap.g. hardware. software safety change analysis. FMECA. See: FMEA.processing-output. high-level language. E. procedures. See: input-process-output chart. 7. software safety test analysis. hazard. allows symbolic naming of operations and addresses. hierarchical decomposition. Digits are 0. A programming language which requires little knowledge of the target computer. but that direction can change. A. half duplex. as opposed to programs. This is a convenient form in which to examine binary data because it collects 4 binary digits per hexadecimal digit.. hierarchy of input-processing-output chart. C. hazard analysis. output on paper. software hazard analysis. Transmissions [communications] which occur in only one direction at a time. B.Vocabulary -HHDD. hertz. 4. hard copy. etc. and associated documentation. hard disk drive. FTA. An interlocked sequence of signals between connected components in which each component waits for the acknowledgement of its previous signal before proceeding with its action. (DOD) An assessment of the consequence of the worst credible mishap that could be caused by a specific hazard. provides features designed to facilitate expression of data structures and program logic. See: input. Printed. hertz. 3.

Each microprocessor and each computer needs a way to communicate with the outside world in order to get the data needed for its programs and in order to communicate the results of its data manipulations. IEC. bottom-up. Contrast with assembly language. Methods include top-down. incremental development. integrated drive electronics. FORTRAN. International Telecommunications Union . input/output. The process of translating a design into hardware components. International Organization for Standardization. See: coding. industry standard. software engineering. implementation. Syn: dead code. Examples are PL/1.Guide to the CASQ CBOK program statement. implementation. incremental integration. depth-first. A structured reformation of the program module by module or function by function with an integration test being performed following each addition. infeasible path. This is accomplished through I/0 ports and devices. and "C". credentialing. software components. integrated circuit. ITU-TSS. (IEEE) A requirement that specifies or constrains the coding or construction of a system or system component. (NBS) A sequence of program statements that can never be executed. (QA) Procedures and criteria recognized as acceptable practices by peer professional. implementation requirement.2 . or accrediting organizations. (IEEE) A software development technique in which requirements definition. information hiding. Institute of Electrical and Electronic Engineers. and testing occur in an overlapping. making them inaccessible to other parts of the program. The practice of "hiding" the details of a function or structure. resulting in incremental completion of the overall software product. BASIC. design. IC. (IEEE) The period of time in the software life cycle during which a software product is created from design documentation and debugged. A-28 Version 6. IEEE. Contrast with rapid prototyping. Ada. IDE. Contrast with nonincremental integration. breadth-first. ISO. Pascal. waterfall model. spiral model.Telecommunications Standards Section. input/output. International Electrotechnical Commission. or both. COBOL. implementation phase. See: abstraction. iterative [rather than sequential] manner. -II/0. encapsulation.

consisting of a rectangle on the left listing inputs. code walkthrough. coverage analysis. code review. instruction. input-processing-output. design). if any. (1) (IEEE) The complete set of instructions recognized by a given computer or provided by a given programming language. See: static analysis. New York.Vocabulary input-process-output chart. source code or user's manuals] are examined in a very formal and disciplined manner to discover errors. (2) (ISO) In a programming language. See: computer instruction set. box diagram. bubble chart. ST-506. integrated drive electronics. and outputs at both general and detailed levels of detail.2 A-29 . (IEEE) The period of time in the software life cycle during which a software product is integrated into its operational environment and tested in this environment to ensure that it performs as required. (ANSI) The phase in the system life cycle that includes assembly and testing of the hardware and software of a computerized system. Checklists are a typical vehicle used in accomplishing this technique. installation. violations of standards and other problems. An organization involved in the generation and promulgation of standards. SCSI. flowchart. inspection. code audit. A standard interface for hard disks which provides for building most of the controller circuitry into the disk drive to save space. or of the programming languages in a programming system. installation. IDE controllers are functionally equivalent to ST-506 standard controllers. A refinement called hierarchical input-process-output identifies the steps. structure chart. inputs. new software or hardware. Version 6. (1) (ANSI/IEEE) A program statement that causes a computer to perform a particular operation or set of operations. Syn: chip. Installation includes installing a new computer system. a rectangle on the right listing outputs. and arrows connecting inputs to processing steps and processing steps to outputs. integrated circuit. practicing professionals in the given field. Contrast with EDSI. a meaningful expression that specifies one operation and identifies its operands. graph. A manual testing technique in which program documents [specifications (requirements. identification of the steps involved in each process to be performed and identifying the inputs to and outputs from each step. tuning. or otherwise modifying the current system. (IEEE) A diagram of a software system or module. installation and checkout phase. instrumentation. See: qualification. a rectangle in the center listing processing steps. 345 East 47th Street. Small wafers of semiconductor material [silicon] etched or printed with extremely small electronic switching circuits. (2) (ISO) The set of the instructions of a computer. of a programming language. A structured software design technique. See: block diagram. instruction set. (NBS) The insertion of additional code into a program in order to collect information about program behavior during program execution. Useful for dynamic analysis techniques such as assertion checking. installation qualification. Institute of Electrical and Electronic Engineers. code inspection. NY 10017. IEEE standards represent the formalization of current norms of professional practice through the process of obtaining the consensus of concerned.

Switzerland. (IEEE) Pertaining to a system or mode of operation in which each user entry causes a response from or action by the system. signal characteristics. An organization that sets standards for electronic products and components which are adopted by the safety standards agencies of many countries. persons. and readability. It deals with all fields except electrical and electronics which is governed by IEC. An organization that sets international standards. or sets forth constraints on formats. Consultive Committee for International Telephony and Telegraphy. common physical interconnection characteristics.. the processor restores its previous operating conditions and continues executing the interrupted program. as appropriate. Geneva. operator.Guide to the CASQ CBOK interactive. An international organization for communications standards. user. completeness. defined by functional characteristics. the device has data for the processor or the device is awaiting data from the processor. i. Entities to evaluate include data items and control items. operator. or other factors caused by such an interaction. accuracy. See: conversational. (IEEE) Evaluation of: (1) software requirements specifications with hardware. and other characteristics. International Organization for Standardization. or other physical entities. Contrast with assembler. After the device is serviced. Formerly. and software interface requirements documentation. (IEEE) A requirement that specifies an external item with which a system or system component must interact. International Electrotechnical Commission. Geneva. and software interface requirements specifications. interface analysis. The processor interrupts its current program.2 . The device sends a signal. compiler. The interpreter must be resident in the computer each time a program [source code file] written in an interpreted language is executed. Geneva. (2) A point of communication between two or more processes. online. compile. International Standards Organization. Contrast with assemble. interpreter. Switzerland.Telecommunications Standards Section. for correctness. The concept involves the specification of the connection of two devices having different functions.e. A-30 Version 6. interface. timing. (2) software design description records with hardware. called an interrupt. See: International Organization for Standardization. to the processor. (1) (ISO) A shared boundary between two functional units. and software interface design documentation. real time. Switzerland. Contrast with polling. (3) A peripheral device which permits two or more devices to communicate. Syn: International Standards Organization. (3) source code with hardware. interface requirement. International Telecommunications Union . interpret. (2) A technique to notify the CPU that a peripheral device needs service. (IEEE) To translate and execute each statement or construct of a computer program before translating and executing the next. interrupt. (IEEE) A computer program that translates and executes each statement or construct of a computer program before translating and executing the next. stores its current operating conditions. A method for handling constantly changing data. Contrast with batch. consistency. (1) The suspension of a process to handle an event external to the process. operator. and executes a program to service the device sending the interrupt.

2 A-31 . the entry of alpha characters or special keyboard characters when only numeric data is valid. the compilation. and control their execution. e. An asynchronous file transfer protocol developed at Columbia University. including its identification. One or more characters. Input/output connector. -KKB. loading. Zmodem. Kermit.Vocabulary interrupt analyzer. key element. invalid inputs. noted for its accuracy over noisy lines. Because computers use a binary number system.. A software tool which analyzes potential conflicts in a system as a result of the occurrences of interrupts. a kilobyte is precisely 210 or 1024 bytes. usually within a set of data. Ymodem. (IEEE) A user-defined unit of work that is to be accomplished by a computer. key. Version 6. This symbol is used to describe the size of computer memory or disk storage space. -LLAN. -JJCL. one thousand lines of code. job control language. (1) (NBS) Test data that lie outside the domain of the function the program represents. (QA) An individual step in an critical control point of the manufacturing process. or the input of abnormal command sequences to a program. For example. describe their requirements to an operating system. Contrast with Xmodem. (2) These are not only inputs outside the valid range for data to be input. See: job control language. kilobyte. job control language. Several versions exist. i. when the specified input range is 50 to 100. job. Approximately one thousand bytes. and execution of a computer program.e. local area network.g. (IEEE) A language used to identify a sequence of jobs. that contains information about the set. KLOC. I/O port. especially when these unexpected inputs may easily occur. kilobyte. but also unexpected inputs.

algorithms.2 . See: programming language. -M- A-32 Version 6. local area network. A program which copies other [object] programs from auxiliary [external] memory to main [internal] memory prior to its execution. low-level language. a network operating system. WAN. longitudinal redundancy check.000 to 100. See: waterfall model. ladder logic. (2) Evaluates the sequence of operations represented by the coded program and detects programming errors that might create hazards. See: software life cycle. by relocating elements. design.000 transistors. It is made up of servers. (IEEE) A computer program that creates a single load module from two or more independently translated object modules or load modules by resolving cross references among the modules and. fault. Syn: link editor. language.Guide to the CASQ CBOK LSI. May be part of a loader. Syn: waiting time. thus increasing the chance of introducing errors during program development and maintenance. For time critical operations. large scale integration. linkage editor. (IEEE) A system of error control based on the formation of a block check following preset rules. A classification of ICs [chips] based on their size as expressed by the number of circuits or logic gates they contain. latent defect. problem oriented. A graphical. (ISO) The time interval between the instant at which a CPU's instruction control unit initiates a call for data and the instant at which the actual transfer of the data starts. life cycle methodology. Contrast with MAN. See: assembly language. logic analysis. An LSI IC contains 3. This makes the source code harder to understand. The advantage of assembly language is that it provides bit-level control of the processor allowing tuning of the program for optimal speed and performance. and operate a system from its conception to the termination of its use. (IEEE) Evaluates the safety-critical equations. latency. loader. test. linker. A communications network that serves users within a confined geographical area. large scale integration. See: bug. possibly. and a communications link. The disadvantage of assembly language is the high-level of complexity and detail required in the programming. implement. programming language which replicates electronic switching blueprints. workstations. The use of any one of several structured methods to plan. life cycle. and control logic of the software design. assembly language may be necessary in order to generate code which executes fast enough for the required operations.

maintainability. MSI. macroinstruction. macro. Syn: modifiability. (IEEE) A source code instruction that is replaced by a predefined sequence of source instructions. metal-oxide semiconductor field effect transistor. Version 6. usually in the same language as the rest of the program and usually during assembly or compilation. subprogram. megabit. MTTR. All source code. overhauling equipment to assure performance in accordance with requirements. See: adaptive maintenance. (QA) Activities such as adjusting. machine language. cleaning. MB. main memory. main program.Vocabulary MAN. megahertz. regardless of the language in which it was programmed. Syn: object code. A non-moving storage device utilizing one of a number of types of electronic circuitry to store information. million instructions per second. a predefined sequence of computer instructions that is inserted into a program.2 A-33 . is eventually converted to machine code. metropolitan area network. Mb. MTTF. MTBF. mainframe. or adapt to a changed environment. megabyte. MIPS. improve performance or other attributes. Term used to describe a large computer. or making enhancements to software. See: machine code. MHz. usually during assembly or compilation. Maintenance to a software system includes correcting software errors. (IEEE) Computer instructions and definitions expressed in a form [binary code] that can be recognized by the CPU of a computer. mean time to repair. medium scale integration. maintenance. MOS. adapting software to a new environment. (IEEE) The ease with which a software system or component can be modified to correct faults. modifying. (IEEE) In software engineering. (IEEE) A software component that is called by the operating system of a computer and that usually calls other software components. corrective maintenance. mean time to failure. metal-oxide semiconductor. at each place that its corresponding macroinstruction appears in the program. MOSFET. machine code. perfective maintenance. mean time between failures. See: routine.

megabyte. microcomputer. measure. and auxiliary. CMOS.000 transistors. Common type of transistor fabricated as a discrete component or into MOS integrated circuits. megabit. Permanent memory that holds the elementary circuit operations a computer must perform for each instruction in its instruction set. RAM.g. Precisely 1024 K bits. memory. (IEEE) A quantitative measure of the degree to which software possesses a given attribute which affects its quality. metric. tape.. e.048. It derives its name from its use of metal. A classification of ICs [chips] based on their size as expressed by the number of circuits or logic gates they contain. megahertz. Approximately one million bits. oxide and semiconductor layers. See: storage device. A measure of the reliability of a computer system. Precisely 1024 K Bytes. A-34 Version 6. WAN. mean time to failure. disk. software quality. or 1. A unit of frequency equal to one million cycles per second. Contrast with LAN. Approximately one million bytes. metal-oxide semiconductor field effect transistor. menu. NMOS.048. 220 bytes.g. metropolitan area network. or 1. Capable of being measured.. The two types of memory are main. Communications network that covers a geographical area such as a city or a suburb. metric based test data generation. See: kilobyte. medium scale integration. Sometimes used to denote a list of programs. One of two major categories of chip design [the other is bipolar].g. measurement. mean time to repair.2 . from which the operator may select one.. Any device or recording medium into which binary data can be stored and held. See: microprocessor. A term used to describe a small computer. and from which the entire original data can be retrieved.Guide to the CASQ CBOK mean time between failures. A computer display listing a number of options. giving the average time before the first failure. measurable. ROM. An MSI IC contains 100 to 3. A measure of reliability. functions. as calculated on a statistical basis from the known failure rates of various components of the system. equal to average operating time of equipment between failures.576 bits. A measure of reliability of a piece of repairable equipment. There are several varieties of MOS technologies including PMOS. giving the average time between repairs. metal-oxide semiconductor. e. e. (NBS) The process of generating test sets for structural testing based upon use of complexity metrics or coverage metrics. (IEEE) A quantitative assessment of the degree to which a software product or process possesses a given attribute. 220 bits. The process of determining the value of some quantity in terms of a standard unit. microcode.576 bytes.

module. Frequently synonymous with a microcomputer. a linkage editor. Construction of programs used to model the effects of a postulated environment for investigating the dimensions of a problem for the effects of algorithmic processes on responsive targets. modem access. an abbreviation such as "MPY" for multiply. A symbol chosen to assist human memory and understanding. mishap. See: unit. modularity. See: maintainability.Vocabulary microprocessor. (IEEE) The degree to which a system or computer program is composed of discrete components such that a change to one component has minimal impact on other components. Syn: functional decomposition. modulate.2 A-35 . MIPS rate is one factor in overall performance. Execution speed of a computer. MODEM access is often used between a remote location and a computer that has a master database and applications software. Bus and channel speed and bandwidth. (1) In programming languages. module interface table. memory speed. occupational illness. See: abstraction. modifiability. a self. Varying the characteristics of a wave in accordance with another wave or signal. usually to make user equipment signals compatible with communication facilities. Contrast with demodulate. breaking a system into components to facilitate design and development. memory management techniques. The term is a contraction of modulator-demodulator.. modular software. modulation. Syn: accident. See: structured design. hierarchical decomposition. A table which provides a graphic illustration of the data elements whose values are input to and output from a module. minicomputer. Converting signals from a binary-digit pattern [pulse form] to a continuous wave form [analog]. A structured software design technique. (ISO) A functional unit that modulates and demodulates signals. or similar routine or subroutine. e.g. modem. million instructions per second.contained subdivision of a program that may be separately compiled. (DOD) An unplanned event or series of events resulting in death. usually processed as a unit. (3) A packaged functional hardware unit suitable for use with other components. modular decomposition. A CPU existing on a single IC. the host computer. Using a modem to communicate between computers. or damage to the environment. mnemonic. One of the functions of a modem is to enable digital data to be transmitted over analog transmission facilities. Contrast with demodulation. by an assembler. injury. a compiler. A term used to describe a medium sized computer. and system software also determine total throughput. (2) A discrete set of instructions. modeling. or damage to or loss of data and equipment or property. Version 6. (IEEE) Software composed of discrete parts.

Gaithersburg. The Institute's overall goal is to strengthen and advance the Nation's science and technology and facilitate their effective application for public benefit. among other things. condition coverage. Syn: parallel processing. See: multi-processing.g. are invoked at least once. (IEEE) A mode of operation in which two or more tasks are executed in an interleaved manner. multiple condition coverage. National Institute for Standards and Technology. NIST.. The National Computer Systems Laboratory conducts research and provides.2 . decision coverage. statement coverage. word processing. See: multi-tasking. multi-programming. Syn: parallel processing. MD 20899. (Myers) A test coverage criteria which requires enough test cases such that all possible combinations of condition outcomes in each decision. A federal agency under the Department of Commerce. (IEEE) Computer systems that perform more than one primary function or task are considered to be multipurpose. (NBS) A method to determine test set thoroughness by measuring the extent to which a test set can discriminate the program from slight variants [mutants] of the program. non-maskable interrupt. originally established by an act of Congress on March 3. A device which takes information from any of several sources and places it on a single line or sends it to a single destination. 1901 as the National Bureau of Standards. the technical foundation for computer related policies of the Federal Government. path coverage. A-36 Version 6. time sharing. NMOS. Contrast with multi-programming. multi-tasking. multi-programming. National Bureau of Standards. multipurpose systems.Guide to the CASQ CBOK multi-processing. e. In some situations the computer may be linked or networked with other computers that are used for administrative functions. mutation analysis. n-channel MOS. n-channel MOS. National Bureau of Standards. accounting. (IEEE) A mode of operation in which two or more programs are executed in an interleaved manner by a single CPU. NMI. A type of microelectronic circuit used for logic and memory chips. Contrast with branch coverage. time sharing. (IEEE) A mode of operation in which two or more processes [programs] are executed concurrently [simultaneously] by separate CPUs that have access to a common main memory. -NNBS. Now National Institute for Standards and Technology. See: time sharing. National Institute for Standards and Technology. Contrast with multi-tasking. multiplexer. Contrast with error seeding. and all points of entry.

object oriented programming. Integration testing is then conducted on the program as a whole. OEM. This value is a representation of the set of no numbers or no value for the operating system in use. original equipment manufacturer. e. OOP. A reformation of a program by immediately relinking the entire program following the testing of each independent module. This analysis is to show this isolation is complete and that interfaces between safety-critical code and nonsafety-critical code do not create hazards. node. object. safety-critical code should be isolated from non-safety-critical code. and math co-processor errors. Generally. a terminal or a computer. (IEEE) A string containing no entries. A single data element can point to multiple data elements and can itself be pointed to by other data elements. Half a byte. Syn: "big bang" integration.Vocabulary network. -OOCR. A self contained module [encapsulation] of data and the programs [services] that manipulate [process] that data. (IEEE) Data for which space is allocated but for which no value currently exists. Contrast with relational database. (2) A system [transmission channels and supporting hardware and software] that connects several remotely located computers via telecommunications. See: object program. It can be used to report malfunctions such as parity. Version 6. In object oriented programming. (1) (ISO) An arrangement of nodes and interconnecting branches. bus. A junction or connection point in a network. network database. null data. Contrast with source code. noncritical code analysis. or four bits. (NIST) A code expressed in machine language ["1"s and "0"s] which is normally an output of a given translation process that is ready to be executed by a computer. Note: It is said that a null string has length zero. Syn: machine code. Contrast with incremental integration. nibble. object code. non-maskable interrupt.g. (IEEE) A value whose definition is to be supplied within the context of a specific operating system. null. A database organization method that allows for data relationships in a netlike form. optical character recognition.2 A-37 . nonincremental integration. A high priority interrupt that cannot be disabled by another interrupt. (IEEE) (1) Examines software elements that are not designated safety-critical and ensures that these elements do not cause a hazard. null string. (2) Examines portions of the code that are not considered safety-critical code to ensure they do not cause hazards.

interactive. octal. (IEEE) A software development technique in which a system or component is expressed in terms of objects and connections between those objects. 3. the prime consideration should be the program's level of accuracy as it applies to the type of document to be scanned. optical character recognition. The modules are created in class hierarchies so that the code or methods of a class can be passed to other modules. optimization. e. 2. A technology for writing programs that are made up of selfsufficient modules that contain all of the information needed to manipulate a given data structure. 1. (NIST) Modifying a program to improve performance.Guide to the CASQ CBOK object oriented design. operation and maintenance phase. operating systems are predominantly software. 4. to identify the characters by their shape from the light that is reflected and creates an output disk file. real time. scheduling. object oriented design.2 . operating system. (IEEE) An exception that occurs when a program encounters an invalid operation code. When choosing an OCR product. Unlike electrical pulses. Thin glass wire designed for light transmission. object oriented programming. See: object. See: end user. For example. input/output control. (IEEE) The period of time in the software life cycle during which a software product is employed in its operational environment. to make it run faster or to make it use fewer resources. object oriented language. Examples include C++. (IEEE) A programming language that allows the user to express a program in terms of objects and messages between those objects. An information processing technology that converts human readable data into another medium for computer input. See: conversational. the printed page must contain only characters of a type that are easily read by the OCR device and located on the page within certain margins. object program. monitored for satisfactory performance.g. An OCR peripheral device accepts a printed document as input. The base 8 number system. and data management. 6. (ISO) Software that controls the execution of programs. and that provides services such as resource allocation. operator. an airline reservation system. but partial or complete hardware implementations are possible. operation exception. Usually. A-38 Version 6. capable of transmitting billions of bits per second.. optical fiber. For best results. (IEEE) A computer program that is the output of an assembler or compiler. on-line. Contrast with batch. 5. & 7. Digits are 0. (IEEE) Pertaining to a system or mode of operation in which input data enter the computer directly from the point of origin or output data are transmitted directly to the point where they are used. and modified as necessary to correct problems or to respond to changing requirements. New object modules can be easily created by inheriting the characteristics of existing classes. light pulses are not affected by random radiation in the environment. Smalltalk and LOGO. Accuracy levels less than 97% are generally considered to be poor.

parallel processing. overflow. the state in which the calculator is unable to accept or process the number of digits in the entry or in the result. and pages are stored in page frames. cyclic redundancy check [CRC]. using separate facilities for the various parts. A relational database programming system incorporating the SQL programming language.Vocabulary Oracle. programmable logic array. parameter. PCB. Version 6. programmable read only memory. (ISO) A redundancy check by which a recalculated parity bit is compared to the predetermined parity bit. byte. (3) Term describing simultaneous transmission of the bits making up a character. (ISO) A binary digit appended to a group of binary digits to make the sum of all the digits. A registered trademark of the Oracle Corp. parity check. as predetermined. PC. such as the bits of a character or the characters of a word. programmable logic device. personal computer. See: multi-processing. (IEEE) A storage allocation technique in which programs or data are divided into fixed length blocks called pages. (1) (IEEE) Pertaining to the simultaneity of two or more processes. overflow exception. main storage/memory is divided into blocks of the same length called page frames. parity. (ISO) In a calculator.programming. message] to cause the bit patterns to have either an odd number of 1-bits [odd parity] or an even number of 1-bits [even parity]. original equipment manufacturer. See: arithmetic overflow. and pages are transferred between main and auxiliary storage as needed. usually eight bits [one byte]. printed circuit board. paging. Contrast with serial. PMOS. variable or expression that is used to pass values between software modules. Syn: argument. either odd or even. A manufacturer of computer hardware.2 A-39 . parity bit. PROM. program design language. including the appended binary digit. An error detection method in data transmissions that consists of selectively adding a 1bit to bit patterns [word. not necessarily contiguously or in logical order. Contrast with check summation. PDL. -PPAL. positive channel MOS. character. multi. parallel. (2) (IEEE) Pertaining to the simultaneous processing of individual parts of a whole. programmable array logic. (IEEE) An exception that occurs when the result of an arithmetic operation exceeds the size of the storage location designated to receive it. PLA. (IEEE) A constant. PLD.

weight. Synonymous with microcomputer. to detect incomplete paths. or other attributes of a computer program. (IEEE) A change made directly to an object program without reassembling or recompiling from the source program. path coverage. perfective maintenance. but is not limited to the operating system or executive software. physical requirement. (2) In computer graphics. microprocessor. maintainability.2 . personal computer. printer.g. See: peripheral device. A platform includes. performance requirement. communication software. network. A-40 Version 6. video system. path. or to output data. (IEEE) (1) In image processing and pattern recognition.g. password. (IEEE) Analysis of a computer program [source code] to identify all possible paths through the program. but serves only one user. size.g. corrective maintenance. (IEEE) A sequence of instructions that may be performed in the execution of a computer program. transducer. a requirement that specifies the speed. as built. or to discover portions of the program that are not on any path. any generic software libraries. valve controller. e. user interface software.. peripheral device. e. path. bar code reader. patch. A high-level programming language designed to encourage structured programming practices. physical configuration audit. a computer that is functionally similar to large computers. and the like. shape. This term is derived from the term "picture element". platform. (ISO) A character string that enables a user to have full or limited access to a system or to a set of data. the smallest element of a digital image that can be assigned a gray level.. material. (IEEE) A requirement that imposes conditions on a functional requirement. pixel. peripheral equipment. See: functional configuration audit.. disk drive.. tape drive. path analysis. laboratory test equipment.g. database management. the smallest element of a display surface that can be assigned independent characteristics. Equipment that is directly connected a computer. e. motor controller. accuracy. (IEEE) An audit conducted to verify that a configuration item. (IEEE) Software maintenance performed to improve the performance. A peripheral device can be used to input data. See: testing.Guide to the CASQ CBOK Pascal. e. conforms to the technical documentation that defines it. Syn: peripheral equipment. or memory usage with which a given function must be performed. Contrast with adaptive maintenance. keypad. (IEEE) A requirement that specifies a physical characteristic that a system or system component must posses. input/output hardware. The hardware and software which must be present and functioning for an application program to run [perform] as intended.

and risk resolution of the selected design approach for one or more configuration items. (2) (ISO) To design. (IEEE) (1) The process of analyzing design alternatives and defining the architecture. to establish the existence and compatibility of the physical and functional interfaces among the configuration items and other items of equipment. program. printed circuit board. preliminary design. how closely the values within a series of replicate measurements agree. bias. used to develop. and test programs. (5) Loosely. (1) (ISO) A sequence of instructions suitable for processing. A type of microelectronic circuit in which the base material is positively charged.Vocabulary polling. and. The relative degree of repeatability. program mutation. (IEEE) A review conducted to evaluate the progress. (2) The result of the process in (1). See: programmable logic device. The computer file that contains the establishment's current production data. interfaces. verification protocols. A programmable logic chip. sometimes. This method is useful if the device's data can wait for a period of time before being processed. software and personnel. Contrast with interrupt. A programmable logic chip. See: testing. to write a routine. or another translator to prepare the program for execution. program design language. a routine. See: programmable logic device. write. preliminary design review. production database. The device must wait until it is polled in order to send or receive data. programmable array logic. an interpreter. since each device must await its turn in the polling scheme before it will be serviced by the processor.e. Version 6. facilities. It is the result of resolution and stability. to evaluate the preliminary operational and support documents. The instructions may include statements and necessary declarations. i. A technique a CPU can use to learn if a peripheral device is ready to receive data or to send data. components. (IEEE) A computer program that has been purposely altered from the intended version to evaluate the ability of program test cases to detect the alteration. technical adequacy. A logic chip that is programmed at the user's site. (3) (ANSI) In programming languages. to evaluate the degree of definition and assess the technical risk associated with the selected manufacturing methods and processes. See: accuracy. calibration. See: detailed design. Processing may include the use of an assembler. a compiler. The board is "printed" with electrically conductive pathways between the components.2 A-41 . and timing and sizing estimates for a system or component. A flat board that holds chips and other electronic components. Contrast with PROM. and document a program design. precision. (4) Loosely. positive channel MOS. as applicable. a set of one or more interrelated modules capable of being executed. mutation. In this method each device is checked or polled in-turn to determine if that device needs service. analyze. to determine each design's compatibility with the requirements for the configuration item. programmable logic array. (IEEE) A specification language with special constructs and. programmable logic device.

See: computer language. A chip which may be programmed by using a PROM programming device. software development plan. quality control. An unprogrammed PROM has all links closed establishing a known state of each bit. It can be programmed only once. programming style analysis. security plans. See: rapid prototyping. (ISO) A set of semantic and syntactic rules that determines the behavior of functional units in achieving communication. (IEEE) Analysis to ensure that all portions of the program follow approved programming guidelines. the schedules to be met. protocol. etc. If used. test plans.Guide to the CASQ CBOK programmable read only memory. (IEEE) An exception that occurs when a program attempts to write into a protected area in storage. proof of correctness. the configuration management and quality assurance procedures to be followed. A-42 Version 6. Each of its bit locations is a fusible link. Project in this context is a generic term. The plan typically describes work to be done. prototyping. opening that link. pseudocode. (IEEE) A language used to express computer programs. programming language. A combination of programming language and natural language used to express a software design. quality assurance. code inspection. See: code audit. -QQA. Some projects may also need integration plans. Using software tools to accelerate the software development process by facilitating the identification of required functionality during analysis and design phases. protection exception. test plan. project plan. A limitation of this technique is the identification of system or software problems and hazards. methods to be used.2 . See: coding standards. See: documentation plan. This causes the "fuse to blow". etc. (NIST) A management document describing the approach taken for a project. it is usually the last document produced prior to writing the source code. coding standards. the project organization. quality assurance plans. Electronic equipment which is used to transfer a program [write instructions and data] into PROM and EPROM chips. low-level language. It cannot be erased and reprogrammed. high-level language. resources required. software engineering. QC. (NBS) The use of techniques of mathematical logic to infer that a relation between program variables assumed true at program entry implies that another relation between program variables holds at program exit. programming standards. Programming the chip consists of sending an electrical current of a specified size through each link which is to be changed to the alternate state. PROM programmer.

(FDA) Establishing confidence through appropriate testing that the finished product produced by a specified process meets all release requirements for functionality and safety. (2) All actions that are taken to ensure that a development organization delivers products that meet performance requirements and adhere to standards and procedures. damage components and cause malfunctions. High frequency electromagnetic waves that emanate from electronic devices such as chips and other electronic devices. Chips which can be called read/write memory. read only memory. radar. ROM. radiofrequency interference. planned and performed. installation. and output. since the data stored in them may be read or new data may be written into any memory address on these chips. (1) (ISO) The planned systematic activities necessary to ensure that a component. -RRAM. update. quality assurance. product performance. radio and TV signals. software. lightning. (FDA) Establishing confidence that process equipment and subsystems are capable of consistently operating within established limits and tolerances. random access memory. (2) A set of activities designed to evaluate the process by which products are developed or manufactured. RFI. (FDA) Establishing confidence that the process is effective and reproducible. See: electromagnetic interference. process performance. which includes input. (3) The policy.2 A-43 . quality assurance. and systematic actions established in an enterprise for the purpose of providing and maintaining some degree of confidence in data integrity and accuracy throughout the life cycle of the data. RISC. operational.Vocabulary qualification. reduced instruction set computer. quality control. The Version 6. or system conforms to established technical requirements. qualification. qualification. (4) (QA) The actions. qualification. (IEEE) (1) A planned and systematic pattern of all actions necessary to provide adequate confidence that an item or product conforms to established technical requirements. procedures. to provide confidence that all systems and components that influence the quality of the product are working as expected individually and collectively. The operational techniques and procedures used to achieve quality requirements. and that manufacturer's recommendations are suitably considered. module. radiofrequency interference. random access memory. (FDA) Establishing confidence that process equipment and ancillary systems are compliant with appropriate codes and approved design intentions. An electromagnetic disturbance caused by such radiating and transmitting sources as electrostatic discharge [ESD]. manipulation. and motors with brushes can induce unwanted voltages in electronic circuits.

hardware and software. record of change. in order that the computation results can be used to control. real time processing. performs computations. See: conversational. interrupt. record. ROM. but they are read only not read/write memories. i. ROM memory is also random access memory. Systems using RISC technology are able to achieve processing speeds of more than five million instructions per second. or respond in a timely manner to the external process.g. A fast-response [immediate response] on-line system which obtains data from an activity or a physical process. but rather a return to simple instructions requiring only one or a very few instruction cycles to execute. A memory chip from which data can only be read by the CPU. See: prototyping. a record is a component of a file (database)]. (1) (ISO) a group of related data elements treated as a unit. EPROM. range check. interactive. PROM.Guide to the CASQ CBOK term random access means that each memory location [usually 8 bits or 1 byte] may be directly accessed [read from or written to] at random. Contrast with incremental development. Another difference between RAM and ROM is that RAM is volatile. A record of change can be a written document or a database.. and therefore are more effectively utilized with innovative architectural and compiler changes. real time. it must have a constant supply of power or the stored data will be lost. (IEEE) Pertaining to a system or mode of operation in which computation is performed during the actual time that an external process occurs. spiral model. a process control application. Changes made to the data are recorded in an audit trail. This advantage applies to all types of ROM chips. This contrasts to devices like magnetic tape where each section of the tape must be searched sequentially by the read/write head from its current location until it finds the desired location. A clearly described area within the computer's storage that is logically and/or physically distinct from other regions.2 . A structured software requirements discovery technique which emphasizes generating prototypes early in the development process to permit early feedback and analysis in support of the development process. rapid prototyping. The CPU may not store data to this memory. waterfall model. on-line. Contrast with batch processing. Contrast with batch. (IEEE) (1) The process of defining or generating a process or data structure in terms of itself. Computer architecture that reduces the complexity of the chip by using simpler instructions. region. Syn: partition. and EEPROM. monitor. and returns a response rapidly enough to affect [control] the outcome of the activity or process. reduced instruction set computer. read only memory. Normally there are two associated with a computer system.e. (ISO) A limit check in which both high and low values are stipulated. A-44 Version 6. Reduced instruction set does not necessarily mean fewer instructions. (2) A process in which a software module calls itself. Documentation of changes made to the system. Regions are used to separate testing from production [normal use]. The advantage of ROM over RAM is that ROM does not require power to retain its program. recursion. e. [A data element (field) is a component of a record.

(ANSI/IEEE) The process of determining the achieved level of reliability for an existing system or system component. (IEEE) The formal notification and distribution of an approved version. test readiness review. Database organization method that links files together as required. interface requirement. registers keep track of the address of the instruction being executed and the data being processed. Types include system requirements review. Each microprocessor has a specific number of registers depending upon its design.g. Contrast with code review. release. requirement. or other formally imposed documents. See: prototyping. (IEEE) A trace produced from historical data recorded during the execution of a computer program. a customer file and an order file can be linked in order to ask a question that relates to information in both files. customers. formal qualification review. Contrast with network database. retrospective trace.. or software requirements. relational database. implementation requirement. (2) A condition or capability that must be met or possessed by a system or system component to satisfy a contract. managers. (IEEE) (1) A condition or capability needed by a user to solve a problem or achieve an objective. reliability. hardware item. See: testing. requirements review. or other interested parties for comment or approval. high speed memory circuit within a microprocessor that holds addresses and values of internal operations. requirements phase. requirements analysis. design review. standard. users. specification. (IEEE) A process or meeting during which the requirements for a system. such as the names of the customers that purchased a particular product. are defined and documented. (IEEE) A software V&V task to determine the extent of V&V analysis and testing that must be repeated when changes are made to any previously examined software products. retention period. See: version. or software item are presented to project personnel. such as functional and performance capabilities for a software product. regression analysis and testing.. (ISO) The length of time specified for data on a data medium to be preserved. (IEEE) The period of time in the software life cycle during which the requirements. performance requirement. or software requirements. (IEEE) (1) The process of studying user needs to arrive at a definition of a system. regression.2 A-45 . software engineering. Routine queries often involve more than one data file. physical requirement.Vocabulary register. (IEEE) The ability of a system or component to perform its required functions under stated conditions for a specified period of time. (2) The process of studying and refining system. A small. Note: this differs from an ordinary trace. hardware. flat file. See: design requirement. e. hardware. which is produced Version 6. (3) A documented representation of a condition or capability as in (1) or (2). A relational system can take any two or more files and generate a new file from the records that meet the matching criteria.g. e. functional requirement. Relationships between files are created by comparing data such as account numbers and names. software requirements review. reliability assessment. See: software reliability.

Types include code review. Data is transmitted and received in serial format. SQL. managers. safety critical path.g. The degree to which a software system or component can function correctly in the presence of invalid inputs or stressful environmental conditions. standard operating procedures. See: version number. Note: This term is defined differently in various programming languages. revision number. and performing the necessary regression testing. safety. formal qualification review. inspection. subroutine trace. RS-232-C. assessing the nature of the change to determine potential ripple effects. revalidation. review. See: software reliability. test readiness review. symbolic trace. Contrast with audit. security. event. (IEEE) A process or meeting during which a work product or set of work products. performance or tolerance is essential to safe system operation or use. (DOD) A comprehensive evaluation of the risk and its associated impact. is presented to project personnel. robustness. See: execution trace. SOPs. safety critical component. safety critical. customers. process or item of whose proper recognition. injury. (DOD) A term applied to a condition.Guide to the CASQ CBOK cumulatively during program execution. (IEEE) A subprogram that is called by other programs and subprograms. An Electronic Industries Association (EIA) standard for connecting electronic equipment. control. risk. See: computer system security. or damage to or loss of equipment or property. risk assessment. (DOD) Those computer software components and units whose errors can result in a potential hazard.. small computer systems interface. or loss of predictability or control of a system. users. requirements review. A-46 Version 6. (DOD) Freedom from those conditions that can cause death. small scale integration. See: module. structured query language. SSI. variable trace. or other interested parties for comment or approval. See: static analysis. occupational illness. Often taken as the simple product of probability and consequence. revalidation means validating the change itself. routine. (IEEE) A measure of the probability and severity of undesired effects.2 . safety critical computer software components. e. Relative to software changes. design review. or damage to the environment. operation. safety critical function. -SSCSI.

such as temperature. and even code segments may be simulated. See: application software. using the same facilities for successive parts. software. such as the bits of a character or the characters of a word. Contrast with emulation. (2) (IEEE) A model that behaves or operates like a given system when provided a set of controlled inputs. (1) (NBS) Use of an executable model to represent the behavior of an object. simulation. small scale integration. Up to seven SCSI devices can be linked to a single SCSI port. Version 6. servomechanism. small computer systems interface. the external environment. Contrast with timing. Contrast with hardware. Contrast with ST-506. system software. IDE. procedures. Contrast with emulator. simulation analysis. severity. EDSI. (ANSI) Programs. (IEEE) The process of estimating the amount of computer storage or the number of source lines required for a software system or component. Contrast with parallel. server. It holds the programs and data that are shared by all users. and any associated documentation pertaining to the operation of a system. An unintended alteration of a program's behavior caused by a change in one part of the program. and converts it to an electrical signal which can be further converted to a digital signal for processing by the computer. See: regression analysis and testing. See: criticality. tape drives and other peripheral devices that require high-speed data transfer. (IEEE) A device. without taking into account the effect the change has on another part of the program. rules. computer program. service program. serial. During testing the computational hardware. sizing and timing analysis. (1) Pertaining to the sequential processing of the individual parts of a whole. (2) Term describing the transmission of data one bit at a time. (IEEE) A software V&V task to obtain program sizing and execution timing information to determine if the program will satisfy processor size and performance requirements allocated to software. Its function is to present data to the system at known speeds and in a proper format. (ANSI) (1) An automatic device that uses feedback to govern the physical position of an element. A high speed computer in a network that is shared by multiple users. A simulator provides inputs or responses that resemble anticipated process parameters.2 A-47 . An SSI IC contains up to 100 transistors. A standard method of interfacing a computer to disk drives. (IEEE) A software V&V task to simulate critical tasks of the software or system environment to analyze logical or performance characteristics that would not be practical to analyze manually. side effect. Syn: utility program. utility software. A classification of ICs [chips] based on their size as expressed by the number of circuits or logic gates they contain. A peripheral input device which senses some variable in the system environment. or system that behaves or operates like a given system when provided a set of controlled inputs. simulator. sizing.Vocabulary sensor. (2) A feedback control system in which at least one of the system signals represents a mechanical motion. operating system.

and may be thought of as a blueprint or model of the system. code listings. e. testing the code. test report. test plans. implementation. (IEEE) A representation of software created to facilitate analysis.. (NIST) Technical data or information. A-48 Version 6. for the module. (5) Program source code. See: developer. notes. (NIST) A collection of material pertinent to the development of a software module. number of states. technical reports. (2) Software requirements and design specifications. that describe or specify the design or details. See: specification. Contrast with software development process. requirements. Contents typically include the requirements. problem reports.2 . or provide operating instructions for using the software to obtain desired results from a software system. (IEEE) A deliverable or in.process document produced or acquired during software development or maintenance. transforming the software requirements into design. etc. possibly accidental. See: structured design. the process involves translating user needs into software requirements. attributes. software element. software design description. and decision making. Syn: software development file. and sometimes installing and checking out the software for operational activities. in human readable form. implementing the design in code. software design description. or property of software. software development plans. trait. See: configuration item. software life cycle. design. including computer listings and printouts. specification. user's guide. additional documentation or reduced probability that programming or compiler errors will influence the end results. specification. quality. functionality. (4) Customer-deliverable documentation.e. software development process. software diversity. design constraints. specification. design. and software verification and validation plans. rapid prototyping. (IEEE) A software development technique in which two or more functionally identical variants of a program are developed from the same specification by different programmers or programming teams with the intent of providing error detection. (NIST) The project plan for the development of a software product. software development notebook. Note: these activities may overlap or be performed iteratively. The software design description is used as a medium for communicating software design information.Guide to the CASQ CBOK software audit. test results. planning.. An inherent. (3) Test documentation. spiral model. i. design description. See: software review. software configuration item. software development plan.g. (IEEE) The process by which user needs are translated into a software product. performance. test plan. waterfall model. software documentation. Specific examples include but are not limited to: (1) Project planning documents. See: incremental development. increased reliability. software developer. software characteristic. lines or branches. schedules. explain the capabilities.

operation. test tools. and firmware used to perform a software engineering effort. and database management systems. testing. See: risk assessment. defect detection. requirements analysis. software engineering. simulators. (IEEE) The hardware. code inspection. documentation tools.. software life cycle. software safety change analysis. software element analysis. review. See: software review.. software safety change analysis. structured design. Contrast with software element. system safety.e. software engineering environment. This evaluation follows a formal process. (IEEE) Source code. project status. (IEEE) An evaluation of software elements to ascertain discrepancies from planned results and to recommend improvement. (NIST) Period of time beginning when a software product is conceived and ending when the product is no longer available for use. design. programming. does not impact on a previously resolved hazard. the classification and estimation of potential hazards. software reliability. Contrast with software item. i. and maintenance of software. installation. static analysis. audit. The software life cycle is typically broken into phases denoting activities such as requirements. software safety design analysis. debuggers. architectural design. if any. i. testing. Typical elements include computer equipment. job control code. (IEEE) The application of a systematic. software.e. the application of engineering to software. operating systems. disciplined. compilers. are encountered. code review. object code. does not make a currently existing hazard Version 6. requirements. software review. (2) The ability of a program to perform its required functions accurately and reproducibly under stated conditions for a specified period of time. specification analysis. design review. See: specification. See: configuration item. software hazard analysis. control data. software safety requirements analysis. (7) Reports. or a collection of these items. See: project plan. i. See: waterfall model.Vocabulary (6) Representation of software solutions implemented in firmware. Syn: software audit. CDRH) The identification of safety-critical software. software requirements specification. The inputs to the system determine whether existing faults. and operation and maintenance. configuration management. emulators. (IEEE) Analysis of the safety-critical design elements affected directly or indirectly by the change to show the change does not create a new hazard. (ODE. software safety test analysis.2 A-49 . (8) Data. Contrast with software development process. (IEEE) (1) the probability that software will not cause the failure of a system for a specified time under specified conditions. assemblers. and identification of program path analysis to identify hazardous combinations of internal and environmental program conditions. quantifiable approach to the development.e. See: code audit.. system safety. software safety code analysis. code walkthrough. The probability is a function of the inputs to and use of the system in the software. test. software item.

functional analysis. and regression testing. robustness. stress testing. See: source code. special test data. See: criticality analysis. (IEEE) Verification that the safety-critical portions of the design are correctly implemented in the code. special case. software safety test analysis. Contrast with object code. unit level tests. timing and sizing analysis. reliability. assembled. Contrast with structured programming. the requirements. Program source code written without a coherent structure. performance. testing. behavior. design. design. programming language. and does not adversely affect any safety-critical software design element. specification. integrity. Tests may include. See: error guessing. specification. consistency. formal. (1) (IEEE) Computer instructions and data definitions expressed in a form suitable for input to an assembler. specification. interface analysis. in a complete. requirements. (IEEE) A computer program that must be compiled. source code. different software systems analyses. internal instrumentation. precise. software safety requirements analysis. software hazard analysis. flexibility. or otherwise translated in order to be executed by a computer. programming style analysis. maintainability. accuracy. noncritical code analysis. timing and sizing analysis. functional. usability. (2) The human readable version of the list of instructions [program] that cause a computer to perform a task. specification. specification analysis.or other characteristics of a system or component.2 . interface. performance. auditability. (IEEE) Evaluation of each safety-critical software requirement with respect to a list of qualities such as completeness. correctness. system safety. and often. testability. portability. specification analysis. See: logic analysis. software hazard analysis. timing and sizing analysis. software hazard analysis. system safety. system safety. (IEEE) Verification that the safety-critical portion of the software design correctly implements the safety-critical requirements and introduces no new hazards. software element analysis. coding standards. (IEEE) A document that specifies. data analysis. constraint analysis. system level testing. specification. reliability analysis. Contrast with object program. See: source program. interface analysis. (IEEE) Analysis demonstrating that safety requirements have been correctly implemented and that the software functions safely within its specified environment. software configuration item testing. (NBS) Test data based on input values that are likely to require special handling by the program. source program. See: logic analysis. Implies the excessive use of GOTO instructions. See: specification. data analysis. system safety. See: software hazard analysis. system safety. specification. constraint analysis. software safety code analysis. verifiable manner. interoperability. the procedures for determining whether these provisions have been satisfied. security and training. See: software hazard analysis.Guide to the CASQ CBOK more severe. Contrast with requirement. (IEEE) Analysis evaluating software and interface requirements to identify errors and deficiencies that could contribute to a hazard. software safety design analysis. design standards. compiler or other translator. A-50 Version 6. spaghetti code. interface tests.

interface requirements. data structures. performance requirements. specification. and testing. preliminary and detailed design. Syn: evolutionary model.2 A-51 . (NIST) A specification that documents the functional requirements for a system or system component. requirements. Contrast with incremental development. These characteristics typically include speed. specification. integration. It typically includes system or component structure. functional. development [coding] standards. typically requirements analysis. specification. specification tree. spiral model. (NIST) (1) A specification written and approved in accordance with established standards. Often part of a requirements specification. system. product. (IEEE) A document which describes the as built version of the software. data set [file] use information. See: test case. and memory usage. waterfall model. interface. accuracy. (IEEE) A model of the software development process in which the constituent activities. coding. Often part of a requirements specification. Written procedures [prescribing and describing the steps to be taken in normal and defined conditions] which are necessary to assure control of production and processes. Version 6. specification. interface descriptions. etc. standard operating procedures. design. performance. specification. (IEEE) A diagram that depicts all of the specifications for a given system and shows their relationship to one another. See: requirements specification. (NIST) A specification that documents how a system is to be built. It typically includes functional requirements. IDE. algorithms. Contrast with requirement. Contrast with design standards. (IEEE) A document that sets forth the performance characteristics that a system or component must possess. A standard electrical interface between the hard disk and controller in IBM PC compatible computers. Contrast with requirement. test case. design. requirement. are performed iteratively until the software is complete. It describes what the system or component is to do rather than how it is to be built. Contrast with requirement. specification. SCSI. (2) A specification expressed in a requirements specification language. Often part of a requirements specification. Contrast with requirement. (NIST) A specification that documents the interface requirements for a system or system component. programming. design requirements [attributes and constraints]. Contrast with EDSI. rapid prototyping. control logic. formal.Vocabulary specification. (NIST) See: specification. ST-506. specification. (NIST) A specification that documents the requirements of a system or system component. specification. specification. See: software design description. Contrast with requirement. input/output formats. etc.

structured query language. data and processing steps are defined broadly at first. modular software. structured design. structure chart. input-processing-output. the inputs. modular decomposition. See: code audit. Contrast with call graph. See: memory. and processing steps. stepwise refinement. stepwise refinement. structured programming. and then further defined with increasing detail. static analyzer. or other entities in a system or computer program and shows how larger or more general entities break down into smaller.g. See: state diagram. e. and the outputs. structured programming. documentation. (2) (IEEE) The process of evaluating a system or component based on its form. component. content. (ANSI/IEEE) A software tool that aides in the evaluation of a computer program without executing the program. structure. transform analysis. SQL commands can be used to interactively work with a data base or can be embedded with a programming language to interface with a database. storage device. Syn: hierarchy chart. code walk-through. retained and retrieved. rapid prototyping. static analysis. (Beizer) A representation of a state graph that specifies the states. and shows the events or circumstances that cause or result from a change from one state to another. activities. See: state-transition table. statement.Guide to the CASQ CBOK state. See: data structure centered design. (2) A linear sequence of entities such as characters or physical elements. string. and stepwise refinement of data. Syn: state graph. See: testing. graphical software specification/design documents. there have been many implementations created for mini and micro computer database applications.2 . A unit into which data or programs can be placed. the pre-flight state of an aircraft navigation program or the input state of a given channel. state-transition table. (IEEE) (1) A sequence of characters. and flowcharters. (IEEE) Any software development technique that includes structured design and results in the development of structured programs. more specific entries. crossreference generators. or simulation may be in. Examples include checkers. design review. statement coverage. (IEEE) (1) A condition or mode of existence that a system. A structured software design technique. Originally developed for IBM mainframes. transaction analysis. the transitions. (IEEE) Any disciplined approach to software design that adheres to specified rules based on principles such as modularity. symbolic execution. (1) (NBS) Analysis of a program that is performed without executing the program. compilers. Note: The result is not necessarily the same as that shown in a call graph. See: structured design. (IEEE) A diagram that identifies modules. (IEEE) A diagram that depicts the states that a system or component can assume. state diagram. Contrast with dynamic analysis. standards enforcers.. code inspection. object oriented design. software engineering. program structure chart. A-52 Version 6. A language used to interrogate and process data in a relational database. code review. system structure. top-down design.

subroutine trace. rather than actual values for input data. timed intervals. See: execution trace. subroutine. timing dependent. Equipment within the system is kept in step on the basis of this timing. tools. The elements of this composite entity are used together in the intended operational or support environment to perform a given task or achieve a specific purpose.e. See: requirements phase.Vocabulary stub. materials. facilities. syntax. components. Version 6. functional design. subprogram. (IEEE) A static analysis technique in which program execution is simulated using symbols. variable trace. i. The person that is charged with the overall administration. (ISO) A process of defining the hardware and software architecture. and other utilities. e. (ISO) A systematic investigation of a real or planned system to determine the functions of the system and how they relate to each other and to any other system. and data for a system to satisfy specified requirements. See: execution trace. Syn: system manager. Note: This term is defined differently in various programming languages. support software. architectural design. The System Administrator is normally an employee or a member of the establishment. modules. retrospective trace. system analysis. optionally. The structural or grammatical rules that define how symbols in a language are to be combined to form words. the values of parameters passed to and returned by each subroutine or function. system administrator. phrases.2 A-53 . symbolic trace. (2) (DOD) A composite. synchronous. symbolic execution. or mission requirement. and other allowable constructs. at any level of complexity. See: coroutine. expressions. (IEEE) A separately compilable. (NBS) Special code segments that when invoked by a code segment under test will simulate the behavior of designed and specified modules not yet constructed. Note: This term is defined differently in various programming languages. and software.. (1) (ANSI) People. and methods organized to accomplish a set of specific functions. (IEEE) A record of the source statements and branch outcomes that are encountered when a computer program is executed using symbolic. main program. Syn: call trace. loaders. equipment. machines. procedures. subroutine. (IEEE) Software that aids in the development and maintenance of other software. subroutine trace. See: design phase. such as variable names. routine. and operation of a computer system. Contrast with asynchronous transmission. synchronous transmission. (IEEE) A record of all or selected subroutines or function calls performed during the execution of a computer program and. of personnel. retrospective trace. variable trace. and program outputs are expressed as logical or mathematical expressions involving these symbols. system. See: module. A method of electrical transfer in which a constant time interval is maintained between successive bits or characters. interfaces. Occurring at regular.g. rather than actual values for input data. executable component of a computer program. (IEEE) A routine that returns control to the program or subprogram that called it. system design. support. compilers. symbolic trace.

acquisition. and modification of a system. Approximately one trillion bytes. system life cycle. The course of developmental changes through which a system passes from its conception to the termination of its use. limitations. (2) (IEEE) Software designed to facilitate the operation and maintenance of a computer system and its associated programs. usually equipped with a CRT display and keyboard. terabyte. A device.627. gigabyte. The devices and functions relating to transmission of data between the central processing system and remotely located users. software safety requirements analysis. See: specification. used to send and receive information to and from a computer via a communication channel. manufacturing considerations.independent software that supports the running of application software. software safety code analysis. development. criteria. terminal. Linear magnetic storage hardware. design. telecommunication system. (ISO) The progressive linking and testing of system components into a complete system. (IEEE) A review conducted to evaluate the manner in which the requirements for a system have been allocated to configuration items. e. operating systems. design. See: kilobyte. user's guide.2 . transmission control protocol/Internet protocol. (1) (ISO) Application. time. operation. utilities. system software..g. software safety change analysis. software safety design analysis. See: incremental integration. system integration. test documentation. e. See: software life cycle. capabilities. system manager. operation. Contrast with application software. precisely 240 or 1. A-54 Version 6.099.776 bytes. the system engineering process that produced the allocation. tape. system safety. megabyte. -TTB. terabyte. assemblers. software engineering. the engineering planning for the next phase of the effort. See: design review. See: support software. and maintenance of an information processing system. See: system administrator. TCP/IP. test. rolled onto a reel or cassette.511.g. system documentation.. integration. software safety test analysis. and the planning for production engineering. (DOD) The application of engineering and management principles. (ISO) The collection of documents that describe the requirements. and cost throughout all phases of the system life cycle.Guide to the CASQ CBOK system design review. maintenance. the phases and activities associated with the analysis. and techniques to optimize all aspects of safety within the constraints of operational effectiveness. See: risk assessment.

cause effect graphing. validation protocol. (IEEE) The period of time in the software life cycle in which the components of a software product are evaluated and integrated. decision coverage. test generator. and the software product is evaluated to determine whether or not requirements have been satisfied. test design.2 A-55 . Syn: test data generator. and report test results. test readiness review. and any risks requiring contingency planning. boundary value analysis. (IEEE) (1) The degree to which a system or component facilitates the establishment of test criteria and the performance of tests to determine whether those criteria have been met. often. the testing tasks. multiple-condition coverage. (IEEE) A document reporting on any event that occurs during testing that requires further investigation. and schedule of intended testing activities. operation. test criteria. (IEEE) An activity in which a system or component is executed under specified conditions. condition coverage. testing. the results are observed or recorded and an evaluation is made of some aspect of the system or component. predicted results. (NIST) A formal document developed from a test plan that presents detailed instructions for the setup. test driver. (IEEE) A software module used to invoke a module under test and. See: test driver. error guessing. It identifies test items. test documentation. test log. Types include test case specification. See: measurable. approach. test incident report. (2) The degree to which a requirement is stated in terms that permit establishment of test criteria and performance of tests to determine whether those criteria have been met. (IEEE) A software tool that accepts as input source code. Syn: test case specification. responsibilities. test phase. determines expected results. test procedure. See: test case. (IEEE) Documentation specifying the scope. or results of. the testing of a system or component. provide test inputs. the features to be tested. branch analysis. and. path analysis. test plan. See: failure analysis. and a set of execution conditions for a test item. test plan. resources. test procedure. (IEEE) A software item which is the object of testing. structural. (IEEE) Documentation describing plans for. test case generator. Syn: test harness. specifications.Vocabulary test. test report. test harness. test case. test incident report. and evaluation of the results for each defined test. See: test procedure. test item. control and monitor execution. resources. to verify that the test procedures for each configuration item Version 6. (IEEE) Documentation specifying inputs. (IEEE) (1) A review conducted to evaluate preliminary test results for one or more configuration items. See: test design. or data structure definitions. statement coverage. testability. See: testing functional. equivalence class partitioning. uses these inputs to generate test input data. test log. sometimes. (IEEE) Documentation specifying the details of the test approach for a software feature or combination of software features and identifying the associated tests. (IEEE) A chronological record of all relevant details about the execution of a test. required.

See: branch coverage. in an environment not controlled by the developer. testing. testing. testing. each possible branch [outcome] be executed at least once. (2) The process of analyzing a software item to detect the differences between existing and required conditions. A testing technique using input values at. statement. 100%. i. (2) A review as in (1) for any hardware or software component. (NBS) A dynamic analysis technique which inserts assertions about the relationship between program variables into the program code. exhaustive. an investigation should be conducted to assess possible comparability problems between the new software and other programs or systems. stress. A-56 Version 6. integration. boundary value. formatting. test result analyzer. comply with test plans and descriptions. just below. formal qualification review. compatibility. and with input values causing outputs to be at. path. (IEEE) Testing conducted to determine whether or not a system satisfies its acceptance criteria and to enable the customer to determine whether or not to accept the system. testing. software engineering. Contrast with testing. acceptance. component. and satisfy test requirements.e. The truth of the assertions is determined as the program executes. The process of determining the ability of two or more systems to exchange information. See: different software system analysis. See: testing. observing or recording the results. qualification. requirements review. (2) For medical device software such use may require an Investigational Device Exemption [IDE] or Institutional Review Board [IRB] approval. See: boundary value analysis. A software tool used to test output data reduction. testing. static analysis. testing. and making an evaluation of some aspect of the system or component. bugs. design review. alpha []. (Pressman) Acceptance testing performed by the customer in a controlled environment at the developer's site. testing. (1) (Pressman) Acceptance testing performed by the customer in a live application of the software. testing. operational. testing.Guide to the CASQ CBOK are complete. See: dynamic analysis. test report. In a situation where the developed software replaces an already working program. (IEEE) (1) The process of operating a system or component under specified conditions. and printing. and to evaluate the features of the software items. testing. testing. and just above. beta []. (NBS) Testing technique to satisfy coverage criteria which require that for each decision point. at one or more end user sites. assertion. the defined limits of an input domain. See: assertion checking. just below. testing. and just above. and to verify that a project is prepared to proceed to formal testing of the configuration items. Contrast with testing. testing. interface. See: testing. See: testing. testing. development.2 . branch. The software is used by the customer in a setting approximating the target environment with the developer observing and recording errors and usage problems. unit. instrumentation. (IEEE) A document describing the conduct and results of the testing carried out for a system or system component. the defined limits of an output domain. testing. Contrast with code review.

to evaluate their interactions. (IEEE) Testing conducted in accordance with test plans and procedures that have been reviewed and approved by a customer. One path from each class is then tested. condition coverage. testing. operational. (IEEE) Testing conducted during the development of a system or component. See: testing. (IEEE) A testing methodology in which two or more program mutations are executed using the same test cases to evaluate the ability of the test cases to detect differences in the mutations. integration. system. design based functional. Syn: black-box testing. testing. branch. Contrast with testing. formal. Antonym: informal testing. exhaustive. path. testing. testing. Contrast with testing. unit. Often paths through the program are grouped into a finite set of classes. interface. interphase. testing. See: testing. (IEEE) Testing conducted to evaluate a system or component in its operational environment. usually in the development environment by the developer. testing. hardware elements. abnormal. See: testing. testing. statement coverage.2 A-57 . multiple condition coverage. statement. invalid case. or designated level of management. until the entire system has been integrated. (NBS) Testing to satisfy coverage criteria that each logical path through the program be tested. (IEEE) (1) Testing that ignores the internal mechanism or structure of a system or component and focuses on the outputs generated in response to selected inputs and execution conditions. testing. operational. Contrast with testing. testing. development. testing. branch coverage. (IEEE) Testing conducted to evaluate whether systems or components pass data and control correctly to one another. Version 6. testing. (2) Testing conducted to evaluate the compliance of a system or component with specified functional requirements and corresponding predicted results. or both are combined and tested. A testing technique using erroneous [invalid. interface. testing. development. functional. See: equivalence class partitioning. acceptance. testing. Contrast with testing. integration. user. testing. Feasible only for small. decision coverage. The other system is considered as the standard of comparison. Syn: parallel run. (ISO) Testing a new or an altered data processing system with the same source data that is used in another system. simple programs. or unexpected] input values or conditions.Vocabulary testing. See: testing. system. mutation. (NBS) Executing the program with all possible combinations of values for program variables. structural. Contrast with testing. input/output driven testing. parallel. functional. testing. Syn: path coverage. (NBS) The application of test data derived through functional analysis extended to include design functions as well as requirement functions. acceptance. (IEEE) An orderly progression of testing in which software elements. testing.

testing. See: testing. Syn: component testing. testing. multiple condition coverage. structural. A-58 Version 6. (1) (NIST) Testing of a module for typographic. unit. to demonstrate that the software meets its specified requirements. "0". Syn: most appropriate challenge conditions. testing. or a collection of software elements. Are the communication device(s) designed in a manner such that the information is displayed in a understandable fashion enabling the operator to correctly interact with the system? testing. Tests designed to evaluate the machine/user interface. testing. statement. and logical errors. storage. (NIST) Testing to satisfy the criterion that each statement in a program be executed at least once during program testing. testing. system. Testing which encompasses upper and lower limits. stress. A testing technique using input values that seem likely to cause program errors. worst case.2 . branch coverage. path. NULL. testing. usability. See: error guessing.. (1) (IEEE) Testing that takes into account the internal mechanism [structure] of a system or component. acceptance. branch. (IEEE) Testing conducted to evaluate a system or component at or beyond the limits of its specified requirements. Testing designed to challenge a system's ability to manage the maximum amount of data over a period of time. A testing technique using valid [normal or expected] input values or conditions. glass-box testing.. e. empty string. (IEEE) The process of testing an integrated hardware and software system to verify that the system meets its specified requirements. valid case. special case. regression. testing. (NIST) Rerunning test cases which a program has previously executed correctly in order to detect errors spawned by changes or corrections made during software development and maintenance.Guide to the CASQ CBOK testing. (2) Testing to insure each program statement is made to execute during testing and that each program statement performs its intended function. path coverage. path testing. e. (2) (IEEE) Testing conducted to verify the implementation of the design for one software element. Syn: testing. See: equivalence class partitioning. This is a determination of whether or not certain processing conditions use more storage [memory] than estimated. syntactic. Contrast with functional testing. decision coverage. Types include branch testing. a unit or module. volume. testing. testing. Contrast with testing. usually conducted by the developer for the consumer. qualification. performance. testing. and for satisfaction of its requirements.g. for correct implementation of its design. (IEEE) Functional testing conducted to evaluate the compliance of a system or component with specified performance requirements. logic driven testing. statement testing. Syn: white-box testing. system.g. This type of testing also evaluates a system's ability to handle overload situations in an orderly fashion. Such testing may be conducted in both the development environment and the target environment. testing. boundary value. and circumstances which pose the greatest chance finding of errors. testing. (IEEE) Formal testing. "1". condition coverage. testing. Syn: statement coverage. testing.

e. (IEEE) A software tool that estimates or measures the execution time of a computer program or portion of a computer program. (2) To produce a record as in (1).. completeness. and accuracy. testing. (2) The degree to which each element in a software development product establishes its reason for existing.2 A-59 . top-down design. or other scheduling methods. consistency. and memory allocation.g. Types include execution trace. subroutine trace. the degree to which each element in a bubble chart references the requirement that it satisfies.g. priority-based interrupts. that transmits pressure information to the software. showing the sequence of instructions executed. an input device. time sharing. traceability matrix. (IEEE) The tracing of (1) Software Requirements Specifications requirements to system requirements in concept documentation. Version 6. special case. Analyze identified relationships for correctness. testing. a touch sensitive keypad or screen.Vocabulary See: testing. e. traceability matrix. timing. touch sensitive. testing. boundary value.. May be implemented by time slicing. (3) To establish a relationship between two or more products of the development process. to establish the relationship between a given requirement and the design element that implements that requirement. touch screen. the names and values of variables. traceability. See: structured design. See: traceability. traceability analysis. (IEEE) A mode of operation that permits two or more users to execute computer programs concurrently on the same computer system by interleaving the execution of their programs. (IEEE) Analysis of the safety implications of safety-critical requirements that relate to execution time. invalid case. e. Pertaining to design methodology that starts with the highest level of abstraction and proceeds through progressively lower levels. (2) software design descriptions to software requirements specifications and software requirements specifications to software design descriptions... See: consistency. retrospective trace. stress. (IEEE) The process of estimating or measuring the amount of execution time required for a software system or component. timing and sizing analysis. timing analyzer. trace. testing. pencil. the degree to which the requirements and design of a given software component match. (IEEE) (1) The degree to which a relationship can be established between two or more products of the development process. e. or other object. (IEEE) (1) A record of the execution of a computer program. (ANSI) Pertaining to a device that allows a user to interact with a computer system by touching an area on the surface of the device with a finger. Contrast with sizing. The panel is a matrix of cells. symbolic trace. especially products having a predecessorsuccessor or master-subordinate relationship to one another. A touch sensitive display screen that uses a clear panel over or on the screen surface. clock time. (3) source code to corresponding design specifications and design specifications to source code. either by summing the execution times of the instructions along specified paths or by inserting probes at specified points in the program and measuring the execution time between probes. or both.g. variable trace. volume. See: traceability analysis.g.

(Beizer) A model of the structure of the system's [program's] behavior. See: traceability. See: bomb. A tool that instruments a program to obtain execution frequencies of statements is a tool with this feature. translation.e. A structured software design technique. typically by providing a useful program which contains code intended to compromise a computer system by secretly providing for unauthorized access. i. transmission control protocol/Internet protocol. (IEEE) A matrix that identifies possible requests for database access and relates each request to information categories or elements in the database. and agencies of the federal government. transaction. the output value. and indicating. twisted pair. The TELENET protocol provides a terminal emulation capability that allows a user to interact with any other type of computer in the network. such as updating a file. worm. transaction matrix. virus. deriving the structure of a system from analyzing the transactions that the system is required to process. (NIST) Determining what parts of a program are being executed the most.2 . a modification. message. tuning.Guide to the CASQ CBOK traceability matrix. See: assembling. The File Transfer Protocol and Simple Mail Transfer Protocol provide file transfer and electronic mail capability. traceability analysis. or a deletion of one or more data elements of a storage structure. A structured software design technique in which system structure is derived from analyzing the flow of data through the system and the transformations that must be performed on the data. and the IP protocol provides the routing mechanism. A set of communications protocols developed for the Defense Advanced Research Projects Agency to internetwork dissimilar systems. The TCP protocol controls the transfer of the data. for each combination. the performance of unintended and unexpected functions. almost all American universities. A pair of thin-diameter insulated wires commonly used in telephone wiring. the unauthorized collection of privileged system or user data. compilation. functionality. (2) A table that describes a logic function by listing all possible combinations of input values. trojan horse. (NIST) Converting from one language form to another. interpret... (1) (ISO) An operation table for a logic operation. The wires are twisted around each other to minimize interference from other twisted pairs in the A-60 Version 6. or the malicious destruction of software and hardware. (2) An exchange between and end user and an interactive system. a matrix that records the relationship between the requirements and the design of a given software component. It is used by many corporations. the unauthorized reading or altering of files. (IEEE) A matrix that records the relationship between two or more products. transaction flowgraph. or input record that explicitly or implicitly calls for a processing action. a unit of processing activity that accomplishes a specific purpose such as a retrieval. an update. (ANSI) (1) A command. truth table. e. transaction analysis.g. A method of attacking a computer system. (3) In a database management system. transform analysis.

Abbreviated UTP for Unshielded Twisted Pair. -VV&V. (ISO) Documentation that describes how to use a functional unit. Syn: twisted wire pair. Syn: user manual. Syn: service program. user. Version 6. multiple-user (time-sharing) operating system developed at Bell Labs to create a favorable environment for programming research and development. (IEEE) An exception that occurs when the result of an arithmetic operation is too small a fraction to be represented by the storage location designated to receive it. (IEEE) The ease with which a user can learn to operate. unit. by the operating system. module. utility software. (2) A logically separable part of a computer program. utility program. and interpret outputs of a system or component.0000. a sort program. virtual address extension. (IEEE) (1) A separately testable element specified in the design of a computer software element. certain.2 A-61 . and that may include description of the rights and responsibilities of the user. -Uunambiguous. underflow. the owner. VAX. UNIX. (4) Clear. See: end user. not vague. (ISO) The state in which a calculator shows a zero indicator for the most significant part of a number while the least significant part of the number is dropped. For example. organization. or functional unit that uses the services of an information processing system. and the supplier of the unit.. user's guide. e. (IEEE) Computer programs or routines designed to perform some general support function required by other application software. very large scale integration. prepare inputs for. if the calculator output capacity is four digits. operator manual. a diagnostic program. See: arithmetic underflow.Vocabulary cable. (ANSI) Any person. verification and validation. making copies of files.0000432 will be shown as . VLSI. (3) Not obscure. underflow exception. a trace program. definite. (2) Not susceptible to different interpretations. usability. (ISO) A computer program in general support of the processes of a computer. A multitasking. or deleting files.g. Syn: component. Twisted pairs have less bandwidth than coaxial cable or optical fiber. See: utility software. (1) Not having two or more possible meanings. the number . They perform general functions such as formatting electronic media. or by the system users.

Validation is usually accomplished by verifying each stage of the software development life cycle. production equipment. subroutine trace. verification. Test data is useful only if the methods and results are adequately specific. symbolic trace. or product made under a revised manufacturing process. validation. prospective.2 . (2) Retrospective validation can also be useful to augment initial premarket prospective validation for new products or changed processes. validation. value trace. (NIST) Used as an entity to define a procedure of review. and testing throughout the software life cycle to discover errors. validation. valid. product characteristics.Guide to the CASQ CBOK VMS. Contrast with constant. A-62 Version 6. validation. (FDA) (1) Validation of a process for a product already in distribution based upon accumulated production. Contrast with data validation. testing and control data. See: execution trace. (NBS) Determination of the correctness of the final program or software produced from a development project with respect to the user needs and requirements. valid input. (FDA) Validation conducted prior to the distribution of either a new product. Syn: data-flow trace. To prove to be valid. validation protocol. label. variable trace. where the revisions may affect the product's characteristics. it is important that the test methodology be qualified to assure that the test results are objective and accurate. verification. retrospective trace. validate. process. analysis. software. virtual memory system. (FDA) Establishing documented evidence which provides a high degree of assurance that a specific process will consistently produce a product meeting its predetermined specifications and quality characteristics. A name. (1) (FDA) Establishing documented evidence which provides a high degree of assurance that a specific process will consistently produce a product meeting its predetermined specifications and quality attributes. validation. and ensure the production of quality software. validation. (3) Able to withstand criticism or objection. See: verification. and testing. (FDA) A written plan stating how validation will be conducted. including test parameters. software. (1) Sound. Whenever test data are used to demonstrate conformance to specifications. VV&T. or data item whose value may be changed many times during processing. and decision points on what constitutes acceptable test results. (IEEE) A record of the name and values of variables accessed or changed during the execution of a computer program. (2) Well grounded on principles of evidence. quantity. data trace. retrospective. variable. and testing. validation. determine functionality. (NBS) Test data that lie within the domain of the function represented by the program. See: test plan.

walkthrough. See: code walkthrough.Vocabulary vendor. virus. test phase. Identifies Digital Equipment Corporation's VAX family of computers. design phase. software. virtual memory system. version number. An initial release or a complete re-release of a software item or software element. A program which secretly alters other programs to include a copy of itself. A unique identifier used to identify software items and the related software documentation which are subject to configuration control. and correctness of the software at each stage and between each stage of the development life cycle. waterfall model. (3) (Webster) To prove to be true by demonstration. installation and checkout phase. e. watchdog timer. ranging from a desktop workstation to a large scale cluster of multiprocessors supporting thousands of simultaneous users.000 transistors. (IEEE) A model of the software development process in which the constituent activities.g. (ANSI) (1) To determine whether a transcription of data or other operation has been accomplished accurately. (ANSI) A portion of data. and operation and Version 6. virtual address extension. that can be handled conveniently as a unit. A person or an organization that provides software and/or hardware and/or firmware and/or documentation to the user for a fee or in exchange for services. software. implementation phase. very large scale integration. See: bomb. requirements phase. A VLSI IC contains 100.000 to 1. verification. a reel of magnetic tape. interactive operating system for the VAX computers. verifiable. Can be proved or confirmed by examination or investigation. a floppy disk. trojan horse. See: release.2 A-63 . typically a concept phase. completeness. verify. a disk pack. The execution of a virus program compromises a computer system by performing unwanted or unintended functions which may be destructive. together with its data carrier. wide area network. e. and executes when the host program is executed.g.. Digital Equipment Corporation's multiprocessing. -WWAN. Such a firm could be a medical device manufacturer. (2) To check the results of data entry. A classification of ICs [chips] based on their size as expressed by the number of circuits or logic gates they contain. worm. See: validation. (IEEE) A form of interval timer that is used to detect a possible malfunction. (NBS) In general the demonstration of consistency..000. See: measurable. keypunching. volume. version.

They do not change other programs. See: bomb. -XXmodem. rapid prototyping. and responds well to changing line conditions due to its variable length blocks. Zmodem. but compromise a computer system through their impact on system performance. Xmodem-1K-G transmits without acknowledgment [for error free channels or when modems are self correcting]. white-box testing. possibly with overlap but with little or no iteration. They may include manual procedures used in conjunction with the computer system. Ymodem. An asynchronous file transfer protocol that is more efficient than Xmodem. Zmodem. Ymodem. Contrast with LAN. Programs typically include both methods and drop back to checksum if CRC is not present at the other end. workstation. word. It uses CRC error correction and is effective in delay-induced satellite transmission. An asynchronous file transfer protocol identical to Xmodem-1K plus batch file transfer [also called Ymodem batch]. but transmission is cancelled upon any error.2 . An independent program which can travel from computer to computer across network connections replicating itself in each computer. MAN. Xmodem transmits 128 byte blocks. Ymodem-G transmits without acknowledgement [for error-free channels or when modems are self correcting]. An asynchronous file transfer protocol initially developed for CP/M personal computers. Xmodem. Contrast with incremental development. trojan horse. Any terminal or personal computer. wide area network. are performed in that order. Contrast with Kermit. See: computer word. A sequence of actions the user should take to avoid a problem or system limitation until the computer program is changed. First versions used a checksum to detect errors. It sends file name. virus. structural. Contrast with Kermit. See: testing. but transmission is cancelled upon any error. A communications network that covers wide geographic areas such as states and countries. A-64 Version 6. Later versions use the more effective CRC method. date and size first. spiral model. worm. workaround. -YYmodem.Guide to the CASQ CBOK maintenance. Contrast with Kermit. -ZZmodem. Xmodem-1K improves speed by transmitting 1024 byte blocks. Xmodem.