You are on page 1of 29

DEPARTMENT OF THE AIR FORCE Thomas N. Barnes Center for Enlisted Education (AETC) Maxwell AFB, AL 36118

1 Oct 13

NONCOMMISSIONED OFFICER ACADEMY STUDENT GUIDE

PART I COVER SHEET

LESSON TITLE: UM10, CONTINUOUS IMPROVEMENT

TIME: 5 Hours METHOD: Guided Discussion/Experiential Exercise REFERENCES:

Air Force Instruction (AFI) 36-2618. The Enlisted Force Structure, 1 December 2004. Chang, Richard Y. Step-By-Step Problem Solving, Irvine, CA: Richard Chang Associates Inc., publications Division, 1993.

Department of the Air Force. Air Force Smart Operations for the 21st (AFSO21) Century Playbook, 27 May 2008.

Department of the Air Force. United States Air Force Core Values, 1 January 1997. George, Michael. Lean Six Sigma for Service. New York, NY: McGraw-Hill, 2003. Kneeland, Steven. Effective Problem Solving: How to Understand and Process and Practice it Successfully. Oxford, UK: How to Books, 1999. NetLibrary e-book. Mackall, Dandi D. Problem Solving. Chicago, IL: Ferguson Publishing Company, 1998. NetLibrary e-book. Newman, Victor. Problem Solving for Results. Hampshire, UK: Gower Publishing Limited, 1995. VanGundy, Arthur B. 101 Activities for Teaching Creativity and Problem Solving. San Francisco, CA: John Wiley and Sons, Inc., 2005. NetLibrary e-book.

STUDENT PREPARATION:

1. Read student guide (9,237 words, approximately 80 minutes)

PART IA
PART IA

GENERAL LEARNING OUTCOME: Students who graduate from the NCOA are better prepared to Lead and Manage Organizations and Resources as evidenced by their comprehension of Continuous Improvement.

SUPPORTED COMPETENCIES/DIRECTIVES:

The Continuous Improvement lesson supports the following Air Force Institutional Competencies:

Managing Organizations and Resources

  • a. Resource Stewardship

  • b. Change Management

  • c. Vision

Strategic Thinking

  • a. Vision

  • b. Adaptability

Fostering Collaborative Relationships

  • a. Builds Teams and Coalitions

  • b. Negotiating

The Continuous Improvement lesson:

  • - Provides the necessary information NCOs need to execute their responsibilities outlined in AFI 36-2618, The Enlisted Force Structure effectively.

  • - Supports the Air Force Core Values.

  • - Supports the Chairman of the Joint Chiefs of Staff Instruction (CJCSI) 1805.01A, Learning Area 6, Operational Leadership which enables leaders to:

Comprehend the ethical dimension of operational leadership and the challenges it may present.

Recognize the skills required of the senior enlisted personnel when leading personnel to include the training and development of subordinates, an understanding of standards and service cultures, and various stressors that affect the force.

TERMINAL COGNITIVE OBJECTIVE: Comprehend Continuous Improvement and its impact on mission effectiveness.

TERMINAL COGNITIVE SAMPLES OF BEHAVIOR:

  • 1. Explain how Continuous Improvement impacts mission effectiveness.

  • 2. Give examples of how Continuous Improvement impacts mission effectiveness.

  • 3. Predict how Continuous Improvement impacts mission effectiveness.

AFFECTIVE OBJECTIVE: Value Continuous Improvement

UM10SG - 2

PART IB
PART IB

ORGANIZATIONAL PATTERN: Topical

LESSON OUTLINE:

CONTENT

INTRODUCTION Attention, Motivation and Overview

MP 1. Continuous Improvement

MP 2. Decision

MP 3. Problem Solving

MP 4. Scenario

CONCLUSION: Summary, Remotivation, and Closure

UM10SG - 3

PART II STUDENT READING

MP 1. Continuous Improvement

As NCOs, we should be looking and thinking daily about innovative ways to use our resources more efficiently. According to our core values, we must pursue excellence in all we do.

One way we can do this is through a culture of Continuous Improvement or “CI” which is the strategic, never-ending, incremental refinement of the way we perform tasks. CI helps to shed light on those “non-value added” tasks to ensure every Airman’s efforts contribute directly to accomplishing the Air Force mission to fly, fight, and win in air, space, and cyberspace.

Air Force Smart Operations For The 21 st Century (AFSO21)

In pursuit of continuous improvement, the Air Force created Air Force Smart Operations for the 21 st Century (AFSO21) which focuses on generating efficiencies and improving combat capabilities across the Air Force and applies to all processes associated with the Air Force mission. Governed by proven process improvement techniques, AFSO21 has already significantly increased Air Force combat capability and it will continue to do so throughout the 21 st century. 1 AFSO21 principles and tools enable Airmen to integrate continuous improvement processes and methodologies into the full spectrum of their day- to-day operations.

The key to AFSO21 success is a culture where every Airman thinks about improvement, and is empowered to communicate with his or her supervisor, commander or a change agent. Even processes that work well can be better. Improvements center on core missions that Airmen handle daily and should encompass AFSO21’s Five Desired Effects

AFSO21’s Five Desired Effects

The Five Desired Effects guide improvement initiatives at every level to contribute to the demands of the Warfighter. Every Airman should know and understand these five desired effects and understand how they improve processes that contribute to Air Force priorities while also generating efficiencies and savings. The AFSO21 Five Desired Effects are:

  • 1. Increase productivity of our people: Doing more of the right things with the

same or less effort.

  • 2. Increase critical equipment availability rates: Improving asset availability in all

areas of warfare (i.e. air, space, and cyberspace) more expeditiously.

  • 3. Improve response time and agility: Quicker response time to the Warfighter.

  • 4. Sustain safe and reliable operations: Reduce injury rates, increase personnel

safety and safe use of materiel assets.

  • 5. Improve energy efficiency: Make energy conservation a consideration in

everything we do.

UM10SG - 4

In addition to the Five Desired Effects, AFSO21 also includes three levels of priorities.

AFSO21’s Three Levels of Priorities

Continuous process improvements come in different forms. Some processes are quick and simple fixes; others are complex and may involve several organizations working

improvement plans over a lengthy period. AFSO21’s three main categories of process

improvement actions are:

  • 1. Just Do It

  • 2. Rapid Improvement Events (RIE)

  • 3. High Value Initiative (HVI)

Just Do It

You are probably thinking this is a familiar slogan for a famous athletic shoeit is, but it means something entirely different in the AFSO21 world. Just do itis a quick fix to a process irritant; a simple answer to an obstacle in an individual process. A “Just Do It” typically does not involve formal process reviews, teams, or an improvement event. It is an improvement that, when implemented, yields immediate results.

Rapid Improvement Events (RIE)

Rapid Improvement Events usually last a week and apply a series of problem solving steps to determine root causes of problems, eliminate waste, set improvement targets and establish clear performance measures to reach desired effects.

Successful RIEs usually have the following four components:

  • - Strong leadership buy-in

  • - Knowledgeable and open-minded participants

  • - A tightly focused event scope

  • - An implementation plan and metrics to track results

High Value Initiative (HVI)

High Value Initiatives produce significant returns against key Air Force challenges. These processes are more complex and involve a cross functional team to ensure that identified improvements are incorporated into the day-to-day operations of an organization. HVIs typically require four to six months in order to successfully define and implement the required process changes.

Understanding AFSO21’s Five Desired Effects and its Three Levels of Priorities are important, but neither is much good without sound problem solving.

UM10SG - 5

MP 2. DECISION ANALYSIS

"We can't solve problems by using the same kind of thinking we used when we created them."

Definition of Decision Making

- Albert Einstein

Decision making is the mental (cognitive) process that results in the selection of a course

of action from among several alternative scenarios.” 2

Every decision-making process

produces a final choice. The output of which can be an action or an opinion of choice.

Two System Approach to Decision Making

If you look back to the days when you were a brand new airman just learning your job for the first time, more than likely you had to really focus on the tasks that you were doing.

Now through repetition, tasks you once thought of as “complex” are now nothing more

than routine actions, decisions, and behaviors. It is only after many hours of complex analysis, inferences and effective judgments that tasks become routine enough as to require little or no thought. As long as you can do things the same way, every time, decisions seem to come easy. However, what happens when things change? As discussed in the Change Management lesson, we are often uncomfortable with change because change takes us out of our “norm.” When we move to something new or different, we are required to consider new ways of doing things. The old ways of doing things were reactive, instinctive, quick and holistic (System-1), the new ways are more deliberative, analytical and procedural

(System-2).

System-1 (Reactive Thinking)

Reactive Thinking (System-1) relies heavily on situational cues, prominent memories, trial and error and heuristic thinking (discovering solutions for self) to arrive quickly and confidently at judgments, particularly when situations are familiar and immediate action is required. Many of the judgments that you make every day are automatic or reactive, rather than reflective. When you wake up in the morning and go to work, chances are that unless something dramatic happens, you are on “auto pilot” until you get to work. You probably do not spend a whole lot of time thinking about how to brush your teeth, how to eat breakfast or how to drive to work. Many freeway accidents are often avoided because drivers are able to see and react to dangerous situations quickly. Good decisions emerging from System-1 thinking often feel intuitive. Decisions good drivers make in those moments of crisis, just like the decisions practiced athletes make in the flow of the game or the decisions an NCO makes in the heat of battle, are born of expertise, training, and practice. Often the process of reactive thinking involves deciding first, reacting and then trying to make sense out of all of it. Many times if you make a decision based entirely from reactive thinking you may look back and ask yourself, What was I thinking?”

System-2 (Reflective Thinking)

Reflective Thinking (System-2) is broad and informed problem-solving and deliberate decision making. It is useful for judgments in unfamiliar situations, for processing abstract

UM10SG - 6

concepts, and for deliberating when there is time for planning and more comprehensive consideration. Argument making is often part of the deliberation process when making System-2 decisions. Critical thinking is considered System-2 thinking because it is often focused on resolving the problem at hand and at the same time monitoring and self- correcting the process of the situation or problem. If you recall, in the successful learning lesson, we discussed reflective journaling and the importance of reflective thinking. In that lesson you understood that the reflection part of this process begins with a state of doubt, hesitation or perplexity and moves through the act of searching for information that will resolve, clarify or address a situation or problem.

As you think about a two-system approach to decision making, do not mistake the process as a “head–versus-heart” or a “right brain-versus-left brain” approach. Human decision- making is not this superficial or simplistic. Likewise, do not categorize individuals as System-1 or System-2 decision makers. We have and use both systems in problem solving and decision making every day. The most important thing to take out of this is that everyone deals with the push and pull of both systems many times while making decisions. The other important thing to take away from these concepts is the fact that rapid-fire decisions and slow deliberate decisions are processes that have steps and require practice.

Using the Appropriate Systems Thinking for Decision Making

If you have ever heard (or maybe said), “that is the way we have always done things” it is

because too often, many of today’s problems are solved by utilizing easy and comfortable approaches to obtain a solution. In reality as you may have discovered, simple and common approaches are not always the most effective way of dealing with complex, dynamic and diverse problems. In your required reading, you have a chance to read about two incidents that occurred in the Air Force involving the improper handling of nuclear weapons and materials. Can you think of an example where the more appropriate system may have been neglected or possibly overlooked altogether?

As a NCO and member of the Profession of Arms, there is an increasing need to improve and create impeccable results through systems thinking. In essence, system thinking is a discipline of seeing the “whole,” recognizing patterns and interrelationships and learning how to structure more effective, efficient decisions.

Many reactive judgments can be good judgments, but can lead to unnecessary risks and mistaken biases. Thus, the true “decision” is which of our reactive judgments should we make reflective?

Kepner-Tregoe Problem Solving and Decision Making Process

One of the most used decision making models in the world was developed in 1958 by Dr. Charles Kepner and Dr. Benjamin Tregoe. The Kepner-Tregoe Problem Solving and Decision Making process is actually four distinct processes, each designed to address a specific type of situation:

Situation Appraisal: Used to separate, clarify, and prioritize concerns. When confusion is mounting, the correct approach is unclear, or priorities overwhelm plans, Situation Appraisal is the tool of choice.

Problem Analysis: Used to find the cause of a positive or negative deviation by conducting a problem analysis. Through this analysis, we may find people,

UM10SG - 7

machinery, systems, or processes that are not performing as expected. Problem Analysis points to the relevant information and leads the way to the root cause.

Decision Analysis: Used for making choices. When the path ahead is uncertain, when there are too many choices, or the risk of making the wrong choice is high, Decision Analysis clarifies the purpose and balances risks and benefits to arrive at a solid and supported choice.

Potential Problem Analysis: To protect actions or plans, we would use a Potential Problem Analysis. When a project simply must go well, risk is high, or a myriad of things could go wrong; Potential Problem Analysis reveals the driving factors and identifies ways to lower risk.

Situational Appraisal, Problem Analysis and Potential Problem Analysis are accomplished when you complete the eight-step problem solving process. The decision analysis stage of the Kepner-Tregoe process is where many of individuals fall short of making good sound decisions.

Decision Analysis

Decision Analysis is a systematic procedure based on the thinking pattern that we all use when making choices. Every day we go through some sort of decision analysisshould I

eat, should I sleep, should I save this money or spend it. Often, we make decisions based

on feelings instead of using a “systematic” process.

Although people enjoy being involved in the decision making process, many avoid the task if there is a potential for controversy. Using systematic decision analysis allows us to make good, confident, reliable and justifiable decisions because it forces us to step back from the situation and evaluate the following four components:

  • - Decision Statement

  • - Determine Objectives

  • - Locate Alternatives

  • - Risk Analysis

By standing back and looking at “the whole picture” of the situation, we can often make the wisest and safest choicethrough careful consideration of ALL the factors.

Decision Statement

If you recall, in the problem solving process, one of the first things you should do is to develop a problem solving statement. In decision analysis, we will do much of the same and develop a decision statement. This statement will accomplish the following:

  • - Determine our objective

  • - Specific level of success or resolution

When we try to determine the objectives of our decision making process what we are really saying is what is our criteria or level of acceptance. We establish these objectives once we have agreed upon our decision or course of action in a decision statement. For example, if we want to buy a new car, our decision statement could be something as simple as “I need

UM10SG - 8

to buy a new car.

The second half of that decision statement must include “to what level” or “to what purpose?” If our objective was to buy a new car to what level would we want our purchase to be? The complete decision statement would sound something like “I need to buy a new car for under $15,000.”

Determine Objectives

All too often, we fail to think about the specific objective(s) of our decisions. Objectives are clear measures of the ends we want to achieve it is only through clear measures that we can make a reasonable sensible decision.

If we wanted to purchase a new car, it should be obvious that we want to make the best purchase possible and the best way to make the best purchase is to identify the qualities we want in a new car BEFORE we begin shopping. It does not make sense to buy the first car that attracts our attention, but amazingly, this happens every day.

To reach our objective(s) we must consider two categoriesour Musts and our Wants.

Note: Musts are mandatory! That is, all alternatives under consideration must meet our Musts. Though Musts may not be our most important objective, they are a minimum objective.

The Wants are optional, nice to have attributes or qualities. Wants give us a comparative picture of alternatives.

Sometimes, we are willing to compromise, move a Must to the Want category, and occasionally, move a Want to the Musts category. It all depends on what the decision is trying to accomplish. Here is an easy way to remember Musts and Wants:

The Musts are mandatory, the Wants are optional. Let’s explore a 4-step decision analysis model using TSgt Amazing’s car-buying experience. STEP 1: Decision Statement Purchase a reliable new car for the least amount of money for college-bound daughter.STEP 2: Determine Musts and Wants

From the decision statement, we can extract TSgt Amazing’s Musts such as reliable, new, and least cost. Thus his Musts may look something like:

Musts

  • - Cost less than $15,000

  • - Come with a 5-year warranty

  • - Have an automatic transmission

  • - Have earned a 5-star crash safety rating

UM10SG - 9

Wants

  • - Have cold air conditioning

  • - Be blue in color

  • - Equipped with a sun roof

  • - Have front wheel drive

  • - Come with an excellent repair record

Before moving to Step 3, TSgt Amazing will need to assign weights to the Wants.

To do

this he used a rating scale of 1 to 10 for his list of Wants1 representing least important

and 10 representing most important.

Wants

Weight

Air Conditioning

9

Blue in Color

4

Sun Roof

3

Front Wheel Drive

7

Excellent repair record

2

Some final thoughts when assigning weights to the Wants. First, always do it BEFORE proceeding to Step 3. Secondly, ensure these weights are based on the personal opinion and/or values of the person or persons making the decision.

STEP 3: Locate Alternatives/Select Best Alternative

After determining the Musts, Wants, and assigning weights to those Wants, it is time to find potential alternatives. Alternatives can come from many sources. In the car-buying scenario, TSgt Amazing might find alternatives (potential cars to purchase) at local car dealers, in newspaper ads, and online.

On the other hand, when you are in need of alternatives for solving a problem, alternatives can come from:

  • - Brainstorming with end users

  • - Research

  • - Your own creative thinking

  • - Subject-Matter Experts

UM10SG - 10

Step 3a: Locating Alternatives Let’s take a look at the five alternatives that TSgt Amazing came up with in Table1:

Alternative 1

Alternative 2

Alternative 3

Alternative 4

Alternative 5

  • - $14,599.00

$16,998.95

  • - $13,495.95

  • - $14,095.00

-

  • - $12,998.95

  • - 6-Yr Warranty

6-Yr Warranty

  • - 7-Yr Warranty

  • - 5-Yr Warranty

-

  • - 6-Yr Warranty

  • - Automatic

Automatic

Transmission

  • - Manual

Transmission

  • - Automatic

Transmission

-

Transmission

  • - Performance Automatic

Transmission

  • - Air Condition

5-Star Crash

Safety Rating

  • - Fair Repair

  • - Good Repair

Record

-

Record

  • - 5-Star Crash Safety Rating

  • - Metallic Blue

Air

Conditioning

  • - 5-Star Safety

  • - 5-Star Safety

Rating

-

Rating

  • - Bluish-Green

  • - Sunroof

Black

  • - Air

(Manual)

  • - Air

Conditioning

-

Conditioning

  • - Sunroof (Manual)

(Dual Control)

  • - FW Drive

FW Drive

  • - Red

  • - Light Blue

-

  • - FW Drive

 
  • - Sunroof

5-Star Safety

  • - FW Drive

-

  • - Air Condition

Rating

   
  • - Sunroof

FW Drive

-

 

(Electronic)

Table 1, Locating Alternatives

Step 3b: Comparing Alternatives to Musts

This step is very simple, especially when you use a comparison sheet like the one shown below. Enter Musts in the criteria column and then enter Yes or No for each alternative that fulfills a corresponding Must. The sheet places criteria and alternatives side-by-side to quickly eliminate alternatives that do not meet ALL of the Musts.

Criteria (Musts)

Alt 1

Alt 2

Alt 3

Alt 4

Alt 5

Costs $15,000 (or less)

No

Yes

Yes

Yes

Yes

5-year warranty

Yes

Yes

Yes

Yes

Yes

Automatic transmission

Yes

Yes

No

Yes

Yes

5-star crash safety rating

Yes

Yes

Yes

Yes

Yes

Table 2, Comparing the Musts

As you can see from the worksheet above, alternative 2, 4, and 5 satisfy all of the Musts. With our alternatives narrowed down to these three, our next step is to weigh the Wants of each alternative.

Step 3c: Weighting the Wants within each Alternative

Although we have narrowed our alternatives from five to three, we still need to determine which one is the best by continuing our systematic process. This next step involves some simple math and the use of another worksheet.

UM10SG - 11

First, we enter our Wants into the Criteria column and our previously determined weights into the Weight column (from Step 2).

Next, we need to consider and weigh Want items against like Want items from each alternative. For example, the first Want item in the worksheet below is Air Conditioning(or A/C). All three alternatives offer A/C; however, alternative #4 has dual control air conditioning. Based on our personal opinion (we really like dual control A/C), we might

assign a value of “7” to alternative #4 and then assign a value of “5” to air conditioning in the

#2 and #5 alternatives.

We repeat this process for each subsequent Want.

Notice the Want

Front Wheel Driveis identical in all three alternatives so we assigned the same value to each one.

 

Previously

             

Criteria (WANTs)

Determined

Weight

Alt 2

Weight

Score

Alt 4

Weight

Score

Alt 5

Weight

Score

Air Conditioning

9

   
  • 5 7

   

5

 

Blue in Color

4

   
  • 9 4

   

3

 

Sun Roof

3

 
  • 2 5

   

2

 

Front Wheel Drive

7

 
  • 5 5

   

5

 

Excellent repair record

2

 
  • 0 3

   

5

 
 

Table 3, Assigning Weights to Alternative Items

 

Step 3d: Computing Scores for Each Alternative

 

Now we perform a little math. Begin by multiplying the Previously Determined Weight by the Weight of the first item in Alternative 2 (9 x 5 = 45). Do this for all remaining items in all remaining alternatives. Next, add up the scores for each alternative column.

   

Previously

           

Criteria (WANTs)

Determined

 

Alt 2

 

Score

 

Alt 4

 

Score

Alt 5

Score

Weight

       

Air Conditioning

 

9

 

45

  • 5 (9x5)

 

63

  • 7 (9x5)

(9x7)

5

45

Blue in Color

 

4

 

36

  • 9 (4x9)

 

16

  • 4 (4x3)

(4x4)

3

12

Sun Roof

 

3

 

6

  • 2 (3x2)

 

15

  • 5 (3x2)

(3x5)

2

6

Front Wheel Drive

 

7

 

35

  • 5 (7x5)

 

35

  • 5 (7x5)

(7x5)

5

35

Excellent Repair Record

 

2

 

0

  • 0 (2X0)

 

6

  • 3 (2x5)

(2X3)

5

10

   

122

 

135

108

 

Total Scores

 

(45+36

 

(63+16

 

(45+12

 

+6+35+0)

+15+35+6)

+6+35+10)

Table 4: Figuring Total Scores

UM10SG - 12

At this point, the decision seems clear; alternative #4 scored highest and therefore, must be our BEST choice; however, there is still one very important step left: Risk Analysis.

STEP 4: Risk Analysis

The last step of Decision Analysis is when you look at your choice and determine four things:

  • - Is there anything I overlooked?

  • - Are there any potential issues as a result of a Must?

  • - Is there a possibility that my choice is invalid? If so, how severe are the implications?

  • - Is there anything that might prevent me from being comfortable with my choice?

For example, if assume that we just learned the dealer offering alternative #4 is filing for bankruptcy, then we may not be comfortable buying from the dealer. To be sure, we rate probability and seriousness of the consequence using the scale: high, medium, and low.

Alternative

Consequence

Probability

Seriousness

Alternative #4

Potential Bankruptcy

High

High

Table 5: Risk Analysis

Given the high probability and high seriousness, we would then consider our second highest scoring alternative (i.e. alternative #2) and conduct a risk analysis on it.

In some cases, as we work through Risk Analysis, we may discover a detail that is so important, we add it to our list of Musts. Anytime we change, (add to or subtract from) our list of Musts, we must work our way through Steps 3 and 4 again. This additional work is well worth the effort because it results in selecting the best alternative.

After completing the Risk Analysis step, we are now in a position to decide whether Alternative #4 (or #2 or #5) is truly our best choice.

Let us assume that because our risk analysis on Alternative 2 did not indicate any concerns, we select Alternative 2. Now, if anyone asks why we selected Alternative 2, we can provide hard data to support our decision.

Enlisted leaders do not need to run every decision they make through such an elaborate process. In fact, the Air Force depends on NCOs to apply their extensive knowledge and experience (System-1 Thinking and Decision Making) to most day-to-day decisions. However, there are times when NCOs have to make critical decisions and solve complex problems. In these cases, using a Decision Analysis process, like the one outlined above will help ensure high quality, fact-based decisions.

UM10SG - 13

MP 3. PROBLEM SOLVING

OODA LOOP

In the 1950s, Col John R. Boyd, USAF, developed an objective description of the decision making process called OODA (Observe, Orient, Decide, Act) Loop. Because of Col Boyd’s emphasis on the infinitely repeating nature of decision making, his model fully supports the concept of Continuous Improvement because both concepts support the notion that the decision making process is never actually complete. The Air Force takes the four steps of the OODA Loop and further breaks them down into an eight-step problem-solving roadmap that is flexible enough to be effective at any level: Air Force, MAJCOM, wing, squadron, and even the individual Airman. 3

Consistent application of the eight-step processvia the OODA Loopprovides Air Force leaders with a common format for presenting data, problem-solving facts, and information. It also provides a common language, which will more easily translate into a common understanding throughout the Air Force. As Air Force leaders begin to hone their understanding of how they and their organizations solve problems and make decisions, they will learn to recognize the difference between time spent constructively solving problems at the root-cause level and wasting time spinning their wheels. 4 As NCOs, problem solving is perhaps our most important skill because we make decisions and solve problems every day.

Just to be clear, the objective of the eight-step problem-solving process is to help us focus on big issues affecting our mission, our workcenter, and our people.

It is a team-centered, systematic, common-sense approach aimed at increasing combat capability, making Air Force units more effective and efficient, and enhancing and enabling the war fighter.

As resources continue to shrink (budgets, human resources, facilities, and equipment), every Airman must be mindful to get the full effect from every effort. In other words, we cannot afford to waste our time on tasks, projects, or other things that do not add direct value or positively affect our organizations, or the Air Force’s mission.

As you read this student guide, do not get too wrapped up in memorizing which steps of the eight-step process correlate with steps within the OODA Loop. In its simplest form, the OODA Loop is a process where problem solvers take a good look at the current situation and form theories about the problem (Observe), gather data and information to substantiate those theories (Orient), develop solutions to address the problem (Decide), and then implement and evaluate their solutions (Act). The real take away from this lesson is that problem solving and decision making are never-ending processes aimed at constant, continuous improvement.

UM10SG - 14

Eight-Step Problem Solving Process

Eight-Step Problem Solving Process Figure 1, 8-Step Problem-Solving Process STEP 1: Clarify & Validate the Problem

Figure 1, 8-Step Problem-Solving Process

STEP 1: Clarify & Validate the Problem (OODA)

We must clarify large, vague, and complicated problems as objectively as possible before we can identify the real problem and properly address it. 5 So knowing what to tackle first is important. This first step is critical to your success, define the problem. Remember, “A problem well stated is a problem half solved.” 6 In order to clarify the problem for everyone, you must develop a clear problem statement.

The following techniques help clarify, validate, and define the problem and they assist in deciding which problem(s) to tackle first.

Go and See

Observe first-hand what is taking place. Actually walking the process or problem area provides first hand data rather than second hand opinions. Ensure you base your information on facts, not assumptions. Assumptions are the lowest form of logic and more often than not result in faulty conclusions, which skew the process of analyzing potential causes.

Voice of the Customer (VOC)

Only one entity can define what is valuable to the customer and that is the

UM10SG - 15

customer. So identifying customers and determining their needs (document the VOC) is a prerequisite to understanding whether or not those needs are being met.

Feedback is the key to understanding customer needs. The best methods for obtaining feedback include customer surveys, hot lines, and face-to-face meetings. Customer surveys are not only a great way to collect data, they often indicate negative and positive trends. Of course, surveys are only as good as the questions they contain. Surveys should target the particular problem you are trying to solve, contain only questions related to the issue and seek unbiased feedback from a proper sample population size. Unbiased feedback helps clarify and validate customers and their concerns.

A problem statement describes the problem in clear, specific, measurable terms and states the current condition exactly. Use terminology that indicates quantity, quality, time, cost, or any term, which quantifies or qualifies the problem. Avoid implying any cause or solution as that keeps you from exploring all possible solutions. A good problem statement is: 7

  • 1. Written Down: Usually in one paragraph because more than one paragraph indicates more than one problem in a single problem statement.

  • 2. Factual: All the descriptive terms should be precise, without emotion, and without names.

  • 3. Agreed to by All Parties: Lack of consensus at this stage indicates the problem is still unclear.

A good problem statement should answer the following questions:

  • 1. What is the problem? Often two or three words (a noun and a verb) are

enough (i.e. target was missed, aircraft still broken, repair was slow, computer has crashed, Airman was late).

  • 2. Where did the problem happen? Clearly explain the location where the

problem occurred (e.g. in the reception/customer service area).

  • 3. When did the problem happen? Clearly identify when the problem occurred

(e.g. during the preliminary inspection, while conducting shift change).

  • 4. What is the significance of the problem? Many problems exist, some are

more critical than others are. When tackling any problem supervisors should ask

themselves: “Does solving this problem support the strategic goals of my organization?”

Consider the following situation: Your boss just put you in charge of the fuels flight distribution section where you supervise the fuel truck operators. Their current average response time to fuel aircraft is 40 minutes. This has been a gradual increase over the last 4 months when the average response time was 25 minutes. The acceptable standard is no more than 30 minutes. Therefore, your problem statement might be:

“Average response time to fuel aircraft on the ground has increased from

25 minutes to 40 minutes over the last 4 months. The increase in time to

refuel the aircraft has caused delays and missed sorties.”

UM10SG - 16

This problem statement is clear, specific, and uses measurable terms of quantity and time.

It’s objective and clearly identifies the current condition STEP 2: Break Down the Problem/Identify Performance Gaps (OODA)

The better we understand the problem, the better the solution, but those who haphazardly want to take action and implement solutions find this step frustrating. Only by thoroughly evaluating a problem are we able to judge the impact of selected and alternative solutions.

Key Process Indicators and Metrics (KPI/M)

The first step in assessing a problem area is gathering and reviewing Key Process Indicators and Metrics (KPI/M). Understanding what objective data is needed and what the data means once it has been gathered is critical to “root cause” problem solving and process improvement.

Metrics are the means to measure results that must complement our organization’s operations and determine whether we have achieved the desired goal(s) and/or objective(s).

There are two types of metrics: leading and lagging. 8

  • 1. Leading metrics (outcome-based) make future predictions about a likely

occurrence, thus allowing us to predict or forecast potential problems and neutralize

or avoid them.

  • 2. Lagging metrics (results-oriented) track overall performance trends that are

collected and reported after-the-fact. Because of the after-the-fact reporting, potential problems may become more of a problem than previously reported.

Value and Waste Analysis

This is another valuable assessment tool that helps break down the problem and identify performance gaps. It helps streamline and improve productivity, quality, and customer service. Knowing the following eight types of waste helps you recognize how the problem impacts your mission: 9

  • 1. Defects Work that contains errors, rework, mistakes, or lacks essentials

  • 2. Over-production Generating more than is currently needed

  • 3. Waiting Idle time created when material, information, people or equipment is unavailable

  • 4. Nonstandard over-processing Efforts that create no value from the customer’s viewpoint

  • 5. Transportation Movement of material or information that does not add value

  • 6. Intellect Any failure to fully utilize the time and talents of people

  • 7. Motion Movement of people that does not add value

  • 8. Excess inventory Excessive information, parts and material are on hand and not needed

Performance Gap Analysis

Performance Gap Analysis identifies the difference between the current level of

UM10SG - 17

performance and the desired level of performance and can assist in understanding the difference between current performance and customersrequirements (VOC).

How do you know when to stop gathering and analyzing problem data? Use the following questions as a guide: 10

  • 1. Does the problem require more analysis or is there enough information to execute a

solution?

  • 2. If more information is needed:

    • a. What measures are available today?

    • b. Do these measures align with the customer driven Key Performance Indicators

(KPI)?

  • c. Is there a gap between the data available and the data required? i.e. Does the data needed not exist yet?

  • d. What is the gap between current performance and the customers’ requirements?

  • e. Does the data point to any specific areas of root cause?

  • f. Does the data indicate a constraint?

STEP 3: Set Improvement Target (OODA)

Once you have developed a clear and objective problem statement, Step 1, Clarify & Validate the Problem, broken down the problem, and identified performance gaps, Step 2, you then need to identify a goal and develop a goal statement. A goal is your desired state, where you want to be when you solve the problem.

If you do not know where you are going, how will you know when you get there? A clear goal statement provides focus and direction and makes it possible to look into the future and target progress as you solve the problem. Consider the following two aspects when crafting improvement targets:

  • 1. Strategic Vision: Strategic Vision is a view into the future that describes how an

organization will strategically perform or conduct business. It implies a gap between the current performance and a better future performance. 11

  • 2. Tactical Targets: Tactical Targets define the performance levels required to make the

goal a reality. Targets should be challenging but achievable and have B-SMART characteristics: 12

  • 3. B-SMART

    • -- Balanced Ensure goals are balanced across the multiple fronts of organizational output and multiple targets

      • -- Specific Have desirable outputs that are based on subject matter expert knowledge and experience and are applicable to the process improvement activity

        • -- Measurable Includes time frames and have data that is obtainable from specific sources

UM10SG - 18

  • -- Attainable Resources are available, may have some risk, but success is possible

    • -- Results Focused Link to the mission, vision, and goals and are meaningful to the user

      • -- Timely Provide step-by-step views versus giant leaps and are measurable at interim milestones

Using the previous problem statement, a goal statement might be,

“Average response time to fuel aircraft will decrease to 20 minutes three months from now, and will not increase thereafter.”

Or, you could write,

“Average response time to fuel aircraft will decrease to 20 minutes by (use an actual date) and won’t increase thereafter.”

In fact, there are many ways to write a goal for this problem, the idea is to ensure it is balanced, specific, measurable, attainable, results focused, and timely. At this point, you should take your problem and goal statements, and begin determining root causes.

STEP 4: Determine Root Cause (OODA) All too often Air Force leaders find themselves addressing problems that have been

“solved” many times before. This is usually due to problem solving efforts directed at the

symptoms of a problem rather than at the root cause of the problem. If an aircraft is constantly breaking down and becomes non-mission capable, should we: reduce the

aircraft usage, improve repair cycle time, improve the quality of replacement parts, improve the aircraft design, or improve the aircraft design process. Clearly, each step becomes increasingly difficult but each step also has a greater impact in preventing the reoccurrence of the problem.

Root Cause Analysis is a tradeoff between digging as deeply as possible and finding the deepest point that is still within your sphere of influence. There are several tools that can assist your with determining the “true” root cause. Root Cause Problem Solving

Root Cause problem solving is not reacting to a decision that seemed to work before. Just because a specific reaction worked before, that does not guarantee it will work again. Other variables could have influenced the problem. To help determine the root cause of a problem consider the perceived initial problem based on standards, clarify the situation, locate the cause of the problem based on first- hand observations, and look for the “direct” cause and effect.

The Five Whys

When trying to understand what is causing a problem, ask “Why?at least five times, more if necessary, what is causing the problem. This technique helps you understand how different causes are related, but more importantly, it helps identify

the true root of the problem. Begin with a problem statement and ask, “Why did the problem occur?” Continue to ask why four more times or until an answer does

not yield any more useful information. By using this method with all of your

UM10SG - 19

potential causes, you may end up with the same root cause popping up everywhere.

Brainstorming

A technique designed to stimulate a chain reaction of ideas relating to a problem. It helps you, or a team, build a variety of ideas in a short time about a specific problem or topic. To get the most from a brainstorming session, use the following rules and techniques:

  • 1. Withhold Judgment: The first and foremost rule is to withhold judgment of any kind.

Make no evaluation, criticism, or judgment about any idea until the brainstorming session

is over.

  • 2. Encourage Freewheeling: The second rule of brainstorming is to encourage the

freewheeling of ideas. This enables all individuals to contribute. Once ideas begin flowing, the leader allows the group to continue under its own steam with little or no guidance.

  • 3. Aim for Quantity, Not Quality: Remember, you are not judging the ideas yet. Once

ideas are flowing, write down every one. The whole idea of brainstorming is to aim for quantity, not quality. Some ideas might be silly, but most will contain at least some quality information.

  • 4. Hitchhike (piggyback) Ideas: The last rule in brainstorming allows an idea to piggyback

or hitchhike on another idea. In a brainstorming session, one member of the group suggests an idea. This idea triggers a thought in the mind of another and the process continues until you have a series of ideas, all prompted by one original thought or idea.

Brainstorming Techniques

Along with certain rules, there are four techniques for conducting a brainstorming session.

  • 1. Structured Approach: This approach means soliciting one idea at a time from each

person on the team. Participants only comment when it is their turn. If they have no

comment, they say, “Pass.” The session ends when everyone says, “Pass.”

  • 2. Unstructured Approach (also called free-form brainstorming): Here, team members call

out ideas as they come to mind. No one takes turns and the session ends when the team feels it has exhausted all ideas.

  • 3. Silent Approach: Used when you want team members to write ideas on small slips of

paper. You then collect the papers and jot down ideas for all to see.

  • 4. Fish Bone Diagram (Cause and Effect Diagram): A diagram used to depict the

relationship between specific categories of process inputs and the undesirable output. This

technique helps to identify potential causes to a problem.

UM10SG - 20

Figure 2, Fish Bone Diagram Fill in the fishbone by generating as many causes as possible

Figure 2, Fish Bone Diagram 13

Fill in the fishbone by generating as many causes as possible using brainstorming. Although the diagram uses the 3Ms and a P (Materials, Methods, Machines, and People), feel free to categorize causes in ways that make sense for your situation. Two other useful categories include the 4Ss (Surroundings, Suppliers, Systems, and Skills) and the 4Ps (Policies, Procedures, People, and Plant). Do not worry too much about the categories, they are not as important as identifying potential causes.

STEP 5: Develop Countermeasures (OODA)

This is where you develop solution(s). The decision making and solution development step is over halfway through the eight-step process. If you completed the first four steps correctly this step should be the easiest. As simple as this step is, there are some very important guidelines to follow in order to ensure the greatest possible likelihood of success.

Analysis of Alternatives Not all countermeasures are workable; therefore, thoroughly analyze alternatives for workability prior to implementation. Use the following to test possible countermeasures for workability 14

Effectiveness Will the countermeasure help achieve the target/goal? How well will it work? Will it prevent reoccurrence of the problem?

Feasibility Is the countermeasure possible given cost, management approval, resources, safety, and time?

Impact Will this countermeasure create more problems than it solves? How will it affect jobs, other operations, teams, and the Air Force?

Begin testing with a very general analysis of the alternatives and eliminate any obviously unworthy alternatives. Then, subject the refined list of alternatives to a detailed analysis

UM10SG - 21

until one or more meet some or all the criteria. After selecting countermeasures, develop a plan to communicate the change to all affected personnel.

The key principal to remember is that the impact of a solution is a combination of the quality of the solution and the acceptance of the solution by the people who must implement it. The relationship is similar to a mathematical formula:

(Quality of solution) X (Acceptance) = Impact

Excellent solutions with no [zero] acceptance equals zero impact. On the other hand, an average solution that receives some support will have some impact. With the entire first half of the eight-step problem solving process focused on the left half of the equation it is now up to you to present the solution in such a way as to gain its acceptance by those who will implement it. 15

When developing countermeasures it is important to gain consensus among stakeholders. This involvement brings commitment and a sense of ownership of the solution. Communication is critical to prevent complaints of the solution process.

Finally, consider other alternatives rather than attempt to implement ineffective countermeasures or ones that are not feasible or have little impact.

STEP 6: See Countermeasures Through (OODA)

With countermeasures developed, it is time to see them through which includes:

communicating the plan to those affected, implementing the new process, and handling unexpected issues that pop up.

Six “S”

Six “S” is a systematic approach to productivity, quality, and safety improvement. It focuses on achieving visual order, organization, cleanliness, and standardization. The following Six “S” areas can help improve profitability, efficiency, and service:

  • 1. Sort clean and organize

  • 2. Straighten identify, organize, and arrange

  • 3. Shine routine cleaning and maintenance

  • 4. Standardize simplify and standardize

  • 5. Sustain continue training and maintaining standards

  • 6. Safety priority in all improvement areas 16

Visual Management

This is the use of visual indicators (displays and controls) to help you and others determine immediately whether you are in a standard condition or deviating from it. This tool is used to establish a visual work environment that’s set up with signs, labels, color-coded markings, etc., such that anyone unfamiliar with the process can acknowledge and understand the process, and knows what’s being done correctly and incorrectly. 17

UM10SG - 22

Standard Work

Standard work is the foundation of continuous improvement. Therefore, the best known approach to completing a task is by ensuring consistency throughout the working process so that the same work will take the same amount of resources to achieve the same results every time. If work processes are not standardized, it is impossible to effectively experiment and test new ideas for improvement. If the current process is not standard, then it will be impossible to assess the impact of improvements upon process outputs. To ensure consistency and standardization in work processes: involve Airmen from all shifts; let the process workers define the work and gain consensus; keep it simple; and document the standard work and train from the documentation. 18

Material/Information Flow

Based on customer requirements and demands, material/information flow determines what material and information is required to implement the countermeasure. Continue to improve and implement product, material, and information flow throughout the See Countermeasures Through process.

In addition to the above areas, most process improvements require some form of training to make the solution work. Normally, the training will link to the communications plan developed in Step Five, Develop Countermeasures, and usually involves several levels of the organization simultaneously. Implementers and front line supervisors need the most intensive training with less detailed, but no less important training, provided to leadership. Customers and suppliers of the targeted process may also need training. 19

STEP 7: Confirm Results & Process (OODA)

Step seven closely mirrors the data collection portion of Step Two, Break Down the Problem/Identify Performance Gaps. Implementation requires a variety of reviews and acknowledgements to confirm results and processes.

Conducting a Review

Incorrect root causes determination is the most common mistake in problem solving. By solely focusing on problems, you must set a positive tone or the reviews will become punishment. Therefore, it is critical to establish a balance between learning and creating an environment where it is unacceptable to hide problems. Reviews should not be concerned with a particular end or purpose but rather planned to accomplish a specific objective, such as:

  • 1. Understand the current situation in comparison to the committed plan

  • 2. Develop higher levels of behavior and performance

  • 3. Create a sense of team from common purposes

  • 4. Instill a sense of pride in accomplishment

  • 5. Establish accountability

  • 6. Work problems familiar to the entire group

  • 7. Provide rewards and recognition 20

UM10SG - 23

Rewards and Recognition

Recognition is a powerful tool that can make workplaces, on average, 15-25 percent more productive, making Airmen more engaged and committed to their work and organization. There are six principles of recognition efforts:

  • 1. Timely given as soon as possible (immediately if possible) after the desired performance

  • 2. Proportional – Don’t overdo for average or mediocre performance

  • 3. Sincere honest and open appreciation of their efforts

  • 4. Specific Recognize notable efforts in detail

  • 5. Individual Recognize personal contributions

  • 6. Personal – Recognition should fit the individual’s desires 21

While confirming results, the review should combine personal/professional development and acknowledgement of desired behavior and performance. This type of review helps standardization and continuous process improvement throughout the organization.

STEP 8: Standardize Successful Processes (OODA)

Step eight is the most commonly skipped and most under completed step of the entire problem solving process. It is very tempting to take newfound knowledge and skills and immediately move on to the next improvement initiative without ensuring the results stick. Upon completion of step 8, consider standardizing improvements, communicating improvements and lessons learned, and identifying opportunities or problems identified in the problem solving process: 22 What is needed to standardize the improvements:

 
  • 1. Changes to:

a.

Technical orders

b.

Air Force Instructions

c.

Other official policies or procedures

d.

Equipment

e.

Material

f.

Vendors or Suppliers

  • 2. Communicate improvements and lessons learned:

a.

Key Meetings

b.

Air Force Publications, message traffic, chain of command

c.

Communities of Practice (Air Force Knowledge Now)

  • 3. What other opportunities (problems) were identified by the problem solving

process?

That covers the OODA Loop and the 8-step problem-solving process. Now it is time to examine a critical part of the problem-solving process, decision making.

UM10SG - 24

MP 4. SCENARIO

“The EPR”

MSgt Yost was a brand new superintendent and new to the unit, but had already attended three Wing Commander staff meetings. As the EPR slide popped up on the screen, she

clinched her fist and watched as Chief Barnes squirmed in his seat. “Chief” the Wing Commander said, “it looks as if your squadron has done it again.” The Chief’s face turned

red and MSgt Yost felt bad for him as the Wing Commander pointed his finger directly at

the Chief and stated, “I would recommend that you work harder on getting your EPRs in on time before next month’s staff meeting!”

Chief Barnes just walked out of the Wing Commander’s monthly staff meeting shaking his head. This is the third month in a row the Wing Commander has chewed him out because his organization keeps showing up on the “Late EPR” slide. In fact, his squadron’s EPR

on-time rate is the worst in the entire wing for the last 4 months.

“You know Master

Sergeant Yost,” the Chief replies, “we’re getting our tails kicked at stand up every month because of late EPRs.” As MSgt Yost walks down the sidewalk towards her office, she asks herself, What can we do to fix this problem?”

As MSgt Yost sits at her desk, she wonders how the EPR issue got so bad. The next day, MSgt Yost grabs her notebook and begins to walk around the unit introducing herself and asking questions about the current EPR process and the existing problems and concerns with it. After just a couple hours, MSgt Yost has collected over four pages of feedback regarding the unit’s EPR process and potential issues. That afternoon, she calls a staff meeting with all her section supervisors and after explaining the EPR situation says, “If we don’t take care of our people, the ones who take care of the mission, our organization as a whole will suffer the consequences.”

UM10SG - 25

Attachment 1

Small Group Activity

The “House Hunter”

Scenario

Technical Sergeant Jones just PCSd to Sheppard Air Force Base, Texas. He and his spouse have decided to buy a house rather than rent. They have three children. A 16-year old daughter and two boys 15 and 13 years old.

After discussing the purchase at length, they plan to use their VA home loan eligibility, since they have saved only $15,000 for a down payment and closing costs. They also looked over their combined income and debt and decided the mortgage payment (with tax and insurance) cannot exceed $1100.00 per month. In addition, they plan on retiring in the area and prefer not to have to work again, so they will limit their mortgage payments to just fifteen years.

After some additional discussions, they realize they prefer being close to the schools and the base and that a house with four bedrooms and two full bathrooms would be ideal, but if that turns out to be too expensive they need a house with at least three bedrooms and one full bathroom. They would also like a garage, fenced yard, wood-burning fireplace, and workshop.

Alternatives

House #1 is a two-story, “Cape Cod” style house 30 minutes from Sheppard AFB and only one block from the largest high school in Wichita Falls. The home is 12 years old and has a formal dining room, three bedrooms, two full bathrooms, and a large family room with a stone fireplace. It also has a fenced-in back yard and a detached two-car garage. The asking price is $135,000. The owner wants $35,000 down. This would require using their $15,000 in savings plus a second loan for the other $20,000. Their mortgage payment on a 30-year mortgage would be $650 and the second loan payment would be $550.

House #2 is a modern design with four bedrooms and one and a half bathrooms. It also has a fireplace, den, formal dining room, a detached two-car garage, fenced backyard, and a small utility building. The house is close to a major shopping mall. The school is just two blocks away, and it is about 10 minutes driving time to the base. The asking price is $125,000. They could use their $15,000 down and get a VA guaranteed loan for the rest. With a 15-year mortgage, their payments would be around $1100 per month.

House #3 is a newer three-bedroom home with one full bathroom. It has a dining room and a spacious den. It is about 35 minutes from the base and the children would need to take a bus to the school. The house also comes with an attached one-car garage/workshop and a large open yard. The asking price is $107,000. With a $15,000 down payment, they have the choice of $1050 per month for 15 years or $950 per month for 30 years.

UM10SG - 26

     

Attachment 2

STEP 1: DECISION STATEMENT:

STEP 2: DETERMINE OBJECTIVES (criteria for decision)

         

List the Musts

     

Weigh the

List the Wants

   

Wants

Musts are mandatory

“Wants are optional”

   

(1-10)

STEP 3: Alternatives (Compared to the Musts)

Enter Yes or No for each item

List Musts

 

Alt #1

Alt #2

 

Alt #3

UM10SG - 27

     

Previously

 

Alt

       

Score

     

Alt

     

Score

 
 

Wants

 

Determined

       

(PDW x Alt

       

(PDW x Alt

 
     

Weight (PDW)

 

_____

Weight

   

Weight)

   

______

Weight

   

Weight)

 
 

(

__

x __

)

= __

(

__

x __

)

=

 
             

__

       

(

__

x __

)

= __

 

(

__

x __

)

=

 
             

__

       

(

__

x __

)

= __

 

(

__

x __

)

=

 
             

__

       

(

__

x __

)

= __

 

(

__

x __

)

=

 
             

__

       

(

__

x __

)

= __

 

(

__

x __

)

=

 
             

__

       

(

__

x __

)

= __

 

(

__

x __

)

=

 
             

__

       

(

__

x __

)

= __

 

(

__

x __

)

=

 
             

__

       

(

__

x __

)

= __

 

(

__

x __

)

=

 
             

__

       
         

Totals

     

_______

     

________

 

Risk Analysis: (What problems might be encountered, what are the unintended consequences?)

         

1) Is there anything I overlooked? 2) Are there any potential issues because of a Must? 3) Is there a possibility that my choice is invalid (if so how severe are the implication)? 4) Is there anything that might prevent me from being comfortable with my choice?

 

Alternative

 

Consequence

             

Probability

   

Seriousness

Alt # __________

   
   

Alt # __________

   
   

Final Decision

We selected alternative # _____________

                             
                                             

UM10SG - 28

NOTES

  • 1 Department of the Air Force. Air Force Smart Operations for the 21st (AFSO21) Century Playbook, 27 May 2008

  • 2 James Reason (1990). Human Error. Ashgate

  • 3 Department of the Air Force, Air Force Smart Operations for the 21st Century Playbook, B-1.

  • 4 Ibid, B-2

  • 5 Air Force Smart Operations for the 21st Century (AFSO21) Playbook, Tools and Methodology, Version 2.0, Volume J, October 2007, J-4.

  • 6 Charles F. Kettering, “Quotes,” Thinkexist.com, http://en.thinkexist.com/quotation/

a_problem_well_stated_is_a_problem_half/159095.html.

  • 7 AFSO21, Tools and Methodology, J-8.

  • 8 AFSO21, Tools and Methodology, J-11.

  • 9 AFSO21, Tools and Methodology, J-12-13.

    • 10 AFSO21, Tools and Methodology, J-17 and B-4.

    • 11 Air Force Smart Operations for the 21st Century (AFSO21) Playbook, Introduction to the Eight Step OODA LOOP AFSO21 Problem Solving Model, Version 2.0, Volume B, October 2007, B-4.

    • 12 AFSO21, Tools and Methodology, J-12.

    • 13 Richard Y. Chang and Keith P. Kelly, Step-By-Step Problem Solving (CA: Richard Chang Associates, Inc. Publishing Division, 1994), 97.

    • 14 AFSO21, Tools and Methodology, J-32.

    • 15 AFSO21, Introduction to the Eight Step OODA LOOP AFSO21 Problem Solving Model, B-6.

    • 16 AFSO21, Tools and Methodology, J-45.

    • 17 AFSO21, Tools and Methodology, J-43.

    • 18 AFSO21, Tools and Methodology, J-67.

    • 19 AFSO21, Tools and Methodology, J-47.

    • 20 AFSO21, Tools and Methodology, J-71.

    • 21 AFSO21, Tools and Methodology, J-72-73.

    • 22 AFSO21, Introduction to the Eight Step OODA LOOP AFSO21 Problem Solving Model, B-8.

UM10SG - 29