MANAGING RISK

In human space flight programs

 “Contact light.”

-Buzz Aldrin, July 20, 1969

 Nov 1968 LM-5 integrated systems test
CSM-107 integrated systems test
Apollo 11
Dec 1968

Dec 1968 LM-5 acceptance test

Dec 1968 LM-5 ascent stage delivered to Kennedy
Dec 1968 LM-5 descent stage delivered to Kennedy
Dec 1968 S-IVB ondock at Kennedy
Dec 1968 CSM ondock at Kennedy
Dec 1968 Command and Service Module mated
Dec 1968 S-II ondock at Kennedy
Dec 1968 S-IC ondock at Kennedy
Dec 1968 Combined CSM-107 systems tests
Dec 1968 S-IU ondock at Kennedy
Dec 1968 CSM-107 altitude testing
Dec 1968 Rollover of CSM from the Operations and
Checkout Bldg. to Vehicle Assembly Bldg.
Dec 1968 Integrated systems test
Dec 1968 CSM electrical mate to Saturn V
Dec 1968 Rollout to Launch Pad 39A
Dec 1968 Flight Readiness Test (FRT)
Dec 1968 Countdown Demonstration Test (CDT)
Dec 1968 Launch
3
 Manned Flight
Over 50 years of NASA leadership in
ground and flight systems
development and human space
exploration.
 Apollo 1
Gus Grissom, Ed White, Roger Chaffee

Jan 27, 1967

The Board noted that the underlying design approach in Apollo was to control the known risk of fire…by isolating and rendering

“ safe all possible ignition sources. The experience in flight and in tests prior to the accident had suggested that the probability of a
spacecraft fire was low…Potential ignition sources inside the spacecraft had been treated so as to be considered safe; neither the
crews nor the test and development personnel felt the risk of spacecraft fire to be high. The Apollo 204 accident now proves this
assumption to have been wrong.
INCREDIBLE ACHIEVEMENT!
THE JOURNEY
Timeline of commercial spaceflight

2001
Dennis Tito flies to space

1995
Office of Commercial Space Transportation is
transferred to the Federal Aviation Administration

1984
Commercial Space Launch Act
 2015
SPACE Act of 2015

2008
Space Exploration Technologies
(SpaceX) reaches orbit

2004
Commercial Space Launch Amendments Act of 2004
 9

What is assurance?
A dinner table perspective
Disciplined application of systems engineering, risk management, quality
assurance and program management principles toward achieving mission success.

Focuses on the detailed engineering of the space system using independent

technical assessments as a cornerstone throughout the lifecycle of requirements
definition, design, development, production, test, deployment and operations
phases.
A hot dog at the game beats
a roast beef at the Ritz.

-Humphrey Bogart
 DESIGN MANUFACTURE AND GROUND TEST, STATIC
CHECKOUT FIRING. AND CHECKOUT

R&QA Plans Models Reliability

Reliability
Profile Reliability
Assessment
Prediction
Apportionment
R&QA FMECA
Requirements

Failure and
FMECA Quality
Goals Quality Operating
Control
Control Time

Mission End Item Ground Test

Design Manufacture Test Data
Requirements Specifications Program

FACI
Ground
Design
Test Plans
Reviews End Item Tests
PDR/CDR
COFW
Parts
Program

Mission Directives
Apollo Reliability and Quality Assurance Program Plan (1966). https://archive.org/details/nasa_techdoc_19700078138 11
 PRE-LAUNCH (STAGE AND MODULE INTEGRATED TEST) MISSION AND POST-FLIGHT

Assessment

Data
Feedback

Quality and Flight Test

Historical Records Data
Evaluation

Reviews
Flight Tests
DCR/FRR

Reliability and
Qualification Test
Status Report

12

Apollo Reliability and Quality Assurance Program Plan (1966). https://archive.org/details/nasa_techdoc_19700078138

 Apollo 1
Gus Grissom, Ed White, Roger Chaffee

Jan 27, 1967

The Board noted that “the underlying design approach in Apollo was to control the known risk of fire…by isolating and rendering safe all
possible ignition sources. The experience in flight and in tests prior to the accident had suggested that the probability of a spacecraft fire was
low…Potential ignition sources inside the spacecraft had been treated so as to be considered safe; neither the crews nor the test and
development personnel felt the risk of spacecraft fire to be high. The Apollo 204 accident now proves this assumption to have been wrong.”
 SPACE SHUTTLE PROGRAM
Notional Strength / Effectiveness of NASA S&MA for the Space Shuttle Program
Comprehensive safety requirements
Safety products must be available early if
they are to influence design and need to be developed and adopted at
program inception. A comprehensive
operations. The hazard reports FMEA/CILs
safety
for the shuttle program were always lagging requirements are key to the success
of a program. With the exception of a good
behind the design process and therefore
setand
were ineffective in influencing the design of reliability and quality requirements,
operation during the DDT&E phase of the a set was lacking at the start of the
such
shuttle program. Use of hazard analysis
program. Prior to the loss of 51L [Challenger]
methodologies was also lacking. This
the flight rules and crew procedures were
resulted in the failure to identify and
developed without them and did not even
adequately control hazards early in the
reference them.
program.
https://spaceflight.nasa.gov/outreach/Significant_Incidents.pdf
SSIAT
Risk management process erosion
set of Culture created by the desire to reduce costs.
Cooperation and coordination
within and between organizations
is needed to increase
organizational effectiveness –
avoid “sandboxing.”
Benefits:
(1) Better coordination and
“…tendency to acceptsharing
risk solely
of technical data
because of prior success.”
resulted -inSSIAT
improved
timeliness and quality of
Columbia Crew Survival
S&MA risk assessments.
Investigation Report
(2) Earlier coordination and
Crew survival should be an integral
resolution of technical issues
part of all human spaceflight programs.
resulted in better S&MA
Safety should extend beyond just
support to the SSP and OSMA.
preventing the accident and include
provisions for minimizing the
consequences of the accident.
14
Challenger The inherent risk of the Space Shuttle program is defined by the combination of a highly dynamic
Mission 51-L

“
environment, enormous energies, mechanical complexities, time consuming preparations and
extremely time-critical decision making. Complacency and failures in supervision and reporting

”
seriously aggravate these risks.

January 28, 1986

Christa McAuliffe
Gregory Jarvis
Judith Resnik
Dick Scobee
Ronald McNair
Michael Smith
Ellison Onizuka
 Mission Assurance Framework
Goal is mission success

Category Process
1) Design Assurance
Program Execution 2) Requirement Analysis and Validation
3) Parts, Materials and Processes
4) Environmental Compatibility
1 5) Reliability Engineering
6) System Safety
7) Configuration/Change Management
8) Integration, Test and Evaluation

9) Risk Assessment and Management

Risk, Oversight, and Assurance
10) Independent Reviews
2 11) Hardware Quality Assurance
12) Software Assurance
13) Supplier Quality Assurance

Triage, Information, and Lessons Learned 14) Failure Review Board

15) Corrective/Preventative Action Board
3 16) Alerts, Information Bulletins

16
Johnson-Roth, G. (2011). Mission assurance guidelines for A-D mission risk classes. Aerospace Report No. TOR-2011(8591)-21. El Segundo, CA: The Aerospace Corporation.
Columbia
Mission STS-106

Ilan Ramon, IAF Willie McCool David Brown Kalpana Chawla Michael Anderson Laurel Clark Rick Husband

Cultural traits and organizational practices detrimental to safety were allowed to develop, including: Reliance on past success as a

“
substitute for sound engineering practices; Organizational barriers that prevented effective communication of critical safety
information and stifled professional differences of opinion; Lack of integrated management across program elements; The
evolution of an informal chain of command and decision-making processes that operated outside the organization’s rules
THE FUTURE
Entrepreneurship and the Private Industry

SpaceX Blue Origin Bigelow Aerospace Virgin Galactic

Elon Musk Jeff Bezos Robert Bigelow Sir Richard Branson

Sierra Nevada Corp. Boeing United Launch Alliance Rocket Lab

Fatih and Eren Ozmen John Mulholland, VP Salvatore Bruno, Pres. & CEO Peter Beck
Launch Costs
Price per pound to orbit

Space Shuttle Space Shuttle SpaceX Falcon 1 ULA Atlas V SpaceX Falcon Heavy

$188,000/lb. $59,000/lb. $22,000/lb. $12,500/lb. $2,100/lb.

1981 1995 2006 2016 2020
THE NEW SPACE RACE
Reusability

SpaceX Sierra Nevada Blue Origin Virgin Galactic

Target Consumer Base
Risk Takers
SAFE SPACE FLIGHT
Done cheaply

“ Our goal is to show that you can develop a robust, safe

manned space program and do it at an extremely low cost.

”
Burt Rutan
Founder, Scaled Composites
Scaled Composites
July 26, 2007 test stand accident at
the Mojave Air and Space Port, Calif.,
killed three Scaled Composites
employees and injured three others.
 Virgin Galactic
In 2014 Virgin Galactic’s SpaceShipTwo
experiences catastrophic structural
failure due to co-pilot (human) error,
killing the co-pilot and severely
injuring the pilot.
 Hard Lessons in Assurance
Space, oil and nuclear

2003 2010 2018 - 2019

Space Shuttle Columbia Mission STS- Deepwater Horizon blowout, similar to Ethiopian Airlines Flight 302 in March
107 experienced structural breakup the Montara mishap. 2019, killing all 157 souls aboard.
over Texas during reentry, leading to a
29-month halt in Space Shuttle flights. Lion Air Flight 610 crashes in October
2018, killing all 189 souls aboard.

1986 2009 2011

Space Shuttle Challenger Mission 51L Montara wellhead suffered a failure at Fukushima Daiichi nuclear disaster was
experienced structural breakup 73 the point where cement secured the the most significant nuclear incident
seconds after liftoff, leading to a 32 well into the ocean floor, leading to a since the 1986 Chernobyl disaster and
month halt in Space Shuttle flights. blowout. the only other disaster to be given the
Level 7 event classification of the
International Nuclear Event Scale.

Derived from Jackson, C. (2019). Small satellites, emerging technology and big opportunities (part three of seven) – No, we really mean ‘Mission Assurance’. 25
https://accendoreliability.com/small-satellites-emerging-technology-big-opportunities-part-three-seven-really-mean-mission-assurance/
 Danger In and From the Seas
Petroleum and Nuclear Energy

25%

National Academies of Sciences, Engineering, and Medicine. 2016. Beyond Compliance: Strengthening the Safety Culture of the Offshore Oil and Gas Industry. Washington, DC: The National Academies Press. 26
https://doi.org/10.17226/23662.
 Boeing 737 MAX
Competing With Airbus A320neo

Cut Costs
Repeated upgrade of an
existing system.

Corner Market
Keep within “type
certification”.

Safety Systems
Self-oversight rather than
increased FAA oversight.

27
Culture
Challenger Columbia

1986 Mission 51L 2003 Mission STS-107

28
 Agile Companies
Millennium Space Systems

“Millennium Space Systems Expands Full-

Service Space Factory to Step up Production
to Hundreds of Satellites Each Year”

“Millennium Space Systems Granted exclusive

license from NASA Goddard”

“Millennium Space Systems ALTAIR™ Pathfinder

Satellite Surpasses 10,000 Hours in Orbit”

http://millennium-space.com/about.html

29
 Agile Product Development
Boston Consulting Group (BCG)

Speed to market
For milestones such as prototype
development and first flight

Nonrecurring costs
Due to shorter development cycles

Recurring costs
More innovative thinking by agile
teams

30
Rein, J. & Hasik, M. (2019) https://www.bcg.com/industries/engineered-products-infrastructure/ten-rules-agile-aerospace-defense.aspx
 BCG’s Ten Rules

01. Aim for

breakthrough
02. Focus on
principles and be
03. Put the right
leaders in place
04. Engage
customers in
05. Create the
right physical
results, but flexible with design work space
explain the processes
“why”

06. Deploy iterative 07. Create cross- 08. Assign talent for 09. Fail fast, and 10. Emphasize
cycles to move functional the duration of learn quality and
toward an MVP teams that the project continuously safety
– and beyond have clear
accountability

31
Rein, J. & Hasik, M. (2019) https://www.bcg.com/industries/engineered-products-infrastructure/ten-rules-agile-aerospace-defense.aspx
 Agile Mission Assurance
The Aerospace Corporation

Leveraging Advanced Tools.

Developing a Standard.

Upgrading Mission Control .

Considering Alternatives.

Switching Launch Vehicles – “Ship and Shoot”.

32
https://aerospace.org/Annual-Report-2018/ensuring-faster-access-space
Launch or Landing Failure
Industry-Wide Effects

Industry Value
Corporate reputation suffers.
Companies fold, industry suffers.

Backlog
Customer payments.
Insurance litigations.

Corrective Actions
FAA/NTSB corrective actions.
Redesign/repair/rebuilding of ground/flight hardware.

Public Perception
Loss of confidence.
Greater public and congressional scrutiny.
FAA/AST RESPONSIBILITY
Public safety and the “learning period”

Public Safety and Security

Protection of the public, property, and national security and foreign policy interests of the U.S.
Prevention of hazards to equipment and personnel at launch sites.

Licensing
FAA license is required for any launch or reentry.
FAA license is required for the operation of any launch or reentry site.

Direct Oversight
The FAA and its Office of Commercial Space Transportation conduct accident investigations.
Contrary to popular belief It has roots in a piece of literature from 45 BC.

Informed Consent
Prior to flight, crew and spaceflight participants must provide their written consent to participate.
Crew and spaceflight participants must be informed, in writing, of mission hazards and risks, vehicle
safety record, and the overall safety record of all launch and reentry vehicles.
FAA/AST Role
Industry Growth While Building Coalitions

Facilitating Growth

Public Safety
Encouraging, facilitating and
promoting the commercial space
industry. Market Demands
Commit to ensuring public safety.

Enabling the commercial space
transportation industry to meet its
market demands with cost
effectiveness.
Standards
Develop a strong coalition within
the industry and with industry trade
groups through partnership and
advising.
RISK
Government to Private

Private Industry Federal Government

Government as one of many customers – Government as both the only launch provider
market bears the risk. and the primary customer – primary risk taker.
Safety Management Systems
The Four Pillars

Establishes senior management's commitment to continually improve safety; defines the

methods, processes, and organizational structure needed to meet safety goals. Establishes
management commitment to safety performance, clear safety objectives, and transparency.
Safety policy

Determines the need for, and adequacy of, new or revised risk controls based on the
assessment of acceptable risk. A formal process that describes the system, identifies the
hazards, assesses the risk, analyzes the risk, and controls the risk.
Risk Management

Evaluates the effectiveness of implemented risk control strategies, and helps to identify new
hazards. Systematically provides confidence that the organization meets or exceeds safety
Safety Assurance requirements, ensures compliance, and provides insight/analysis for improvement.

Training, communication, and other actions to create a positive safety culture within all levels
of the organization. Advocates, matches competency requirements to system requirements,
Safety Promotion and disseminates safety lessons learned. Everyone has a role in promoting safety.
Effective Risk Management
Development of mitigation plans to
manage, eliminate, or reduce risk to
an acceptable level.
Impact of each risk on the program’s cost,
schedule, and technical performance, their
dependencies, interdependencies, and
timeframe.
Plan is continually monitored for efficacy,
revise course-of-action if needed.
Risk Monitoring and Control
Mitigation, Planning, Implementation
Risk events, their impact assessments, and their
probabilities of occurrences rank-ordered most-to-
least-critical of identified risks. Forms a basis for
Risk Prioritization
resource allocation.
Risk Assessment
Risk events having negative impacts on the program’s
ability to achieve performance or capability outcome goals,
either within the project or from external sources
Risk ID
Assurance Programs
Benefits to Industry

Risk Management
Improved timeliness and quality of RAs.
Balancing of schedules, milestones, operational tempo.
Improved RA capability and risk-informed decisions.
Committee of Sponsoring Organizations of the Treadway Commission
Enterprise Risk Management Framework

COSO Defined
“… a process, effected by an entity's board of directors, management and other personnel,
applied in strategy setting and across the enterprise, designed to identify potential events
that may affect the entity, and manage risks to be within its risk appetite, to provide
reasonable assurance regarding the achievement of entity objectives.”
”
ERM Framework

Objective Setting
Risk and Objective Setting.
Risk Appetite.
Risk Tolerance Aligned with Risk Appetite.

Internal Environment
Risk Management Philosophy.
Risk Culture.
How Actions Affect Risk Culture
ERM Framework

Risk Assessment
Understand Potential Events.
Assess Risks – Likelihood & Impact
Qualitative & Quantitative.
Time & Objective Horizons.
Event Identification Inherent and Residual Risks.
Risks and Opportunities.
Negative impact Events.
Positive Impact Events.
Internal and External Factors.
Influence Risk Profile.

Risk Response
Responses.
Evaluate Options.
Select Responses.
ERM Framework
Control Activities
Policies and Procedures.
All Levels of the Organization.
Technology Controls.

Information and Communication

Communicate to Organization.
Vertical and Horizontal Information Flow.

Monitoring
Ongoing.
Evaluations.
Combination of the Two.
ERM Framework

Internal Audit
Monitoring, but NOT Implementing/Maintaining
Assists Management and BOD

Internal Control
ERM Roles & Responsibilities
Moving Forward
Learn From The Past, Improve Into The Future

Value
Effective, reliable, and safe capability
for space launch systems and
Value commercial space programs.

Risk Management
Risk-based safety allows commercial
operators to focus on interconnected
Risk Management
and competing sides of risks.

Assurance Strategies
Leverage proven industry standards
Assurance Strategies and knowledge, and the expertise
that has been gained through decades
of trial and failure.
https://spaceflight.nasa.gov/outreach/Significant_Incidents.pdf

46
47
https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20120002677.pdf

48
 RESEARCH
Make Risk Management and Assurance Commonplace

Historical Applications
New opportunities for leveraging what has worked
with lean principles and graded approaches.
SAFE SPACE FLIGHT
Done cheaply

“ What is fundamental to reliability is an almost religious

vigilance and attention to detail on every member of a
development team.

”
Wernher von Braun
American-German aerospace engineer
THANK YOU...
Questions?
 53

My contact info: