How to Assess a Joint

Confidence Level (JCL) Model

Brian Berry
2017 NASA Cost and Schedule Symposium
August 29 – 31
Washington, D.C.
Outline

 Background
 JCL Process
 NASA JCL Modeling Lessons Learned
 JCL Model Assessment
 Identify and track model and input changes over time
 Validate calculations
 Evaluate Active Risks
 Evaluate deterministic critical path and probabilistic critical
path and tasks
 Quickly access/understand model results
 Summary

2017 NASA Cost and Schedule Symposium 2

Background

 Numerous JCL analyses have been performed since NASA

policy established JCL requirement in 2009
 Many lessons learned with improvements to processes, tools,
and analysis
 70% JCL value is a high bar metric—difficult to meet
 JCL model output value can become focus of analysis
 Analysts need eyes wide open during entire process
 Need to assess model, inputs, and outputs to help ensure the
JCL value is robust

2017 NASA Cost and Schedule Symposium 3

JCL Process*

 0. Identify goals of the JCL analysis

 1. Develop a summary analysis schedule
 2. Load cost onto the schedule activities
 3. Incorporate the risk list
 4. Conduct an uncertainty analysis
 5. Calculate and view the results

* NASA Cost Estimating Handbook V4.0, pJ-11

2017 NASA Cost and Schedule Symposium 4

 NASA JCL Modeling Lessons Learned

 Durations that may be lengthy (> 44 days per best practices) can be
acceptable for JCL analysis schedules
 Analysis schedules must be structurally sound with solid logic and be
accurately statused. For use in SRAs and JCLs the network must be able to
move freely without limitations of constraints
 All main phases of the schedule are linked together so as durations fluctuate during
simulation, the schedule on a whole expands and contracts appropriately and no
unnecessary constraints
 Linkage (relationship) types are important—Strive for >90% FS and avoid tempting
shortcuts such as creating many Start-To-Start linkages with lags to mimic Finish-To-
Start
 JACS Reporting capabilities can assist in model verification
 Preds tool-see what tasks each task feeds—should support milestone
 Cost contributors chart-delta between estimate & mean should reflect intended
cost uncertainty
 Criticality Index-ensure relative differences between tasks makes sense
 Risk Event Ranking-Use Prob./Impact outputs to understand model impacts

2017 NASA Cost and Schedule Symposium 5

NASA Emphasized JACS Development and Reports Features to
Support Key Elements of the JCL Analysis Lifecycle

Health Checks
Assess Model Quality 
 JCL Inputs

Identify Model Changes  Model Comparison

Validate Calculations  Baseline Comparison

 Critical Path Report

Insight into Model Results  Insight Application

Compare Scenario Results  Insight Report Compare

2017 NASA Cost and Schedule Symposium 6

Assessing Model Quality

 Key to a JCL Model is the underlying Schedule Structure and Cost, Risk,
and Uncertainty Inputs
 Several features were implemented in JACS to provide rapid access to
these areas
 Health Checks
– Checks quality of underlying schedule and reviews syntax, logic, and specification of
JCL model inputs to ensure they are proper
 JCL Inputs
– Provides insight into application of JCL uncertainty distributions and risks to the
underlying schedule and cost

2017 NASA Cost and Schedule Symposium 7

 Health Checks

 Health Check
Report Includes
 GAO Best
Practices
 DCMA 14-Pt
Assessment
 JACS Health
 Issue Counts
 Issue List
 Provides many
ways to help
ensure that a
sound schedule,
uncertainty, and
cost loading will
underpin the JCL
2017 NASA Cost and Schedule Symposium 8
 JCL Inputs
Schedule Statistics

 Comprehensive metrics/ Description

Total Lines in IMS Inputs
Count
54
Total
% of Total
100%
WITH Duration Uncertainty
Count
26
% of Total
48%
WITHOUT Duration Uncertainty
Count
28
% of Total
52%
Total Activities 15 28% 15 100% 0 0%
Summary
information on Total Milestones
Schedule Milestones
Risk Events
Risk Factors
25
8
7
4
46%
32%
28%
16%
11
0
7
4
44%
0%
100%
100%
14

0
0
8
56%
100%
0%
0%

 Baseline Schedule
Start/Finish Hammock Milestones 6 24% 0 0% 6 100%
Total # of Summary tasks 14 26% 0 0% 14 100%
Schedule Summary Tasks 8 57% 0 0% 8 100%
Hammock tasks 3 21% 0 0% 3 100%

 Risk events, risk factors Multi-Impact Risk Events

Risk Factor Summary Section Heading
COMPLETED Activities
2
1
0
14%
7%
0%
0
0
0
0%
0%
0% 0
2
1
100%
100%
0%
IN PROGRESS Detail Activities 0 0% 0 0% 0 0%

 Cost Loading Activities with <5 workdays remaining 0 0% 0 0% 0 0%

Activities with 6 to 20 workdays remaining 0 0% 0 0% 0 0%
Activities with 20 to 100 workdays remaining 0 0% 0 0% 0 0%
0 0% 0 0%


Task with >100 workdays remaining 0 0%

Uncertainty FUTURE Detail Activities

Activities with <5 workdays duration
Activities with 6 to 20 workdays duration
15
0
0
100%
0%
0%
15
0
0
100%
0%
0%
0
0
0
0%
0%
0%


Activities with 20 to 100 workdays duration 1 7% 1 100% 0 0%

Correlation Task with >100 workdays duration 14

Total
93% 14 100%

WITH Cost Uncertainty

0 0%

WITHOUT Cost Uncertainty

COST STATISTICS Count % of Total Count % of Total Count % of Total
Total Lines in IMS 54 100% 27 50% 27 50%

 Goes beyond basic health Total Activities 15 28% 14 93% 1 7%

Total Milestones 25 46% 10 40% 15 60%
Total # of Summary tasks 14 26% 3 21% 11 79%
Total # of Hammock tasks 3 6% 3 100% 0 0%

characteristics to assess
Total number of Complete Cost loaded Tasks 0 0% 0 0% 0 0%
Total number of In Progress and Future Cost loaded Tasks 14 93% 14 100% 0 0%
Tasks with only Time-Independent (TI) Cost Specified 0 0% 0 0% 0 0%
Tasks with only Time-Dependent (TD) Cost Specified 1 7% 1 100% 0 0%
Tasks with both TI and TD Cost Specified 13 87% 13 100% 0 0%

reasonableness of inputs Point Estimate Cost

Sum of all TI cost
Sum of all TD cost
$267,476.000
$116,067.500
$151,408.500
100%
43%
57%

Uncertainty Overview
Total BOTH Duration and Cost With DURATION Uncertainty Only With COST Uncertainty Only
RISK EVENT/FACTOR STATISTICS Count % of Total Count % of Total Count % of Total Count % of Total
Task Unique Duration
Total Number of Risk Events/Factors
Remaining13 Constraint
100%
Actual Actual
10 77% 1 8%
WBS Total0 Cost TI Cost 0%TD Cost
ID Task ID Name Task Details Uncertainty Correlation Group Task Type Status Start
Number of Finish
RISK EVENTS(Days) % Complete Duration 9 Constraint69% Date 7 Start Finish
78% Predecessors
0 Successors0% Number ($)
0 ($) 0% ($)
4 331 Delayed Award Activity With Duration Uncertainty
N/A Activity Future Task 10/1/2013 12/2/2013
Likelihood of occurrence45
<25% 0.00% 45 4 ASAP 44%10/1/2013 2 NA NA
50% 3 0 5 0% 1.1.2 $0
0 $0 0% $0
12 16 EMD HW System Design Activity With Duration Uncertainty
EMDDURATION, EMDCOSTS
Activity Future Task 12/3/2013 8/4/2014
Likelihood of occurrence175
25% to 75% 0.00% 175 1 ASAP 11%10/1/2013 1 NA NA
100% 5 0 7,27,52 0% 1.4.1.1 $2,400
0 $720 0%$1,680
13 15 EMD HW Initial Design Activity With Duration Uncertainty
EMDDURATION, EMDCOSTS
Activity Future Task 8/5/2014 4/13/2015
Likelihood of occurrence180
> 75% 0.00% 180 4 ASAP 44%10/1/2013 4 NA NA
100% 7 0 8,30,52 0% 1.4.1.2 $4,800
0 $1,440 0%$3,360
14 14 EMD HW Detailed Design Activity With Duration Uncertainty
EMDDURATION, EMDCOSTS
Activity Future Task Number of5/16/2016
4/14/2015 RISK FACTORS 285 0.00% 285 4 ASAP 31%10/1/2013 3 NA 75%
NA 8 1 9,51,52 25% 1.4.1.3 0
$9,000 $2,700 0%$6,300
15 13 EMD HW Final Design Activity With Duration Uncertainty
EMDDURATION, EMDCOSTS
Activity Future Task Likelihood of
5/17/2016 8/7/2017 occurrence <25%
320 0.00% 320 4 ASAP 100% 10/1/2013 3 NA 75%
NA 9 1 25%
32,25FF,16,51,52
1.4.1.4 0
$5,200 $1,560 0%$3,640
Inputs Details
Likelihood of occurrence 25% to 75% 0
18 78 EMD SW System Design Activity With Duration Uncertainty
EMDDURATION, EMDCOSTS
Activity Future Task 12/3/2013 8/4/2014 175 0.00% 175 ASAP 0%10/1/2013 0 NA 0%
NA 5 0 7,28,53 0% 1.4.2.1 0
$6,150 $1,538 0%$4,613
Likelihood of occurrence > 75% 0
19 83 EMD SW Initial Design Activity With Duration Uncertainty
EMDDURATION, EMDCOSTS
Activity Future Task 8/5/2014 4/13/2015 180 0.00% 180 ASAP 0%10/1/2013 0
NA 0%
NA 7 0
8,31,53 0%
1.4.2.2 0
$12,300 0%
$4,920 $7,380
20 88 EMD SW Detailed Design Activity With Duration Uncertainty
EMDDURATION, EMDCOSTS
Activity Future Task 4/14/2015 5/16/2016 285 0.00% 285 ASAP 10/1/2013 NA NA 8 9,21,51,53 1.4.2.3 $21,600 $5,400 $16,200
22 110 EMD SW Refinements Activity With Duration Uncertainty
EMDDURATION, EMDCOSTS
Activity Future Task 5/17/2016 1/15/2018 435 0.00% 435 ASAP 10/1/2013 NA NA 9 32,25FF,51,53 1.4.2.5 $19,450 $3,890 $15,560
36 313 Prod Procure Materials Activity With Duration Uncertainty
PRODDURATION, PRODCOSTS
Activity Future Task 1/16/2018 1/27/2020 530 0.00% 530 ASAP 10/1/2013 NA NA 34 37SS+140Days,49,54
1.5.2.1 $28,700 $20,090 $8,610
37 314 Prod Air Vehicle Activity With Duration Uncertainty
PRODDURATION, PRODCOSTS
Activity Future Task 7/31/2018 6/14/2021 750 0.00% 750 ASAP 10/1/2013 NA NA 36SS+140Days 38SS+80Days,49,48FF,54
1.5.2.2 $48,650 $43,785 $4,865
38 316 Prod IAT&C Activity With Duration Uncertainty
PRODDURATION, PRODCOSTS
Activity Future Task 11/20/2018 3/8/2021 600 0.00% 600 ASAP 10/1/2013 NA NA 37SS+80Days 41SS,42SS,40SS+180Days,39,48FF,45FF+45Days
1.5.2.3 $16,426 $0 $16,426
40 317 Prod ST&E Activity With Duration Uncertainty
PRODDURATION, PRODCOSTS
Activity Future Task 7/30/2019 7/26/2021 520 0.00% 520 ASAP 10/1/2013 NA NA 38SS+180Days 48FF,49 1.5.3 $8,000 $6,400 $1,600
41 319 Prod Common Support Equipment
Activity With Duration Uncertainty
PRODDURATION, PRODCOSTS
Activity Future Task 11/20/2018 12/16/2019 280 0.00% 280 ASAP 10/1/2013 NA NA 38SS 49,54 1.5.4 $12,000 $9,600 $2,400
42 320 Prod Peculiar Support Equipment
Activity With Duration Uncertainty
PRODDURATION, PRODCOSTS
Activity Future Task 11/20/2018 10/21/2019 240 0.00% 240 ASAP 10/1/2013 NA NA 38SS 49,54 1.5.5 $16,500 $14,025 $2,475

2017 NASA Cost and Schedule Symposium 9

Identifying Model Changes

 JCL models go through several changes during their lifecycle

 During project development the underlying schedule structure may change
 During drops to the SRB leading up to the final review the schedule and modeling
inputs may change
 After model is provided to SRB, SRB analysts may change model inputs
(e.g., uncertainty distributions, risk attributes, add/delete risks)
 The JACS development team assessed the requirement and identified the
need to compare differences in schedule structure and model inputs
 Design was based on adding functionality complementary (not supplementary) to
MS Project
– MS Project desktop does not have a model tracking feature
– MS Project Server has functionality to conduct journal tracking if the feature is enabled
and server database is structured appropriately
 Design was based on flow of models, typically models are dropped/released over
periods of time; so a process structured on comparing model snapshots was
deemed the best approach
 A new feature was implemented in JACS to provide model comparison
 Model Comparison – checks structure and inputs between JCL model files and JCL
scenarios within a specific Model

2017 NASA Cost and Schedule Symposium 10

Model Comparison—Example
SUMMARY
JCL Model Comparison - Summary
Report Date 8/7/2017

 Feature compares Active Project Name: JACS Example File 02a Advanced - with Risk Factors
Comparison Project Name: JACS Example File 02a Advanced - with Risk Factors chng for model comp
 Tasks
Schedule Status
 Calendars Description
Status Date
Active Schedule
9/1/2013
Comparison Schedule
9/1/2013


Current Start 10/1/2013 10/1/2013
Dates Current Finish
Project Duration
7/26/2021
2040 Days
8/23/2021
2060 Days


Project Total Cost $267,476.000 $267,476.000
Duration Estimated Workdays Remaining
Project Percent Complete
2040 Days
0.0%
2060 Days
0.0%

 Logic Schedule Statistics

 Costs Description Active Schedule

Total
Comparison Schedule Delta
WITH Duration Uncertainty
Active Schedule Comparison Schedule Delta Activ


Total Lines in IMS 54 52 2 26 24 2
Distributions Total Activities
Total Milestones
15
25
15
23 2
15
11
15
9 2


Schedule Milestones 8 8 0 0
Risks Risk Events
Risk Factors
7
4
5
4
2 7
4
5
4
2

Start/Finish Hammock Milestones 6 6 0 0

Go To Excel File
DETAILS

JCL Model Comparison - Details

Calendar Start Finish Duration (Days)
Unique Active Comparison Comparis
Task ID Task ID Task ID Active Schedule Task Names Comparison Schedule Task Names Task Type Active Comparison Match Active Comparison Delta Active Comparison Delta Active on Delta
230 1 1 Missile System Project Missile System Project Summary Standard Standard Yes 10/1/2013 10/1/2013 7/26/2021 8/23/2021 -20 Days 2040 2060 -20
329 2 2 Contract Award Delay Contract Award Delay Summary Standard Standard Yes 10/1/2013 10/1/2013 12/2/2013 12/2/2013 45 45
330 3 3 Planned Award Planned Award Milestone Standard Standard Yes 10/1/2013 10/1/2013 10/1/2013 10/1/2013 0 0
331 4 4 Delayed Award Delayed Award Activity Standard Standard Yes 10/1/2013 10/1/2013 12/2/2013 12/2/2013 45 45
284 5 5 Contract Award Contract Award ProgramEvent Standard Standard Yes 12/2/2013 12/2/2013 12/2/2013 12/2/2013 0 0
280 6 6 Milestones Milestones Summary Standard Standard Yes 8/4/2014 8/4/2014 5/16/2016 6/13/2016 -20 Days 465 485 -20
281 7 7 EMD SDR Milestone EMD SDR Milestone ProgramEvent Standard Standard Yes 8/4/2014 8/4/2014 8/4/2014 8/4/2014 0 0
282 8 8 EMD PDR Milestone EMD PDR Milestone ProgramEvent Standard Standard Yes 4/13/2015 5/11/2015 -20 Days 4/13/2015 5/11/2015 -20 Days 0 0
283 9 9 EMD CDR Milestone EMD CDR Milestone ProgramEvent Standard Standard Yes 5/16/2016 6/13/2016 -20 Days 5/16/2016 6/13/2016 -20 Days 0 0
190 10 10 EMD Phase EMD Phase Summary Standard Standard Yes 12/2/2013 12/2/2013 1/15/2018 2/12/2018 -20 Days 1075 1095 -20
2 11 11 EMD Hardware EMD Hardware Summary Standard Standard Yes 12/3/2013 12/3/2013 8/7/2017 9/4/2017 -20 Days 960 980 -20
16 12 12 EMD HW System Design EMD HW System Design Activity Standard Standard Yes 12/3/2013 12/3/2013 8/4/2014 8/4/2014 175 175
15 13 13 EMD HW Initial Design EMD HW Initial Design Activity Standard Standard Yes 8/5/2014 8/5/2014 4/13/2015 5/11/2015 -20 Days 180 200 -20
14 14 14 EMD HW Detailed Design EMD HW Detailed Design Activity Standard Standard Yes 4/14/2015 5/12/2015 -20 Days 5/16/2016 6/13/2016 -20 Days 285 285
13 15 15 EMD HW Final Design EMD HW Final Design Activity Standard Standard Yes 5/17/2016 6/14/2016 -20 Days 8/7/2017 9/4/2017 -20 Days 320 320
286 16 n/a EMD HW Malfunction (RR#1) no match RiskEvent

2017 NASA Cost and Schedule Symposium 11

 Validating Calculations

 Baseline Delta Report was added to verify that JACS calculation

will return similar values* as entered into the Project/P6 model
Duration Duration Duration Δ Start Start Start Δ Finish Finish Finish Δ Is Critical Is Critical Is Critical Total Slack Total Slack Total Slack Δ
Task Name
(Project) (JACS) (days) (Project) (JACS) (days) (Project) (JACS) (days) (Project) (JACS) Match (Project) (JACS) (days)

Missile System Project 2040 2039 (1) 10/1/2013 10/1/2013 0.00 7/26/2021 7/26/2021 0.00 Yes Yes Yes 0 -1 (1)
Contract Award Delay 45 45 0 10/1/2013 10/1/2013 0.00 12/2/2013 12/2/2013 0.00 Yes Yes Yes 0 0 0
Planned Award 0 0 0 10/1/2013 10/1/2013 0.00 10/1/2013 10/1/2013 0.00 Yes Yes Yes 0 0 0
Delayed Award 45 45 0 10/1/2013 10/1/2013 0.00 12/2/2013 12/2/2013 0.00 Yes Yes Yes 0 -1 (1)
Contract Award 0 0 0 12/2/2013 12/2/2013 0.00 12/2/2013 12/2/2013 0.00 Yes Yes Yes 0 -1 (1)
Milestones 465 465 0 8/4/2014 8/4/2014 0.00 5/16/2016 5/16/2016 0.00 Yes Yes Yes 0 0 0
EMD SDR Milestone 0 0 0 8/4/2014 8/4/2014 0.00 8/4/2014 8/4/2014 0.00 Yes Yes Yes 0 0 0
EMD PDR Milestone 0 0 0 4/13/2015 4/13/2015 0.00 4/13/2015 4/13/2015 0.00 Yes Yes Yes 0 0 0
EMD CDR Milestone 0 0 0 5/16/2016 5/16/2016 0.00 5/16/2016 5/16/2016 0.00 Yes Yes Yes 0 0 0
EMD Phase 1075 1075 0 12/2/2013 12/2/2013 0.00 1/15/2018 1/15/2018 0.00 Yes Yes Yes 0 0 0
EMD Hardware 960 960 0 12/3/2013 12/2/2013 0.00 8/7/2017 8/7/2017 0.00 Yes Yes Yes 0 0 0
EMD HW System Design 175 175 0 12/3/2013 12/2/2013 0.00 8/4/2014 8/4/2014 0.00 Yes Yes Yes 0 0 0
EMD HW Initial Design 180 180 0 8/5/2014 8/4/2014 (0.13) 4/13/2015 4/13/2015 0.00 Yes Yes Yes 0 0 0
EMD HW Detailed Design 285 285 0 4/14/2015 4/13/2015 (0.25) 5/16/2016 5/16/2016 0.00 Yes Yes Yes 0 0 0
EMD HW Final Design 320 320 0 5/17/2016 5/16/2016 (0.38) 8/7/2017 8/7/2017 0.00 No No Yes 115 115 0
EMD HW Malfunction (RR#1) 0 0 0 8/7/2017 8/7/2017 0.00 8/7/2017 8/7/2017 0.00 No No Yes 115 115 0
EMD Software 1075 1075 0 12/3/2013 12/2/2013 0.00 1/15/2018 1/15/2018 0.00 Yes Yes Yes 0 0 0
EMD SW System Design 175 175 0 12/3/2013 12/2/2013 0.00 8/4/2014 8/4/2014 0.00 Yes Yes Yes 0 0 0
EMD SW Initial Design 180 180 0 8/5/2014 8/4/2014 (0.13) 4/13/2015 4/13/2015 0.00 Yes Yes Yes 0 0 0

* Note: JACS calculation engine recreates the schedule logic internally in order to quickly and efficiently run the CPM (critical path method) engine

2017 NASA Cost and Schedule Symposium 12

Results Insight

 The ability to quickly access and understand the model results is a

significant strength of JACS and a key need for JCL analysts
 Several New Features were implemented in JACS to provide rapid
access for JCL analysts
 Critical Path Report
– Provides insight into the deterministic critical path and the probabilistic
critical path
– Enables identification of tasks that are hidden drivers (on probabilistic and not
on deterministic) or are non-drivers (on deterministic and not on probabilistic)
 Active Risk Impact
– Provides insight into true impact of how a risk will affect results when it occurs
– This focuses only on the conditional scenarios when the risk is active and gives
better insight into how much of a potential driver a risk can be if it’s likelihood
were to increase
 Enhanced Insight Viewer
– Additional reports were added to the Insight Viewer and also the ability to
specifically identify a task or summary to evaluate

2017 NASA Cost and Schedule Symposium 13

 Results Insight – Critical Path Analysis
ACTIVITIES

Critical Path Analysis Report for Activities

Schedule: JACS Example File 02a Advanced - with Risk Factors.mpp
Report Date: 8/7/2017

Unique Mean
Task ID Activity Name ID Risk ID Start Duration On Plan CP Criticality Duration Uncertainty Successors Predecessors
4 Delayed Award 331 10/1/2013 55.06 Yes 81% Tri(0,45,120,0,100) 5 3
12 EMD HW System Design 16 12/2/2013 196.73 Yes 46% Tri*(90,100,120,10,90);Correl(EMDDURATION=0.5)
7,27,52 5
18 EMD SW System Design 78 12/2/2013 199.91 Yes 39% Tri*(90,100,120,10,90);Correl(EMDDURATION=0.5)
7,28,53 5
13 EMD HW Initial Design 15 8/4/2014 202.23 Yes 44% Tri*(90,100,120,10,90);Correl(EMDDURATION=0.5)
8,30,52 7
19 EMD SW Initial Design 83 8/4/2014 205.63 Yes 39% Tri*(90,100,120,10,90);Correl(EMDDURATION=0.5)
8,31,53 7
26 EMD System Design Multi-Impact Risk (RR#3) 289 EMD_SysDes_Multi 8/4/2014 42.23 Yes 89%
27 HW System Design Risk 290 HWSysDes3.1 8/4/2014 4.40 Yes 52% Tri(15,25,45,10,90);Correl(SYSDESRISK=0.7)
7 12
28 SW System Design Risk 291 SWSysDes3.2 8/4/2014 5.12 Yes 45% Tri(20,30,50,10,90);Correl(SYSDESRISK=0.7)
7 18
14 EMD HW Detailed Design 14 4/13/2015 343.02 Yes 33% Tri*(90,100,120,10,90);Correl(EMDDURATION=0.5)
9,51,52 8
20 EMD SW Detailed Design 88 4/13/2015 348.68 Yes 49% Tri*(90,100,120,10,90);Correl(EMDDURATION=0.5)
9,21,51,53 8
29 EMD Initial Design Multi-Impact Risk (RR#4) 292 EMD_InitDes_Multi4/13/2015 45.25 Yes 76%
30 HW Initial Design Risk 293 HWInitDes4.1 4/13/2015 5.85 Yes 32% Tri(15,25,45,10,90);Correl(INITDESRISK=0.7)
8 13
31 SW Initial Design Risk 294 SWInitDes4.2 4/13/2015 6.69 Yes 45% Tri(20,30,50,10,90);Correl(INITDESRISK=0.7)
8 19
15 EMD HW Final Design 13 5/16/2016 384.86 No 18% Tri*(90,100,120,10,90);Correl(EMDDURATION=0.5)
32,25FF,16,51,52 9
21 EMD SW Design Failure (RR#2 ) 287 SW_RR#2 5/16/2016 39.18 Yes 75% Tri(80,120,180,10,90) 9 20
22 EMD SW Refinements 110 5/16/2016 532.50 Yes 63% Tri*(90,100,120,10,90);Correl(EMDDURATION=0.5)
32,25FF,51,53 9
16 EMD HW Malfunction (RR#1) 286 HW_RR#1 8/7/2017 34.86 No 63% Tri(140,160,220,10,90) 32,25FF 15
36 Prod Procure Materials 313 1/15/2018 643.67 Yes 81% Tri*(85,100,125,10,90);Correl(PRODDURATION=0.7)
37SS+140Days,49,54 34
37 Prod Air Vehicle 314 7/30/2018 909.57 Yes 81% Tri*(85,100,125,10,90);Correl(PRODDURATION=0.7)
38SS+80Days,49,48FF,5436SS+140Days
38 Prod IAT&C 316 11/19/2018 625.86 Yes 40% Tri*(85,100,125,10,90);Correl(PRODDURATION=0.7)
41SS,42SS,40SS+180Days,39,48FF,45FF+45Days
37SS+80Days
41 Prod Common Support Equipment 319 11/19/2018 341.58 No 0% Tri*(70,105,140,10,90);Correl(PRODDURATION=0.7)
49,54 38SS
42 Prod Peculiar Support Equipment 320 11/19/2018 292.75 No 0% Tri*(70,105,140,10,90);Correl(PRODDURATION=0.7)
49,54 38SS
40 Prod ST&E 317 7/29/2019 546.06 Yes 40% Tri*(70,105,140,10,90);Correl(PRODDURATION=0.7)
48FF,49 38SS+180Days
39 Air Vehicle Test Failure RR#5 332 AV_RR#5 3/8/2021 14.88 No 0% Tri(80,140,220,10,90) 48FF,45FF+45Days 38

RISKS

Critical Path Analysis Report for Risks

Schedule: JACS Example File 02a Advanced - with Risk Factors.mpp
Report Date: #######
When Risk is Active
Unique Conditional
Activated Task ID Risk Name ID Risk ID Start On Plan CP Criticality Likelihood Criticality Mean Duration Mean TI Cost Mean TD Cost Successors Predecessors
Yes 26 EMD System Design Multi-Impact Risk (RR#3) 289 EMD_SysDes_Multi
Yes 27 HW System Design Risk 290 HWSysDes3.1 8/4/2014 Yes 7.80% 15.00% 52.00% 29.00 $543.00 $0.00 7 12
Yes 28 SW System Design Risk 291 SWSysDes3.2 8/4/2014 Yes 6.80% 15.00% 45.33% 34.00 $888.00 $0.00 7 18
Yes 29 EMD Initial Design Multi-Impact Risk (RR#4) 292 EMD_InitDes_Multi
Yes 30 HW Initial Design Risk 293 HWInitDes4.1 4/13/2015 Yes 6.40% 20.00% 32.00% 29.00 $582.00 $0.00 8 13
Yes 31 SW Initial Design Risk 294 SWInitDes4.2 4/13/2015 Yes 9.00% 20.00% 45.00% 33.00 $1,006.00 $0.00 8 19
Yes 21 EMD SW Design Failure (RR#2 ) 287 SW_RR#2 5/16/2016 Yes 22.60% 30.00% 75.33% 131.00 $3,564.00 $0.00 9 20
Yes 16 EMD HW Malfunction (RR#1) 286 HW_RR#1 8/7/2017 No 12.60% 20.00% 63.00% 174.00 $1,041.00 $0.00 32,25FF 15
Yes 39 Air Vehicle Test Failure RR#5 332 AV_RR#5 3/8/2021 No 0.00% 10.00% 0.00% 149.00 $11,191.00 $0.00 48FF,45FF+45Days 38

2017 NASA Cost and Schedule Symposium 14

 Results Insight – Active Risk Impact
Risk Event Ranking
Schedule: JACS Example File 02a Advanced - with Risk Factors.mpp
Report Date:8/7/2017

Information Ranking Relative Contribution Probabilities Impacts

Activity Based on Conditional Overall Mean Active Mean CoC Mean Overall CoC Active CoC
Activated Risk Description Task ID Risk ID Overall Active Likelihood Criticality
Impacted Active Criticality Duration Duration Duration Duration Impact Duration Impact
Yes EMD HW Malfunction (RR#1) 16 HW_RR#1 15 1 35.9% 41.5% 20.0% 12.6% 63.0% 34.9 174.3 174.9 22.0 110.2
Yes EMD SW Design Failure (RR#2 ) 21 SW_RR#2 20 2 48.4% 37.3% 30.0% 22.6% 75.3% 39.2 130.6 131.6 29.7 99.1
Yes EMD System Design Multi-Impact Risk (RR#3) 26 EMD_SysDes_Multi
Yes HW System Design Risk 27 HWSysDes3.1 12 4 3.8% 5.8% 15.0% 7.8% 52.0% 4.4 29.3 29.8 2.3 15.5
Yes SW System Design Risk 28 SWSysDes3.2 18 3 3.9% 6.1% 15.0% 6.8% 45.3% 5.1 34.1 35.5 2.4 16.1
Yes EMD Initial Design Multi-Impact Risk (RR#4) 29 EMD_InitDes_Multi
Yes HW Initial Design Risk 30 HWInitDes4.1 13 6 3.3% 3.8% 20.0% 6.4% 32.0% 5.8 29.2 31.5 2.0 10.1
Yes SW Initial Design Risk 31 SWInitDes4.2 19 5 4.8% 5.5% 20.0% 9.0% 45.0% 6.7 33.4 32.6 2.9 14.7
Yes Air Vehicle Test Failure RR#5 39 AV_RR#5 38 7 0.0% 0.0% 10.0% 0.0% 0.0% 14.9 148.8 0.0 0.0 0.0

 Ranking: Ranking of risk events based on contribution to schedule when risk is on the critical path (order of the Active
Relative Contribution values)
 Relative Contribution: Proportion of a risk to the sum of all risks for the Overall/Active CoC Duration Impacts
 Probabilities
 Likelihood: Probability risk is active (user-defined)
 Criticality: Probability risk is on the critical path
 Conditional Criticality: Probability risk is critical, given it is active

 Impacts
 Overall Mean Duration: Average duration of risk, considering all iterations
 Active Mean Duration: Average duration of risk, considering only those iterations when it is active
 Conditional Criticality Mean Duration: Average duration of risk, considering only those iterations when it is active and lands on the critical path
 Overall Conditional Criticality Duration Impact: Expected impact to the critical path considering all iterations (CoC Mean * Criticality)
 Active Conditional Criticality Duration Impact: Expected impact to the critical path considering only those iterations when it lands on the critical
path (CoC Mean * Conditional Criticality)

2017 NASA Cost and Schedule Symposium 15

Compare Scenario Results

 A key aspect of JCL analysis is to assess scenarios, including in the review

process where the SRB may adjust inputs they feel should be modified
(e.g., Risk and Distributions), and assess the change
 A new capability was added to Insight to allow comparison

2017 NASA Cost and Schedule Symposium 16

JCL Model Assessment

JACS Reports menu—access to tools/outputs to assess models

Note: Dashboard style reports available in JACS – Insight analysis tool

Assesses both the schedule and JACS specific entries and generates a report to help you find and
correct deviations from best practices
Summarizes and outlines tasks impacted by risk factors

Summarizes every aspect of the contents of the schedule at a high level as well as a detailed
breakdown for each row
Compares the values JACS calculates without uncertainty (often referred to as the plan values or
the point estimate values) to the values Microsoft Project calculates

Provides a comprehensive output of all the costs on all the rows in the model broken out by FY

Compares 2 JACS files in great detail—Flags any differences in inputs, outputs, and attributes

Reveals critical information regarding the effects of risk events—metrics include

conditional criticality
Provides an overview of each Task and Risk Event’s relationship to the critical path individually
that includes key metrics

Classic risk statistics report with filter options for duration & cost by task

2017 NASA Cost and Schedule Symposium 17

JACS is available to all NASA users

 JACS is included within the ACEIT software suite

 NASA has volume licensing for nine (9) NASA centers
 NASA Civil Servants and in-situ NASA Contractors have full
access to ACEIT and JACS
 JACS software is available for download from
www.oncedata.com
 One of the most frequent downloads
 JACS technical support is fully covered by the ACEIT license
 Contact ACEIT_Support@tecolote.com to enter a help ticket
 Formal JACS Training Course is available
 Has previously been held at NASA HQ, NASA LaRC, and NASA JSC

2017 NASA Cost and Schedule Symposium 18

Summary

 Developing a sound JCL model is a challenging task involving

many teams, inputs, and moving parts
 JACS’ analytical tools help assess Schedule Structure, Cost,
Risk, and Uncertainty Inputs during JCL modeling process
 Exploiting these tools/reports helps ensure the JCL model best
simulates the program/project

2017 NASA Cost and Schedule Symposium 19

 THANK YOU

2017 NASA Cost and Schedule Symposium 20