AIAA SPACE 2011 Conference & Exposition AIAA 2011-7358

27 - 29 September 2011, Long Beach, California

Modeling High Fidelity Tools for Rocket Propulsion

Subsystem Design

Jonathan L. Sharma1, Eric Stuber2, and Dimitri N. Mavris3

Aerospace Systems Design Laboratory, Georgia Institute of Technology, Atlanta, GA 30332

Moving beyond the Conceptual Design phase requires the use of higher fidelity tools.
Unfortunately, utilizing these tools requires more time and effort that push traditional
methodologies towards a single point design, resulting in a large loss of design knowledge
Downloaded by NORTHWESTERN POLYTECHICAL UNIV. on October 10, 2022 | http://arc.aiaa.org | DOI: 10.2514/6.2011-7358

and freedom. Mitigating this loss can be achieved through the use of advanced modeling
techniques that capture the behavior of higher fidelity tools. These models can then be used
to explore a vehicle design space to improve the knowledge about the design as well as
increase freedom throughout the design process. Through an example rocket propulsion
subsystem, this paper presents a method to create surrogate models of the industry standard
tool, Numerical Propulsion System Simulation.

Nomenclature
DoE = design of experiments
in2 = square inch
Isp = specific thrust
MFE = model fit error
MRE = model representation error
NPSS = numerical propulsion system simulation
psia = pounds per square inch absolute
RSE = response surface equation
RSM = response surface methodology
TVC = thrust control valve

I. Introduction

E ASIER access to Space is a goal shared by groups across the United States. Current options provide only
limited operational capabilities while future options are years away from being implemented using traditional
design methods. During the design process, going beyond Conceptual Design requires the using of higher fidelity
tools and analyses. Unfortunately, utilizing these tools requires more time and effort that push traditional
methodologies towards a single point design, resulting in a large loss of freedom in the design. In addition,
knowledge about the design space is narrowed down to a small region. Replacing these complicated tools and codes
with fast and efficient surrogate models would allow designers the ability to maintain design freedom and focus
across a larger region of the design space.
One of the major disciplines in launch vehicle sizing is propulsion. The size and power of a rocket engine is the
driving force behind what a launch vehicle looks like and what its capabilities are. Correspondingly, sizing the
propulsion system is extremely complicated. Given a certain mission or capability requirement of a launch vehicle,
one or more trajectories are defined. From these trajectories a set of propulsion performance requirements are
determined. As a vehicle progresses further along its design process, the methods to calculate the propulsions system
necessary to meet the requirements varies. Early on empirical methods are used, which progress further towards
simple theoretic calculations on the way to higher fidelity models and tools. The cost of the advancement of the
design knowledge becomes lack of design freedom due to the complexity and time involved that limits the design

1
Ph.D. Student, School of Aerospace Engineering, 270 Ferst Dr, Mail Stop 0150, AIAA Member.
2
Masters Student, School of Aerospace Engineering, 270 Ferst Dr, Mail Stop 0150, AIAA Member.
3
Boeing Professor of Advanced Aerospace Systems Analysis, School of Aerospace Engineering, 270 Ferst Dr, Mail
Stop 0150, Associate Fellow
1
American Institute of Aeronautics and Astronautics

Copyright © 2011 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved.
 space as the fidelity level increases. This paper presents a technique for approximating these high level models and
tools in a simplified form that can be applied earlier in the design process. The result is a surrogate model of the
performance of a family of rocket engines that maintains the fidelity of the industry standard code, Numerical
Propulsion System Simulation (NPSS).

II. Background

A. The Design Paradigm Shift

In the traditional design process, design requirements and technology assumptions are usually fixed, as the use of
high fidelity tools to analyze a large design space requires a significant amount of time and effort. This leads to the
exploration of only a handful of concepts, or point solutions, in the design space. There are several problems with
this point design process, as it can be described as manual, deterministic, data-driven, and discipline centric. Design
Downloaded by NORTHWESTERN POLYTECHICAL UNIV. on October 10, 2022 | http://arc.aiaa.org | DOI: 10.2514/6.2011-7358

freedom rapidly drops off, locking in a design before adequate knowledge is obtained to allow a proper assessment.
This can drive up the cost committed to the project, as design flaws must be compensated for, which can in turn lead
to costly design changes later in the design process.
Today, there is a paradigm shift in the world of design, with the goal of bringing more design knowledge and
thus maintaining more design freedom in the earlier stages of conceptual design. As new generations of
unconventional vehicles are being designed, historical data cannot be relied upon, as it has in the past. As such,
several techniques and methods exist which allow for quick and easy comparisons between designs in a large design
space. For example, parametric tradeoff environments and surrogate models. This paradigm shift can be visualized
in Figure 1, where the dashed lines represent the traditional design process and the solid lines represent the change
in design freedom, design knowledge, and cost committed due to the new methods being implemented in this
paradigm shift.1

100%

― Design Freedom
― Design Knowledge
― Cost Committed

0%
Requirements Conceptual Preliminary Detailed Production &
Definition Design Design Design Support
Figure 1. Paradigm Shift in the Traditional Design Process (Adapted from Ref. 1)

B. Traditional Vehicle Design

Traditional vehicle design is sequential and deterministic. It involves taking a few point designs and performing
more detailed design work and then choosing between the point designs. This occurs through several phases of the
design process (shown in Figure 2). The first phase is the Requirements Definition. During this stage, the
requirements imposed by the customer are processed and initial assumptions about the design are made. These
assumptions damper the design freedom by creating a very narrow design space. The next phase, Conceptual
Design, is where the initial design begins to take shape. The design space is explored as a first level of analysis is
performed with basic parametric models. Though the design knowledge increases a significant amount in this phase,
this knowledge is unevenly distributed among various aerospace disciplines. From Conceptual Design, the product
moves on to Preliminary Design, where sophisticated analyses and multidisciplinary optimization occur in an effort
to reduce risk and ensure the product will make money. The result of the Preliminary Design phase, the Preliminary
Baseline, is passed from the Advanced Design team to the Project Design team, where Detailed Design begins. It is
in the Detailed Design phase where the design is finalized and rigorous testing and fine-tuning is performed,
resulting in a Production Baseline. At this point, there is very little design freedom left, as the costs of making
changes at this stage in the design would be immense.
2
American Institute of Aeronautics and Astronautics
 The lack of design knowledge during the
traditional design process makes it very difficult Requirements Conceptual Preliminary

Advanced Design
to foresee design flaws early on during Definition Design Design
Conceptual Design. By the time proper
knowledge of these flaws is attained, the cost of Conceptual Preliminary
fixing these flaws can be substantial. Therefore, Baseline Baseline
increasing the design knowledge during the
Conceptual Design phase becomes a priority.
Analyzing the complete Conceptual Design Production and

Project Design
Allocated Baseline Detailed Design
process for ways to increase knowledge would Support
be far too complex and beyond the scope of this
work. As such, this paper focuses in on the Production
Downloaded by NORTHWESTERN POLYTECHICAL UNIV. on October 10, 2022 | http://arc.aiaa.org | DOI: 10.2514/6.2011-7358

rocket propulsion subsystem and how design Baseline

knowledge can be expanded while maintaining
design freedom. Figure 2. Traditional Development Process

C. Traditional Rocket Engine Design

Traditional design and sizing of a rocket propulsion subsystem is a complex and iterative process. Establishing a
baseline design is accomplished through an input/process/output methodology as illustrated in Figure 3. The inputs
into this method are a variety of requirements and constraints. These pass through a preliminary design process
involving various decisions and design choices in an interactive process. Once a preliminary engine design is found,
the initial sizing of the system takes place. This process continues through multiple design iterations until it
converges upon a satisfactory design. The resulting outputs include the masses, configuration, and performance
estimates.2

Figure 3. Traditional Liquid Rocket Propulsion System Design Process2

3
American Institute of Aeronautics and Astronautics
 D. Numerical Propulsion System Simulation
NPSS can take the place of the entire Sizing, Design, & Trade-offs portion of the traditional liquid rocket
propulsion system design process and return a high fidelity calculation of the performance. NPSS is a complex
object-oriented code developed at NASA Glenn that was created as a realistic physics-based model of complex
thermodynamic cycles in the form of complete aircraft engines. Over the years this capability has been expanded to
include rocket engines as well.3 NPSS allows a greater level of detail than a simple top-down design approach by
breaking down the rocket propulsion subsystem into its individual components. These components consist of valves,
propellant lines, tanks, pumps, thrust chamber, nozzle, and more. Changes to the physical and performance
parameters of these individual components combine into the overall performance of an engine system.
The NPSS engine model used in this paper is based off an RL10A-3-3A rocket engine. The RL10 engine is an
expander cycle, as shown in Figure 4. When viewed as a model in NPSS, the RL10 appears as a series of
components with fluid, heat, or torque links between them, as seen in Figure 5. The engine uses liquid oxygen as the
Downloaded by NORTHWESTERN POLYTECHICAL UNIV. on October 10, 2022 | http://arc.aiaa.org | DOI: 10.2514/6.2011-7358

oxidizer and liquid hydrogen as the fuel. The propellants are fed into the turbopumps with some initial pressure as
determined by their storage tanks. The turbopump has a two-stage pump for the fuel and singe stage for the oxidizer,
all of which share a main shaft. This shaft is driven by a turbine with fuel that has been heated through regenerative
cooling of the nozzle. Basic throttling of the engine is done through a Thrust Control Valve (TCV) that bypasses the
turbine on the fuel side of the engine.

Figure 4. RL10-3-3A Engine System Schematic5

4
American Institute of Aeronautics and Astronautics
Downloaded by NORTHWESTERN POLYTECHICAL UNIV. on October 10, 2022 | http://arc.aiaa.org | DOI: 10.2514/6.2011-7358

Figure 5. Simplified RL10 NPSS Model

E. Key Enablers
One of the key enablers for this paradigm shift is the use of physics-based approximation models, or surrogate
models. These models can replace higher fidelity tool such as NPSS that are generally too complex or time-
consuming to adequately explore the design space during Conceptual Design, while retaining a high level of
accuracy. 4 The use of probability models in conjunction with these surrogate models allows for the transition from
single point designs to the exploration and analysis of large parametric trade environments.1
A technique for surrogate modeling called Response Surface Methodology (RSM) is used in this study. It is a
linear multivariate regression technique designed to model the response of a complex system. The first step of RSM
is to identify the design variables with their ranges and the responses that are to be modeled. Next is to fit Response
Surface Equations (RSE) from the responses to data generated through an intelligent Design of Experiments (DoE),
which is a statistical tool that allows the determination of the most critical variables to the design and how much
they affect the outcome of the experiment. Finally, the RSEs are tested for goodness of fits and any transformations
or addition of higher order terms is performed as necessary. Typical RSEs are created using a second-order
quadratic equation, as seen in Eq. 1.4 If higher order terms are necessary, it is easy to modify a quadratic RSE to a
cubic RSE, or more by simply adding another set of regression coefficients for the higher order terms.

   ∑    ∑    ∑ 
 ∑
  
 
(1)

The bi,j parameters are known as the regression coefficients while the x terms are the input factors, known as the
regressors. The regressors are the factors that have the largest influence on the variability of the response, as
determined through the DoE. The regression coefficients are correlated to the impact each regressor has on the
response. These regression coefficients are typically determined through the method of least squares, which
introduces an error term that must be minimized. The resulting Eq. 2 becomes:

    ∑    ∑    ∑ 
 ∑
  
 
  (2)

To determine the regression coefficients, the least squares function4, as seen in Eq. 3, is applied and minimized.

  ∑    ∑    ∑    ∑ 
 ∑
  
 
 (3)

There are five main goodness of fit tests to be completed for each RSE. The first is to check the R squared value,
which is a measure of how much the variability in the observed response can be explained by experimental factors
and their interactions. The second test is the Actual by Predicted plot, which plots the actual data points versus their
corresponding predicted points. The third test is the Residual by Predicted plot. This shows the error between the
actual value and predicted value of the response. The final tests check the Model Fit Error (MFE) and Model
Representation Error (MRE) distributions for a low mean and standard deviation. The MFE shows the magnitude
and shape of the relative error in the fit while the MRE checks to see how the RSE predicts previously unseen data
points.

5
American Institute of Aeronautics and Astronautics
 III. Applying Response Surface Methodology

F. Design Variables and Responses

Performance of a propulsion subsystem is based off what is required to perform the mission the vehicle is being
designed for. In most cases, this comes down to the thrust and the specific impulse, Isp, of an engine. These
correspond to the power and efficiency of the engine. Therefore, the surrogate models for performance are created
with thrust and Isp as the responses.
Determining the design variables to use for the surrogate models is a combination of analyzing the traditional
liquid rocket propulsion system design process discussed earlier and the limitations of the NPSS RL10 model. The
physical geometry aspects of a rocket engine include the throat area, the area ratio of the nozzle, and the small TCV
of the RL10 engine model. The propellant aspect drives the oxidizer-to-fuel ratio to be a design variable. The
pressure distribution through an engine is a major driving force behind its performance, making the beginning
Downloaded by NORTHWESTERN POLYTECHICAL UNIV. on October 10, 2022 | http://arc.aiaa.org | DOI: 10.2514/6.2011-7358

pressures of the oxidizer and fuel tanks as well as the thrust chamber pressure all design variables. There are some
limitations on what NPSS calculates for a nozzle, leading to a loss coefficient in the nozzle to become the last design
variable for a total of eight. These variables and their ranges are summarized in Table 1.

Table 1. Variables and ranges for the 5,000 case Monte Carlo DOE
Variable Units Description Default Min Max Range
OTNK.Pt psia Oxidizer tank pressure 42.35 35 350 315
FTNK.Pt psia Fuel tank pressure 25.5 25 230 205
NOZL.Ath in2 Nozzle throat area 19.34 15 40 25
MCC.Pt_tc psia Thrust chamber pressure 468.62 400 550 150
MCC.OFR Oxidizer-to-fuel ratio 5 4 6 2
NOZL.AR Nozzle area ratio 61 40 100 60
NOZL.realLossCoef Nozzle loss coefficient 0.95 0.95 1 0.05
FTBV.Across in2 Throttle valve area 0.03 0 0.06 0.06

G. Design of Experiments
A 5,000 case Monte Carlo DoE is used with NPSS to calculate the performance of the engine as various factors
that help define the size of the engine change. A Monte Carlo DoE is one that contains random points following a
uniform distribution over a given range. The inputs into NPSS are fixed in the model, except for the oxidizer-to-fuel
ratio and the chamber pressure. These two variables serve as an initial guess for the NPSS optimizer in solving the
system of equations.
There are several reasons behind using a Monte Carlo DoE. The first of which is the nature of the problem being
tackled in this paper. The surrogate models being created use a mix of NPSS inputs and outputs among their design
variables, which impedes the ability to intelligently choose a DoE that maintains a high level of orthogonality.
The second reason involves the way the NPSS RL10 model is set up. The system of equations behind the model
solves 47 independent variables, which include two of the DoE input variables as mentioned earlier. The RL10
model does not have the necessary dependent relations to keep those two variables as anything beyond initial guess
values. As such, there is no way to intelligently control several design variables.
Lastly, NPSS fails to converge for 374 of the cases. This amounts to over 7% of the cases. When creating
surrogate models, it is a general rule of thumb that no more then 7-8% of orthogonal cases can be excluded. This
limit is reached before surrogate modeling even takes place.
In the end, instead of using an orthogonal DoE, a 5,000 case Monte Carlo is used and portioned into 3,500 cases
for fitting the surrogate models. Among these are 98 excluded cases with unrealistic values for chamber pressure.
The remaining 1,126 cases after the failed cases are removed are kept for checking the goodness of fit of the
surrogates. This data set also includes 34 excluded cases with unrealistic chamber pressures.

H. Fitting the Thrust Response Surface

The initial fit tests for thrust are shown in Figure 7. The initial fit for thrust shows an extremely good R squared
value. The second test is the Actual by Predicted plot, where the initial fit is a thin straight line very close to a
perfect fit. There is no large clumping of points and the points are spread evenly along the perfect fit line. The third

6
American Institute of Aeronautics and Astronautics
 test is the Residual by Predicted plot. Here the initial fit has trouble. Instead of a random scattering of points, fore
and aft tails sprout off the main grouping of points. This shows a pattern to the error in the initial fit which hints to
the need for higher order terms or a transformation of the response. The final tests check the MFE and MRE
distributions for a low mean and standard deviation. The initial thrust fit passes both the MFE and MRE tests.
Unfortunately, while this initial fit passes the majority of the goodness of fit tests, the Residual by Predicted plot
suggests further improvement to the fit might be necessary.
Downloaded by NORTHWESTERN POLYTECHICAL UNIV. on October 10, 2022 | http://arc.aiaa.org | DOI: 10.2514/6.2011-7358

Residual by Predicted Plot

30
20
10
0
-10
-20
-30
10000 20000 30000
Thrust Predicted

Figure 6. Third Order RSE Fit for Thrust

Upon further investigation, it is found that adding 3rd order terms to the RSE helps in modeling thrust more than
adding a transformation to the response. The resulting goodness of fit tests can be seen in Figure 6. In particular, the
Residual by Predicted plot shows improvement over the initial fit for thrust. The fore and aft tails have been reduced
and the resulting plots appears as more of a random spread. The other goodness of fit tests were previously
satisfactory and the addition of 3rd order terms does not have a detrimental effect upon them.

7
American Institute of Aeronautics and Astronautics
 Thrust Init MFE Thrust Init MRE

0.4 0.3
0.3 0.2
0.2
0.1
0.1
0
0
-0.1 -0.1
-0.2 -0.2
-0.3 -0.3
-0.4
-0.4
-0.5
-0.6 -0.5
Downloaded by NORTHWESTERN POLYTECHICAL UNIV. on October 10, 2022 | http://arc.aiaa.org | DOI: 10.2514/6.2011-7358

-0.7 -0.6

Quantiles Quantiles
100.0% maximum 0.41769 100.0% maximum 0.32648
99.5% 0.2222 99.5% 0.2131
97.5% 0.1486 97.5% 0.14552
90.0% 0.08792 90.0% 0.08726
75.0% quartile 0.0397 75.0% quartile 0.03953
50.0% median -0.0024 50.0% median -0.004
25.0% quartile -0.0385 25.0% quartile -0.0425
10.0% -0.074 10.0% -0.0823
2.5% -0.1541 2.5% -0.1843
0.5% -0.2551 0.5% -0.3465
0.0% minimum -0.7234 0.0% minimum -0.6016
Moments Moments
Mean 1.9516e-5 Mean -0.004996
Std Dev 0.0748924 Std Dev 0.0822351
Std Err Mean 0.001284 Std Err Mean 0.0024885
Upper 95% Mean 0.002537 Upper 95% Mean -0.000113
Lower 95% Mean -0.002498 Lower 95% Mean -0.009879
N 3402 N 1092

Figure 7. Initial RSE Fit for Thrust

I. Fitting the Isp Response Surface

The initial fit tests for Isp are shown in Figure 8. The initial fit for Isp shows a very good R2 value. The second test
is the Actual by Predicted plot, which shows several serious problems. The data points create a line unacceptably
thick which seems to not follow the perfect fit line. Modifications will be required to create a better fit. The initial fit
for Isp has little trouble with the third test, the Residual by Predicted plot. There is minor concern about a light
spread of points along the low-end of Isp predicted values. The final tests check the MFE and MRE distributions for
a low mean and standard deviation. The initial Isp fit passes both the MFE and MRE tests. While overall, the results
are favorable, the trouble with the Actual by Predicted plot leads to the investigation of adding higher order terms or
a transformation of the response.

8
American Institute of Aeronautics and Astronautics
Downloaded by NORTHWESTERN POLYTECHICAL UNIV. on October 10, 2022 | http://arc.aiaa.org | DOI: 10.2514/6.2011-7358

Figure 8. Initial RSE Fit for Isp

Compared to performing a transformation on the response, it is found that adding 3rd order terms to the RSE
helps in modeling Isp. The resulting goodness of fit tests can be seen in Figure 9. In particular, the Actual by
Predicted plot shows great improvement over the initial fit. The thickness previously seen in the initial fit has been
somewhat reduced and the collection of predicted points seem to follow along the perfect fit line. The Residual by
Predicted plot also contains improvement in the form of a more scattered random pattern similar to that of a shotgun
blast pattern.

9
American Institute of Aeronautics and Astronautics
Downloaded by NORTHWESTERN POLYTECHICAL UNIV. on October 10, 2022 | http://arc.aiaa.org | DOI: 10.2514/6.2011-7358

Figure 9. Third Order RSE Fit for Isp

IV. Conclusion
Advanced design methods not currently used in the traditional design process were utilized in this paper with the
intent of bringing more design knowledge earlier into the design process. This design knowledge can be obtained by
intelligently regressing high fidelity tools using surrogate modeling techniques. RSM is one such technique and is
utilized for the rocket propulsion subsystem example in this paper through modeling the industry standard code,
NPSS. This allows all the knowledge that NPSS can bring to the overall vehicle design to be used during Conceptual
Design. Furthermore, RSM can be used beyond just rocket propulsion subsystems to bring disciplinary codes related
to aerodynamics, thermodynamics, and more into the early phases of design. In the end, vehicle-centric parametric
tradeoff environments can be created during the Conceptual Design phase that have the same high fidelity level as
expected of the Preliminary Design phase. With this, designers can explore a large design space instead of only
looking at the handful of point solutions that the traditional design process generate. These techniques enable
designers to go beyond the Conceptual Design phase without sacrificing crucial freedom in the design, thus
shortening design time and reducing cost.

References
1
Mavris, D.N., DeLaurentis, D.A., Bandte, O., Hale, M.A., “A Stochastic Approach to Multi-disciplinary Aircraft Analysis
and Design”, AIAA 98-0912.
2
Henry, Gary N., Wiley J. Larson, and Ronald W. Humble. Space Propulsion Analysis and Design. New York: McGraw-Hill,
1995. Print.
3
NASA. NPSS User Guide (software release NPSS_1.6.5). 2008.
4
Myers, Raymond H., Douglas C. Montgomery, and Christine M. Anderson-Cook. Response Surface Methodology: Process
and Product Optimization Using Designed Experiments. Hoboken, NJ: Wiley, 2009.
5
Binder, M., Tomsik, T., Veres, J.P. “RL10A-3-3A Rocket Engine Modeling Project.” NASA TM-107318. 1997.

10
American Institute of Aeronautics and Astronautics