Journal of Fluids and Structures 104 (2021) 103316

Contents lists available at ScienceDirect

Journal of Fluids and Structures

journal homepage: www.elsevier.com/locate/jfs

Data-driven prediction of unsteady pressure distributions

based on deep learning
∗
Vladyslav Rozov , Christian Breitsamter
Chair of Aerodynamics and Fluid Mechanics, Technical University of Munich, Boltzmannstr. 15, 85748 Garching, Germany

article info a b s t r a c t

Article history: In the present work, an efficient Reduced-Order Model is developed for the prediction
Received 19 September 2020 of motion-induced unsteady pressure distributions. The model is trained on the basis of
Received in revised form 4 February 2021 synthetic data generated by full-order Computational Fluid Dynamics (CFD) simulations.
Accepted 12 May 2021
The nonlinear identification task is to predict a snapshot representing the pressure
Available online 27 May 2021
distribution for the current time step based on respective snapshots of previous time
Keywords: steps and applied excitation. Once a Reduced-Order Model is conditioned on training
Deep learning data, it can predict sequences of the pressure distribution in a recurrent manner based on
Unsteady aerodynamics the excitation signal. Hence, it is able to capture the motion-induced nonlinear unsteady
Reduced-Order Models aerodynamics for a given configuration at fixed free-stream conditions. In this way,
Computational Fluid Dynamics computationally extensive CFD simulations can be substituted by the application of the
LANN model more efficient Reduced-Order Model. The nonlinear behavior of the aerodynamic system
Transonic flight
is captured based on a deep convolutional neural network. The performance of the
Reduced-Order Model is demonstrated based on the LANN (Lockheed-Georgia, Air Force
Flight Dynamics Laboratory, NASA-Langley and NLR) wing performing high-amplitude
pitching motion in transonic flow. The unsteady aerodynamics of the considered test
case is dominated by nonlinear effects due to complex moving shock structures both on
the upper and lower surface of the wing. The Reduced-Order Model yields a superior
prediction accuracy at a speed-up of more than three orders of magnitude compared to
the employed CFD method.
© 2021 Elsevier Ltd. All rights reserved.

1. Introduction

Unsteady aerodynamic loads acting on an aircraft over the entire flight envelope are one of the driving factors
determining the aircraft’s structural design. Therefore, an accurate prediction of aerodynamic loads is of high importance
for the aircraft design and qualification.
A wide range of efficient computational methods for the prediction of unsteady aerodynamic loads is available. For
subsonic applications, the corrected Doublet-Lattice Method (DLM) (Albano and Rodden, 1969; Giesing et al., 1971;
Rodden et al., 1972) and the Vortex-Lattice Method (Murua et al., 2012) are well established means for the prediction
of unsteady pressure distributions. For the supersonic flight regime, techniques such as supersonic-DLM (Giesing and
Kalman, 1975), Mach Box Method (Pines et al., 1955) and Harmonic Gradient Method (Chen and Liu, 1985) are used.
However, the methods are based on linear potential theory. Therefore, they fail as soon as the relevance of viscosity or
nonlinear compressibility effects of the flow is no longer negligible. This is for instance the case for flow problems that are

∗ Corresponding author.
E-mail address: vladyslav.rozov@aer.mw.tum.de (V. Rozov).

https://doi.org/10.1016/j.jfluidstructs.2021.103316
0889-9746/© 2021 Elsevier Ltd. All rights reserved.
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

dominated by nonlinear flow phenomena such as transonic flow regions or flow separations. Hence, high-fidelity methods
such as Computational Fluid Dynamics (CFD) have to be employed for problems where nonlinear effects are involved.
However, although essential for the development of transport and military aircraft, the accurate prediction of unsteady
aerodynamic loads based on time-accurate unsteady CFD is a challenge particularly due to its high computational cost.
Small Disturbance CFD (SD CFD) methods represent a means to address this limitation. Such linearized methods can be
derived from Euler or Navier–Stokes equations by assuming a sufficiently small harmonic disturbance of the flow variables
at frequencies of interest around a reference state (Hall and Crawley, 1989; Kreiselmaier and Laschka, 2000; Pechloff
and Laschka, 2006). The reference state itself may well be characterized by nonlinear flow phenomena. This approach
eliminates the time dependence of the problem due to its formulation in the frequency domain. The equations are solved
directly for the complex-valued first harmonic of the disturbed flow. Thus, about one order of magnitude faster analyses of
the unsteady aerodynamics are facilitated compared to time-accurate CFD simulations (Dufour et al., 2010; Widhalm and
Thormann, 2017). Nevertheless, these techniques are less suitable for high-amplitude oscillations or dynamic maneuvers
of aircraft, where dynamic nonlinear effects may occur.
An alternative approach to the prediction of aerodynamic loads is based on system identification. This class of methods
is characterized by the derivation of a mathematical black-box model based on input and output data of the system. The
approach is data-driven and, therefore, is in contrast to modeling techniques based on first principles such as CFD and
DLM. The black-box models are characterized by a reduced number of degrees of freedom compared to the full-order
system. For this reason they are often referred to as Reduced-Order Models (ROMs).
The identification process of a black-box model, often referred to as training, can be performed on the basis of either
experimental or simulation data. ROMs are often trained with synthetic data generated by high-fidelity modeling methods,
such as CFD. While providing satisfactory prediction accuracy, such ROMs often exceed high-fidelity methods in terms of
efficiency by several orders of magnitude. Data synthesis for system identification based on simulations provides several
advantages. The design of experiments for aerodynamic systems is less restrictive in simulations than in experiments.
Furthermore, the data acquisition based on high-fidelity simulations requires often less expenditure compared to the
experimental approach and can be highly automated.
Depending on whether the dynamic behavior of the system is either linear or nonlinear, different methods of system
identification are employed. However, the main focus of this work is on nonlinear modeling. As nonlinear system
identification does not impose constraints arising from linearity assumptions, it is applicable to a wider range of flow
problems. Various techniques such as Volterra series (Silva, 1993), Wiener models (Kou et al., 2016) and Kriging (Glaz et al.,
2010) have been applied to nonlinear system identification in the context of aerodynamics. In addition, artificial dense
neural networks have found a widespread application in the context of aerodynamic as well as aeroelastic applications.
Recurrent Neural Networks (RNNs) conditioned on sensor measurements in the wind tunnel have been used in Faller and
Schreck (1997) to predict the dynamics of pressure coefficients at considered sensor positions on an airfoil. In Mannarino
and Mantegazza (2014), RNNs have been used to predict limit cycles of an aeroelastic system. In Marques and Anderson
(2001), the identification of a temporal neural network for the prediction of unsteady aerodynamic forces in transonic flow
has been performed on the basis of Euler CFD simulation data. An artificial neural network has been trained on simulation
data in Voitcu and Wong (2003) to model an aeroelastic system with structural nonlinearities. Radial Basis Function Neural
Networks (RBF-NNs) have been employed to the analysis of limit-cycle oscillations and flutter in Zhang et al. (2012, 2016).
In Winter and Breitsamter (2014), RBF-NNs have been used to predict unsteady aerodynamic loads on an airfoil. Moreover,
a neuro-fuzzy approach with local linear sub-models proved to be well-suited for unsteady aerodynamic applications due
to its robust behavior for multi-step ahead predictions (Winter and Breitsamter, 2016b, 2018).
As soon as the spatio-temporal behavior of an aerodynamic system is of interest, the application of the nonlinear ROMs
mentioned above is not feasible due to the immense number of spatially distributed output variables. This is for instance
the case when it comes to the task of reduced-order modeling of unsteady surface pressure distributions. Several authors
addressed this issue by combining established methods for capturing of the temporal behavior with techniques aiming
at a dimensionality reduction of the spatially distributed degrees of freedom of the aerodynamic system. In Park et al.
(2013), Proper Orthogonal Decomposition (POD) is combined with a neural network to perform accurate and efficient aero-
structural wing design optimization. A combination of the POD and RBF-NNs has been used in Lindhorst et al. (2014) to
model a two-dimensional aeroelastic system with strong aerodynamic nonlinearities occurring in the transonic regime. A
recurrent local linear neuro-fuzzy approach is employed in combination with POD to predict the motion-induced unsteady
pressure distribution around the LANN (Lockheed-Georgia, Air Force Flight Dynamics Laboratory, NASA-Langley and NLR)
wing, which is dominated by nonlinear effects in the transonic regime (Winter and Breitsamter, 2016a). Generalization
capabilities of two different RNNs have been compared in Wang et al. (2020) for the use in conjunction with POD to predict
both integral and spatially distributed aerodynamic quantities of an airfoil in transonic flow. ROM predictions using POD
for the spatial dimensionality reduction show greater deviations even when using a high number of POD modes for areas
where higher spatial gradients occur. Hence, the prediction of unsteady pressure distributions in the transonic regime is
a challenge for such ROMs due to moving shocks specified by high gradients of pressure.
In the field of computer vision, great advances in image pattern recognition have been observed due to the use of
convolutional neural networks (CNNs). In contrast to dense neural networks, CNNs are regularized by design as they
are based on the convolutional operation, which leads to sparse connectivity (Goodfellow et al., 2016). Particularly
successful are deep architectures of neural networks consisting of several stacked layers. The methodology based on such
2
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

Fig. 1. Processing of CFD data for the ROM input. (For interpretation of the references to color in this figure legend, the reader is referred to the
web version of this article.)

architectures is referred to as Deep Learning (DL). Deep CNNs are considered as state-of-the-art methods for numerous
applications in computer vision (Goodfellow et al., 2016). Among other areas, remarkable results were achieved in
predicting future frames of image sequences from videos (Finn et al., 2016; Lotter et al., 2016; Vougioukas et al., 2018).
Based on videos from sports, it was shown in Mathieu et al. (2016) that CNNs are able to capture the dynamics of moving
persons and objects subjected to the influence of the laws of nature. In Shi et al. (2015, 2017), recurrent CNNs were
successfully applied to precipitation forecasting demonstrating the ability to capture complex spatio-temporal dynamics
of physical systems. Inspired by these research activities, the objective of the present work is the employment of modern
CNN architectures for an accurate and efficient prediction of complex motion-induced unsteady pressure distributions
over a wing.

2. Reduced order modeling with deep learning

This section presents an overview of the reduced-order modeling, that is based on a deep convolutional neural network
architecture. Details regarding the processing of CFD data for the ROM input and specific modules of the network are
introduced. Furthermore, relevant information related to the training algorithm is provided.

2.1. CFD data processing for the Reduced Order Modeling

The aim of the Reduced-Order Model is to predict motion-induced unsteady surface pressure distributions. As the input
to the ROM needs to be a multidimensional array, CFD data have to be preprocessed accordingly. Throughout the work,
the dimensionless formulation of pressure with the following definition is used:
p − p∞
cp = (1)
1
2
ρ∞ U∞
2

with pressure, density and flow velocity p, ρ and U, respectively. Flow variables of the free stream are denoted by the
subscript ∞.
For each simulated time step k, pressure distributions over cell-midpoints of the CFD surface mesh cp (τk , x, y, z) are
available. Here, τk denotes the dimensionless time at time step k. Since a structured CFD mesh is employed for the
simulation process, the cells are aligned along straight lines both in span-wise and chord-wise direction. Moreover, the
wing geometry is resolved chord-wise by a constant number of cells over the span. Hence, the discretization of the wing
surface corresponds topologically to a regular rectangular grid, where each cell can be addressed by its index (i, j). Here,
indices i and j specify the number of a cell in chord-wise and span-wise directions, respectively. Thus, the data structure
representing the pressure distribution cp (τk , x, y, z) over the cell-midpoints of the CFD surface mesh can be unfolded
yielding a two-dimensional array C p,k . The size of C p,k is H × W if the wing is discretized with H /2 and W cells in
chord-wise and span-wise directions, respectively. The necessary spatial information is retained after the unfolding, as
the connection of each cell to its neighbors can be derived from the index assignment. Therefore, the coordinates of
the cell-midpoints can be omitted as input to the ROM. The unfolding transformation is illustrated in Fig. 1. Orange and
green lines indicate the edges of the top side and bottom side of the wing, respectively. It is important to note that an
3
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

unstructured discretization of the computational domain does not impose a limitation for the use with the proposed ROM.
To make the input ROM-compatible for such a case, structured data can be derived from unstructured data employing
interpolation techniques, e.g. Harder and Desmarais (1972).
In a subsequent step, the array size H × W is downscaled by linear interpolation from 146 × 37 to 128 × 32 (27 × 25 ).
While the slightly reduced resolution does not affect the spatial level of detail of the pressure distribution, powers of two
as the input array size are more suitable for the processing by the neural network used in this work.
As the last preprocessing step, all three data sets are normalized. For this purpose, each temporal snapshot of the
pressure distribution C p,k is decomposed as follows:

C p,k = C p + ∆C p,k (2)

with array C p corresponding to the mean pressure distribution (see Fig. 5) and instantaneous difference ∆C p,k . As C p does
not carry additional information regarding the unsteady aerodynamics of the wing, it is subtracted from C p,k . Therefore,
only instantaneous differences ∆C p,k are relevant as input to the model. In the final step, minimum and maximum values
occurring in the training set min ∆C train
p and max ∆C train
p are used to normalize all three data sets. In this way, the value
range of the training set is rescaled to [−1, 1]. The value range of the validation and test set may slightly differ from [−1, 1]
after normalization. Nevertheless, normalization of all data using min ∆C train p and max ∆C train
p prevents an information
leak from training to the validation and test phase.
The aerodynamic system is excited by the forced pitching motion. Therefore, the signal of pitching motion, that is
defined in terms of the incidence amplitude ∆α (τ ) around the mean incidence angle α , is provided to the ROM as an
additional input. As preprocessing procedure, a normalization is applied here to rescale the signal range from [−2◦ , 2◦ ]
to [−1, 1].

2.2. Convolution

The neural network developed in this work is based on convolutional operations. The class of such neural networks is
referred to as convolutional neural networks (CNNs). The underlying operation is defined for a one-dimensional function
x(a) as follows (Goodfellow et al., 2016):
∫ +∞
s(t) = x(a)w (t − a)da (3)
−∞

whereas according to the CNN terminology, functions x, w and s are referred to as input, kernel and feature map,
respectively. For functions x and w defined on the set of integers, the discrete convolution is given by:
+∞
∑
st = xt wt −a (4)
a=−∞

Here, all functions can be regarded as sequences of numbers. Hence, index notation is used to refer to their elements.
Due to the commutativity of convolution, its equivalent formulation is (Damelin, 2012):
+∞
∑
st = wa xt −a (5)
a=−∞

In practical applications, the kernel has a more compact support compared to the input (Goodfellow et al., 2016).
Therefore, the latter formulation is more convenient for computations. Within this work, the Python neural network
library PyTorch (Paszke et al., 2019) is employed. Instead of Eq. (5), it implements a related operation:
+∞
∑
st = wa xt +a (6)
a=−∞

Although it corresponds to the formulation of the discrete cross-correlation (Rabiner, 1975), it is nonetheless referred to
as convolution in the context of CNNs.
In practice, the inputs are often two-dimensional arrays of data elements stored in a grid layout. An index tuple (i, j)
can be used to refer to any element of such an array. A two-dimensional convolution for inputs of this type is defined as
follows:
Hw −1 Ww −1
∑ ∑
Si,j = Wm,n Xi+m,j+n (7)
m=0 n=0

with Xi,j , Wi,j denoting elements of two-dimensional input X and kernel W . Here, zero-based indexing is used. Moreover,
the upper bound of the summations depends on the kernel size Hw × Ww .
4
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

Usually, the input is not a two-dimensional grid of values, but rather a grid of vector-valued observations (Goodfellow
et al., 2016). Consequently, apart from two spatial dimensions, an additional dimension occurs. It is referred to as the
channel dimension. Typically, CNNs operate using multi-channel convolutions, which are defined as follows:
Cin −1 Hw −1 Ww −1
∑ ∑ ∑
Sk,i,j = Wk,l,m,n Xl,i+m,j+n (8)
l=0 m=0 n=0

The four-dimensional kernel W associated with the multi-channel convolution has the size Cout × Cin × Hw × Ww . The
size of input X is Cin × Hx × Wx . Here, Cin is the number of input channels. Furthermore, Cout is the number of feature
maps arranged across the channel dimension in output S. With offsets m and n in two spatial directions between input
and feature map elements, Wk,l,m,n specifies the connection strength between an input element in channel l and a feature
map element in channel k (Goodfellow et al., 2016).
In addition, a more general formulation of the multi-channel convolution is introduced, that is used throughout this
work:
Cin −1 Hw −1 Ww −1
∑ ∑ ∑
Sk,i,j = Wk,l,m,n Xl,i×s+m,j×s+n + bk (9)
l=0 m=0 n=0

The operation samples every s elements in each spatial direction with s referred to as stride. The output of a convolution
with s > 1 can be considered as a downsampled output of a full convolution with s = 1. After the application of the
operation, the output has a reduced size of spatial dimensions compared to the input. Furthermore, an additional bias
term bk occurs in Eq. (9) compared to Eq. (8). In the context of CNNs, elements of kernels Wk,l,m,n and biases bk associated
with the neural network are free parameters of the model, that are optimized during the training procedure.
If a full convolution is performed without the kernel exceeding the input limits (i.e. full kernel support at all times),
the size of the spatial output dimensions decreases by Hw − 1 and Ww − 1, respectively. This is a limiting factor for deep
neural network architectures where many convolutional operations are performed consecutively. For this reason, zero
order extrapolation is used prior to the convolution to extend the input by several elements in both spatial dimensions.
This procedure, referred to as padding in the context of CNNs, ensures that the sizes of the spatial dimensions of input
and output are equal.

2.3. Model architecture

The aim of the model is to predict the pressure distribution at time step k + 1 based on several consecutive snapshots
of the pressure distribution at time steps k − n + 1 to k. Here, n is the number of time steps used for the prediction.
Hence, as model input, arrays ∆C p,k −n+1 to ∆C p,k are combined into a three-dimensional array X (k) p along its channel
dimension. The size of this input array is n × 128 × 32. Furthermore, information regarding the pitching motion of the
wing is used as input. Here, incidence amplitudes at time steps k − n + 2 to k + 1 are combined into an additional
input array X (k) (k)
α . To keep the dimension assignment of both input arrays consistent, X α is also three-dimensional. The
scalar values [∆αk−n+2 , . . . , ∆αk+1 ] are aligned along the channel dimension. The remaining two axes of the array are
(k)
dummy dimensions. Consequently, the effective size of X (k) α is n × 1 × 1. The output of the model is array Ỹ , that is an
(k)
approximation of target Y = ∆C p,k +1 :
(k)
p , Xα , θ
= f X (k) (k)
( )
Ỹ (10)

with the nonlinear mapping function f defined by the model. Free parameters of the model are denoted by θ .
The model architecture is illustrated in Fig. 2. Similar to U-Net (Ronneberger et al., 2015), the CNN consists of a
contracting path (green) and an expansive path (orange). The input array X (k) p is fed to the contracting path. Each path
is divided into several levels. At each level a number of operations is performed on incoming arrays. They can be broken
down into a repeated application of a sequence of operations referred to as a convolutional block. As the first step in such
a sequence, a multi-channel convolution (Conv) is performed. The corresponding spatial kernel size is 3 × 3. A padding
of one element is chosen to preserve the size of the spatial array dimensions. Following the convolution, a normalization
technique referred to as layer normalization (LN) (Ba et al., 2016) is carried out to accelerate the training phase. As the
last step of the sequence, a non-linearity is imposed. Here, the rectified linear unit (ReLU) (Glorot et al., 2011) is applied
to each element of the incoming array after layer normalization. The rectified linear unit is defined as:

f (x) = max(0,x) (11)

The convolutional block is applied twice at each level of the contracting path. Output of the first convolution has twice
as many channels as input. An exception is the first convolution applied directly to X (k)
p , that increases the number of
channels from n to 64. The second convolution at each level keeps the number of input and output channels constant. A
downsampling step is applied between the levels. It consists of a convolution with stride s = 2 without padding, which
reduces the size of the spatial dimensions by a factor of two. The corresponding spatial kernel size is 2 × 2.
5
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

Fig. 2. Architecture of the deep learning model. (For interpretation of the references to color in this figure legend, the reader is referred to the
web version of this article.)

At the last level of the contracting path, features carrying information regarding the pitching motion and pressure
distributions are combined. Prior to the combination, a sequence of operations is applied to input X (k)α to align the size of
the feature map arrays originating from two different input sources. First, X (k)
α is passed through a convolutional block.
Due to dummy spatial dimensions of the input array, the spatial kernel size of the corresponding convolution is also
1 × 1. This convolutional operation is equivalent to a dense layer, which is used to obtain the required size of the channel
dimension. Subsequently, for each channel, the scalar-valued feature map is replicated for the spatial dimensions to obtain
a feature map with a correct spatial size. This procedure is referred to as spatial tiling in Levine et al. (2017). Eventually,
both feature map arrays from two different input sources are of the same size and can be combined by summation. The
resulting array is fed to the expansive path, that consists of several levels, with two folding blocks each. A transposed
convolution is applied between the levels to upscale the spatial dimensions of the processed arrays. At each level, prior
to the application of the convolutional blocks, the upscaled output is concatenated with the output of the same level of
the contracting path. The first convolutional block reduces the number of channels by factor two, while the second block
retains the number of channels. Spatial kernel sizes, values of stride and padding are identical for all convolutions at the
same level of the contracting and expanding paths.
To obtain the final prediction, the output of the uppermost level is passed through a convolution with kernel size of
1 × 1 followed by a hyperbolic tangent activation (Tanh). Hence, each 64-component feature vector is used to predict
(k)
an element of Ỹ . The hyperbolic tangent as the final nonlinear activation layer is chosen to ensure all elements of the
(k)
predicted array Ỹ to be in the range [−1, 1].
The model performs one time step ahead prediction based on data of previous n time steps. However, a more realistic
application scenario of the model is the prediction of multiple time steps ahead. Here, instead of using input samples
that exclusively originate from CFD simulations to predict the current time step, previous predictions are fed back into
the model as inputs to advance in time. This operating mode of the model, referred to as multi-step prediction, yields
an output sequence representing the predicted dynamics of the pressure distribution over the wing for a given signal of
pitching motion.
The number of involved time steps per sample n and the depth of the model defined by the number of levels are
hyperparameters. They are adjusted by evaluating the model’s performance on the validation data set.
6
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

2.4. Training procedure

The aim of the training procedure is to optimize the free parameters of the model θ so that a loss L measuring the
mean prediction performance of the model averaged over the training set is minimized:
θ opt = arg max L (12)
θ

where the loss L is defined as:

N
1 ∑ ( (k) (k) )
L= l Y , Ỹ (13)
N
k=1

with N denoting the number of training samples. For each sample, the loss function l evaluates the deviation between
the model prediction and target. Within this work, the mean squared error (MSE) ε̄ is used as loss:
N N
1 ∑ 1 ∑( (k) 2
)
L = ε̄ = ε (k) = Y (k) − Ỹ (14)
N N
k=1 k=1

Hence, the loss function is the per-sample squared error:

(k) 2
( )
ε (k) = Y (k) − Ỹ (15)

The free parameters θ are optimized by a gradient-based algorithm. The gradients of the MSE with respect to θ are
computed with the backpropagation algorithm, which is briefly outlined here. If a stack of layers within a neural network
is regarded, a distinct layer i can be described by the following relation between layer input X i−1 and layer output X i :

X i = f i (X i−1 , θ i ) (16)
where θ i denotes a subset of the neural network’s free parameters associated with the considered layer. The input X i−1
is simultaneously the output of the previous layer i − 1 in the stack. The gradients of the loss function with respect to θ i
are computed according to the chain rule of calculus (Montavon, 2012):
∂L ∂ L ∂ f ⏐⏐
⏐
= (17)
∂θ i ∂ X i ∂θ ⏐X =X i , θ=θi
where the last term is the Jacobian of f with respect to θ evaluated at (X i , θ i ). The gradients can be backpropagated
through the layer by computing:
∂L ∂ L ∂ f ⏐⏐
⏐
= (18)
∂ X i−1 ∂ X i ∂ X ⏐X =X i , θ=θi
Gradients of the loss function with respect to all free parameters θ can be computed by applying Eqs. (17) and (18)
recursively from the last to the first layer of the stack. The library PyTorch provides automatic differentiation functionality
based on backpropagation of gradients. The model parameters θ are iteratively adjusted by the first-order gradient-based
optimization algorithm Adam (Kingma and Ba, 2014). Due to its robustness and versatility, Adam is a well-established
extension of the stochastic gradient descent algorithm (SGD). For the family of SGD methods, instead of computing the
loss gradient based on the entire batch of training samples, the gradient is stochastically estimated based on mini-batches.
Hence, at each iteration of the optimization algorithm, the gradient of mini-batch loss LN ∗ is used to update the model
parameters. The mini-batch loss is defined as:
N∗
1 ∑ ( (k) (k) )
LN ∗ = l Y , Ỹ (19)
N∗
k=1

with N ∗ representing randomly selected samples at each iteration. Adam extends the basic SGD method by using the
moving average of the gradient. In addition, based on moving average of the squared gradient, the global learning rate
is adapted for each parameter. Within this work, the exponential decay rates β1 , β2 and parameter ϵ , that are hyper-
parameters of the algorithm, are set to β1 = 0.9, β1 = 0.999 and ϵ = 10−8 as recommended by the authors. Moreover,
the mini-batch size of N ∗ = 4 is used. The initial global learning rate is set to 10−4 . Further, it is reduced by one order of
magnitude every 70 epochs.
During the training, the entire training set is repeatedly presented to the model in mini-batches, whereas a single
pass through the training set is referred to as an epoch. For each mini-batch, an iteration of the optimization algorithm is
performed. First, the training is carried out for 100 epochs. It then continues until it is terminated according to the early
stopping algorithm (Goodfellow et al., 2016). Here, in addition to the loss evaluated for the training set, the loss with
respect to the validation set is computed after each epoch. As soon as the validation loss has not improved for 30 epochs,
the training is terminated and the model with the lowest validation loss is retained for further application.
7
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

Fig. 3. Convergence trends of training and validation losses. (For interpretation of the references to color in this figure legend, the reader is referred
to the web version of this article.)

As can be seen according to Eq. (15), the accuracy of one time step ahead prediction is evaluated for the model training.
However, the ROM performance is evaluated on the basis of both operating modes, i.e., one-step and multi-step prediction.
Convergence trends during the training are shown in Fig. 3. The blue line represents the change of the training loss
over the number of epochs ne . In addition, losses on the validation set in both one-step and multi-step operating modes
of the model are also considered (green and orange lines). The training procedure is terminated after 192 epochs on the
basis of early stopping. The model with the lowest one-step validation loss is taken for further investigations.

3. CFD-based data synthesis

The Reduced-Order Model is trained and evaluated on synthetic data. The data are generated by a high-fidelity
nonlinear model of the wing aerodynamics in transonic flow. In this section, the underlying high-fidelity computational
modeling is introduced. Furthermore, the details related to the use case, that is employed to demonstrate the ROM
performance, are presented.

3.1. CFD solver

The data synthesis is based on the solution of the Euler equations. This system of nonlinear partial differential equations
describes the conservation of mass, momentum, and energy of an inviscid flow. The conservative formulation of the Euler
equations is:
∂Q ∂F ∂G ∂H
+ + + =0 (20)
∂τ ∂ξ ∂η ∂ζ
where Q is the vector of conservative flow variables:

Q = J [ ρ, ρ u, ρv, ρw, e ]T (21)

with τ , ρ , u, v , w , e, and J denoting time, density, velocity components in x, y, z directions, total energy, and volume of
a cell, respectively. F , G, and H are convective fluxes in the ξ , η, ζ directions. The equations are solved employing the
structured Finite-Volume (FV) solver AER-Eu (Kreiselmaier and Laschka, 2000; Pechloff and Laschka, 2006). Numerical
fluxes are computed by Roe’s flux difference splitting (Roe, 1981). The total-variation-diminishing (TVD) property is
ensured by the reconstruction of conservative variables at FV cell faces with the MUSCL scheme (LeVeque, 2007). Due
to the reconstruction, a spatial accuracy of second order is achieved. Dual-time stepping is employed for the integration
in time. Time marching in physical time is discretized in terms of the dimensionless time step ∆τ , that is defined as:
U∞
∆τ = ∆t (22)
lref
with free-stream velocity U∞ , reference length lref and respective increment of dimensional physical time ∆t. Iterations
in pseudo time are performed using the lower–upper-symmetric successive overrelaxation (LU-SSOR) scheme (Blazek,
1994).
8
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

Table 1
Reference geometrical values of the LANN wing (AGARD, 1985).
Reference planform area Aref 0.2526 m2
Model span b 1 m
Root chord croot 0.36 m
Aspect ratio AR 7.92
Leading-edge sweep angle φLE 27.493◦
Quarter chord sweep angle φ1/4 25◦
Trailing-edge sweep angle φTE 16.908◦
Twist angle ϵ −4.8◦
Taper ratio λ 0.4

Fig. 4. Computational domain and CFD surface mesh of the LANN wing. Grid surfaces with imposed wall, symmetry and far-field boundary condition
are depicted in orange, blue and green respectively. (For interpretation of the references to color in this figure legend, the reader is referred to the
web version of this article.)

3.2. Computational aerodynamic model

As use case the LANN wing is selected, which represents a semi-span wind tunnel model of a transport-type wing
with a supercritical airfoil and moderate aspect ratio (AGARD, 1985). The model has been developed during a co-operative
programme of Lockheed-Georgia, Air Force Flight Dynamics Laboratory, NASA-Langley and NLR, through which it received
its name LANN. The wing is a well-established test configuration in the field of unsteady aerodynamics. The most relevant
reference values of the LANN wing geometry are listed in Table 1.
The computational domain around the model is discretized in a structured manner with 368640 FV-cells in a two-
block C-H topology. It is shown in Fig. 4a. The wing surface is resolved with 37 and 73 cells in span-wise and chord-wise
directions, respectively. The CFD surface mesh of the LANN wing as well as of the symmetry plane is depicted in Fig. 4b.
A grid sensitivity study was carried out in Kreiselmaier (1998) to ensure an independence of the computational solution
from the grid resolution. Furthermore, a sensitivity study with respect to the physical time step ∆τ is conducted. It yields
∆τ = 0.3 as sufficient so that simulation results do not change with a further reduction of the time step size. The root
chord is used as the reference length to nondimensionalize the time step according to Eq. (22). For each physical time
step, iterations in pseudo time are performed until the Euclidean norm of the normalized density change over an iteration
falls below 10−5 . The density change after the first iteration is used as the normalization factor.
CFD data are generated for a free-stream Mach number of Ma∞ = 0.82 and mean incidence angle of α = 0.6◦ .
The mean cp -distribution over the wing associated with the respective flow conditions is shown in Fig. 5. The pressure
coefficient falls below its critical value of cp∗ = −1.17. As it can be seen, this case is transonic and is characterized by a
strong λ-shaped shock system on the suction side of the wing.
Three data sets are generated on the basis of CFD simulations. For the first two data sets, the aerodynamic system is
excited by enforcing pitching motions, where the incidence amplitude ∆α around α is prescribed by amplitude-modulated
pseudo-random binary signals (APRBS) (Nelles, 2001). For this purpose, 25 signal sequences with 213 time steps each are
generated. According to the APRBS algorithm, plateaus with a predefined minimum hold time are modulated on the basis
9
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

Fig. 5. Mean cp -distribution of the LANN wing associated with the free-stream condition Ma∞ = 0.82, α = 0.6◦ . (For interpretation of the references
to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 6. Exemplary smoothed APRBS signal prescribing the pitching motion of the wing for the generation of the training and validation set.

of pseudo-random binary sequences. Subsequently, random amplitudes are assigned to each plateau. Here, the minimum
hold time is set to 2∆τ . The incidence amplitude varies in the range of −2◦ < ∆α < 2◦ around α . In addition, Gauss
filtering is applied to smooth discontinuities as described in Winter and Breitsamter (2018). One of the employed APRBS
signals is depicted in Fig. 6. Smoothed APRBS cover a wide range of amplitudes as well as corresponding time derivatives
and proved to be well-suited for tasks of nonlinear system identification (Winter and Breitsamter, 2016a, 2018). Data
yielded by the simulations are split into the training and validation set. The training set consists of data associated with
23 APRBS signals. It is used to condition free parameters of the model during the training. Remaining data are used as
the validation set. Evaluation of the model on the validation set is employed to monitor the training process. It is also
used to tune hyperparameters of the model. The third data set is generated by the excitation of the aerodynamic system
with harmonic pitching motions at various reduced frequencies. This data set, referred to as test set, is neither exposed
to the model during the training nor for the purpose of hyperparameter tuning. Hence, model evaluation on the test set
represents the application scenario. In this way, the ability of the model to generalize to unseen data is assessed.
10
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

4. Application of the Reduced Order Model

In this section, the model is applied to predict snapshots of unsteady pressure fluctuations of the LANN wing that
undergoes harmonic pitching oscillations. First, the dynamics of the unsteady cp -distribution due to harmonic pitching
motion of the wing is discussed. Second, the prediction accuracy of the Reduced-Order Model conditioned on the training
set is investigated. Subsequently, the gain in computational efficiency through the use of the presented ROM for the
prediction of unsteady cp -distributions compared to CFD is quantified.

4.1. Test set generation

The model yielded after the training procedure is evaluated on the test set. For the generation of the first part of
the test set, the aerodynamic system is excited by harmonic pitching motions at different frequencies. Harmonic signals
differ significantly from the APRBS employed for training and hyperparameter tuning of the model. Therefore, they are
well-suited to assess the ability of the ROM to generalize to unseen data. The corresponding harmonic signals are defined
as:
∆α (τ ) = A sin(kred τ ) (23)
with the maximum incidence amplitude A and reduced frequency kred :
ωcroot 2π croot
kred = = (24)
U∞ TU∞
where ω and T denote dimensional angular frequency and period of the harmonic oscillation, respectively. Three reduced
frequencies are taken into account kred = [0.102, 0.204, 1.5]. The two lower reduced frequencies have been considered
in the context of POD-based reduced-order modeling applied to the prediction of unsteady cp -distribution over the LANN
wing (Winter and Breitsamter, 2016a). In addition, the reduced frequency of kred = 1.5 is taken into account in order
to investigate the performance of the model for a high-frequency excitation. Moreover, harmonic pitching motions with
both a small and a large incidence amplitude A = [0.25◦ , 2◦ ] are investigated. Unsteady CFD simulations are carried out
over six oscillation periods for all reduced frequencies.
Figs. 7a–7f illustrate the dynamics of the cp -distribution over the fifth oscillation period for A = 2◦ and all reduced
frequencies under consideration. Two slices in chord-wise direction at y/b = 0.22 and y/b = 0.7 are taken into
account. For each slice, the chord-wise position is normalized by the local chord length cloc . The phase angle of the
fifth oscillation period φ = kred (τ − 4T ) is encoded by color, with green, yellow and red indicating φ = [0, π, 2π ],
respectively. As can be seen, the dynamics of the cp -distribution varies only slightly for both lower reduced frequencies.
However, significant differences in the unsteady behavior of the cp -distribution are observed when the two lower reduced
frequencies are compared with the higher reduced frequency of kred = 1.5. It highlights the dependence of the unsteady
wing aerodynamics on the frequency of the pitching motion. Nonetheless, unsteady pressure fluctuations share common
characteristics across different excitation frequencies, which are further briefly discussed.
Due to the λ-shaped shock structure on the upper surface of the wing, two shocks can be observed at y/b = 0.22,
while one shock wave is present at y/b = 0.7. With negative incidence amplitudes A in the second half of the period,
a shock front forms on the pressure side of the wing. It is stronger at y/b = 0.22 than further outboard. Moreover, at
y/b = 0.7 the shock front is more pronounced for lower reduced frequencies. All shock waves move fore and aft as the
angle of incidence changes. In addition, their intensity varies over the period of oscillation.
To obtain the second part of the test set, the aerodynamic system is excited by the pitching motion defined by two
APRBS signals with 211 time steps each. The hold time for the APRBS signals associated with the test set is 5∆τ as opposed
to 2∆τ used to construct the APRBS signals employed in the training set generation. The test APRBS signals are referred
to as APRBS-T1 and APRBS-T2.

4.2. Investigation of prediction accuracy

To start the iteration of recurrent time-step predictions in the multi-step mode, three snapshots of the cp -distribution
originating from a priori conducted CFD runs are provided for initialization. These snapshots are associated with the
initial three time steps of the signal specifying the amplitudes of the incidence angle. As the prediction steps advance,
the snapshots from CFD are successively substituted by snapshots predicted by the model itself. With the exception of
the warm-up phase outlined above, no CFD data are used throughout the iterations in the multi-step prediction mode.
The performance of the model is investigated in terms of the mean squared error ε̄ for one-step prediction mode as
well as multi-step prediction mode. Table 2 provides an overview of ε̄ evaluated on different data sets. All available data
are taken into account, whereas the MSE is computed separately for each data set. Additionally, test set predictions for
harmonic pitching signals of different maximum amplitudes and APRBS are evaluated separately.
One-step and multi-step prediction modes yield MSE-values in the range of 10−7 and 10−6 , respectively for all data sets
under consideration. The highest values of the MSE are obtained for the harmonic pitching motion with A = 2◦ . A slight
overfitting is observed for training and validation sets as the respective MSE-values are lower than the values associated
11
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

Fig. 7. Temporal development of the cp -distribution for two slices at y/b = 0.22 and y/b = 0.7 due to harmonic pitching motions at
kred = [0.102, 0.204, 1.5] at Ma∞ = 0.82 and α = 0.6◦ . (For interpretation of the references to color in this figure legend, the reader is referred to
the web version of this article.)

with the test set. This is due to the fact that the training set is used to optimize free parameters of the model and, therefore,
is exposed to the model during training, on the one hand. On the other hand, also the validation set is implicitly exposed
to the model as it is employed for hyperparameter tuning and early stopping. Furthermore, it is interesting to note, that
model predictions for signals with smaller maximum incidence amplitude are more accurate. This is indicated by a low
MSE for test data set associated with the harmonic pitching motion with A = 0.25◦ . The MSE values for APRBS signals
are ranged between the corresponding values attributed to the harmonic signals with A = 0.25◦ and A = 2◦ .
Further, the per-sample mean squared error ε (k) of the model predictions for the test set is evaluated at each time step
k. The trends are illustrated in Figs. 8 and 9. In the case of harmonic signals the oscillation period depends on kred . Hence, a
different number of computed time steps is obtained for each reduced frequency. Predictions are available starting from
the fourth time step of each signal. The considered signals are depicted by blue lines in the bottom subplots. The top
subplots show the corresponding ε (k) -trends. Both operating modes of the model are investigated. The green dash-dotted
line represents ε (k) for one-step ahead predictions. The per-sample MSE of the predicted cp -distributions is in the range
of 10−7 and 10−6 for A = 0.25◦ and A = 2◦ , respectively. Individual peaks can be observed for time steps with maximum
angle of incidence. The phases of the oscillation periods are characterized by a high intensity of the shock front at the aft
part of the wing (see φ = π/2 in Fig. 7). The range of the per-sample MSE for APRBS signals is in the same range as for
the harmonic signals with A = 2◦ .
12
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

Table 2
Overview of the MSE of model predictions evaluated on different data sets. Training set,
validation set and test set are taken into account. Additionally, test set predictions for
harmonic pitching signals of different maximum amplitudes and APRBS are evaluated
separately.
ε̄, One-step ε̄, Multi-step
Training set 3.62 · 10−7 2.21 · 10−6
Validation set 4.94 · 10−7 3.03 · 10−6
Test set, A = 2◦ 4.99 · 10−7 4.53 · 10−6
Test set, A = 0.25◦ 1.34 · 10−7 1.95 · 10−6
Test set, APRBS 3.56 · 10−7 2.68 · 10−6

For the multi-step prediction mode, a periodic behavior of ε (k) over time is observed for all harmonic signals under
consideration. However, the error of the multi-step predictions does not necessarily follow the trend of the error in
the one-step prediction mode. The multi-step operating mode provides very accurate predictions. The highest ε (k) -value
observed across all predicted sequences of cp -distributions does not exceed 3 · 10−5 . It should be noted that the prediction
accuracy does not deteriorate even for long sequences as in the case of kred = 0.102 and kred = 0.204 with 1229 and 613
predicted time steps, respectively.
The local accuracy of the predicted cp -distributions in multi-step operating mode is investigated. For this purpose, the
snapshot of the cp -distribution with maximum ε (k) in the test set is compared with the corresponding snapshot of the
cp -distribution obtained with CFD. The maximum per-sample mean squared error is observed for the signal of pitching
motion with A = 2◦ and kred = 1.5 and associated with the time step τ = 70 (φ ≈ 0). In Figs. 10 and 11, the cp -distribution
on the upper and lower surfaces of the wing are compared. As can be seen, an excellent agreement of the model prediction
and the CFD result is achieved for both surfaces of the LANN wing even for this snapshot associated with the maximum
MSE among all prediction sequences under consideration.
In addition, two slices of the cp -distribution in chord-wise direction at y/b = 0.22 and y/b = 0.7 for the investigated
time step are considered. Here, cp -distributions yielded by the model both in one-step and multi-step prediction modes
are compared to the CFD result. Both predictions are almost indiscernible from the cp -distribution obtained by the CFD
simulation for both slices (see Fig. 12).
The distribution of the squared error for the snapshot under investigation predicted in multi-step operating mode is
presented in Fig. 13. First, the upper surface of the wing is considered. As can be seen, an excellent agreement of the model
prediction with the CFD result can be concluded, since the squared error is almost zero at most regions of the surface.
Solely, a confined area with marginal deviations is observed, that is associated with the shock front at the outboard half
of the wing. The maximum squared error does not exceed 4 · 10−3 for the upper side of the wing and is negligible for
practical applications. An excellent agreement is also found for the lower surface of the wing. Similar to the upper surface,
a confined area of small deviations is present, which can be attributed to the shock front at the inboard part of the wing.
However, the maximum squared error is even one order of magnitude lower than for the upper surface with 4 · 10−4 .

4.3. Investigation of the model stability for the multi-step operating mode

For the multi-step operating mode, the stability behavior of the model is investigated. For this purpose, the model
prediction is started at several points in time using two different types of initialization for the warm-up phase. The
first type is the same approach used in Section 4.2. Here, the process of the multi-step prediction is initialized based
on the corresponding three snapshots of the cp -distribution obtained from CFD. In contrast, the second type of warm-
up initialization uses three matrices filled with random numbers from a uniform distribution on the interval [−0.1, 0.1)
instead of three cp -snapshots. This type of initialization is referred to as random initialization. It is used to assess the ability
of the ROM to recover accurate predictions in the multi-step operating mode after a high prediction error is introduced.
The investigation results are presented exemplary for the pitching signal APRBS-T1. Multi-step prediction processes
are started at four different points in time, with a warm-up initialization at time steps 25l to 25l + 2 with l = [0, 1, 2, 3].
Each run is performed for 100 steps. Fig. 14 shows trends of per-sample MSE ε (k) over time steps for both initialization
types in semi-logarithmic representation. As expected, in the case of random initialization a high per-sample MSE in
the range of 10−2 is introduced at the beginning of the multi-step predictions. However, the randomly initialized multi-
step prediction processes do not become unstable. On the contrary, over several prediction steps, the ε (k) -trends of the
randomly initialized prediction runs gradually converge towards the ε (k) -trends corresponding to prediction runs which
are initialized based on CFD. After a maximum of 20–30 steps, the full prediction accuracy of the model is recovered. This
error recovery mechanism of the multi-step operating mode is observed for all signals considered in this work.

4.4. Computational efficiency

The gain in computational efficiency through the use of the presented ROM for the prediction of unsteady cp -
distributions compared to the CFD simulation is quantified. To ensure an accurate comparison, both the CFD simulation
13
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

Fig. 8. Trends of the per-sample mean squared error ε (k) of the predicted cp -distributions for each time step τk . Harmonic pitching signals with
kred = [0.102, 0.204, 1.5] and A = [0.25◦ , 2◦ ].

Fig. 9. Trends of the per-sample mean squared error ε (k) of the predicted cp -distributions for each time step τk . Pitching signals APRBS-T1 and
APRBS-T2.

14
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

Fig. 10. Comparison of the cp -distribution on the upper surface of the wing obtained with CFD on the one hand and predicted by the ROM in
multi-step operating mode on the other hand. A = 2◦ , kred = 1.5, τ = 70, Ma∞ = 0.82. (For interpretation of the references to color in this figure
legend, the reader is referred to the web version of this article.)

Fig. 11. Comparison of the cp -distribution on the lower surface of the wing obtained with CFD on the one hand and predicted by the ROM in
multi-step operating mode on the other hand. A = 2◦ , kred = 1.5, τ = 70, Ma∞ = 0.82. (For interpretation of the references to color in this figure
legend, the reader is referred to the web version of this article.)

15
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

Fig. 12. Comparison of the cp -distribution obtained with CFD on the one hand and predicted by the ROM in one-step as well as multi-step operating
modes on the other hand. Two slices at y/b = 0.22 and y/b = 0.7 are considered. A = 2◦ , kred = 1.5, τ = 70, Ma∞ = 0.82.

Fig. 13. Distribution of the squared error for the predicted snapshot of the pressure distribution in multi-step operating mode of the ROM. A = 2◦ ,
kred = 1.5, τ = 70, Ma∞ = 0.82. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this
article.)

and the model inference are performed on a single AMD EPYC 7401P processor. The comparison is performed in terms
of the CPU time required to compute a time step with the CFD solver and with the ROM in multi-step prediction mode.
In average, it takes 168 s with CFD to compute one time step, whereas the prediction of a single time step with the ROM
takes merely 0.136 s. Thus, the use of the Reduced-Order Model presented within this work provides a speed-up of more
than 1200 compared to the full-order CFD simulation. The training of the model requires approximately 60 s for an epoch
on a single GeForce RTX 2080 GPU.

4.5. Performance comparison with a POD-based ROM

The CNN presented in this paper is compared with a state-of-the-art model based on POD. The ROM drawn for
comparison employs a neural network based on the recurrent local linear neuro-fuzzy approach in combination with
POD (Winter and Breitsamter, 2016a). The model is referred to here as POD-NN. The training of POD-NN is conducted on
the basis of the training set as well as validation set. The optimal number of previous time steps to perform a prediction of
16
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

Fig. 14. Trends of the per-sample mean squared error ε (k) of the predicted cp -distributions for each time step τk for both CFD-based as well as
random initialization of the multi-step ahead prediction.

Table 3
Comparison of the MSE for the POD-NN-based and CNN-based model evaluated on the
test set in the multi-step prediction mode. Predictions for harmonic pitching signals of
different maximum amplitudes and APRBS are evaluated separately.
ε̄, POD-NN ε̄, CNN
Test set, A = 2◦ 5.65 · 10−4 4.53 · 10−6
Test set, A = 0.25◦ 3.82 · 10−4 1.95 · 10−6
Test set, APRBS 3.92 · 10−4 2.68 · 10−6

the subsequent time step is set to n = 5. It is derived for the model from the available data using the procedure outlined
in He and Asada (1993) and Winter and Breitsamter (2016a).
The prediction accuracy of both models is juxtaposed with respect to the mean squared error ε̄ averaged over all
predicted cp -snapshots. Both models are evaluated on the test set in multi-step operating mode. Table 3 provides an
overview of ε̄ , whereas the MSE is computed separately for harmonic pitching signals of different maximum amplitudes
and APRBS. While both ROM approaches provide a speed-up factor of three orders of magnitude compared to CFD,
CNN-based ROM is two orders of magnitude more accurate than POD-NN-based ROM in terms of MSE across all signals
considered in the test set.
To visually compare the prediction accuracy, cp -distribution obtained with CFD, POD-NN- and CNN-based ROMs is
illustrated in Fig. 15 exemplary for the chord-wise slice at y/b = 0.7. Two phase angles φ = [π/2, 3π/2] of the fifth
oscillation period of the harmonic pitching signals with kred = [0.102, 0.204, 1.5] and A = 2◦ are taken into account.
Predictions are performed in multi-step operating mode for both ROM approaches. For POD-NN, offset errors are apparent
at φ = π /2, while ringing artifacts are present at φ = 3π/2. In contrast, perfect agreement with CFD is observed for
CNN-based ROM across all time steps.

5. Conclusion and outlook

Within this work, a novel Reduced-Order Model based on deep learning is introduced to predict unsteady pressure
distributions over a wing for a challenging transonic case. The well-established LANN wing configuration is employed
to demonstrate the performance of the Reduced-Order Model. Synthetic training data are generated by a high-fidelity
nonlinear model of the wing aerodynamics in transonic flow. The underlying computational modeling represented by a
CFD-method is introduced. Furthermore, an overview of the system identification technique based on a deep convolutional
17
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

Fig. 15. Comparison of the cp -distribution obtained with CFD, POD-NN- and CNN-based ROMs in multi-step operating mode for the chord-wise slice
at y/b = 0.7. Two phase angles φ = [π/2, 3π/2] of the fifth oscillation period of harmonic pitching signals with kred = [0.102, 0.204, 1.5] and
A = 2◦ .

neural network is given. Specific modules of the network are discussed and relevant information related to the training
algorithm is provided. The Reduced-Order Model is applied to predict snapshots of the unsteady pressure distribution of
the LANN wing undergoing harmonic pitching oscillations in transonic flow. It is shown, that the Reduced-Order Model
yields an unprecedented prediction accuracy for all considered test cases at a speed-up of more than three orders of
magnitude compared to CFD.
For future work, the model can be applied to the prediction of pressure distributions for unsteady aerodynamic
phenomena such as buffet and buzz. Models based on deep convolutional neural networks are very successfully used in
image processing. Therefore, the model architecture presented in this work is well-suited to be trained on aerodynamic
data acquired with optical experimental techniques such as pressure-sensitive paint and particle image velocimetry.
In addition, the architecture of the Reduced-Order Model can be extended by convolutional recurrent layers such
as convolutional long short-term memory or convolutional gated recurrent unit to account for possible long-term
dependencies in data.

CRediT authorship contribution statement

Vladyslav Rozov: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Data curation,
Writing - original draft, Visualization. Christian Breitsamter: Conceptualization, Resources, Supervision.
18
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have
appeared to influence the work reported in this paper.

Acknowledgments

The authors gratefully acknowledge the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) for
funding this work in the framework of the research unit FOR 2895 (Unsteady flow and interaction phenomena at
high speed stall conditions), subproject TP7, grant number BR1511/14-1. The Gauss Centre for Supercomputing e.V.
(www.gauss-centre.eu) is gratefully acknowledged for funding this project by providing computing time on the Linux-
Cluster at Leibniz Supercomputing Centre (www.lrz.de). The authors would like to thank Sergey Litvinov, Ludger Paehler
and Maximilian Winter for fruitful and valuable discussions. In addition, Maximilian Winter is gratefully acknowledged
for training and evaluating POD-NN to provide data for model comparison.

References

AGARD, 1985. Compendium of Unsteady Aerodynamic Measurements: Addendum No. 1. North Atlantic Treaty Organization, Advisory Group for
Aerospace Research & Development, Neuilly sur Seine, France.
Albano, E., Rodden, W.P., 1969. A doublet-lattice method for calculating lift distributions on oscillating surfaces in subsonic flows. AIAA J. 7 (2),
279–285.
Ba, J.L., Kiros, J.R., Hinton, G.E., 2016. Layer normalization. arXiv preprint arXiv:1607.06450.
Blazek, J., 1994. A Multigrid LU-SSOR scheme for the solution of hypersonic flow problems. In: 32nd Aerospace Sciences Meeting and Exhibit, Reno,
Nevada.
Chen, P.-C., Liu, D.D., 1985. A harmonic gradient method for unsteady supersonic flow calculations. J. Aircr. 22 (5), 371–379. http://dx.doi.org/10.
2514/3.45134.
Damelin, S., 2012. The Mathematics of Signal Processing. Cambridge University Press, Cambridge New York.
Dufour, G., Sicot, F., Puigt, G., Liauzun, C., Dugeai, A., 2010. Contrasting the harmonic balance and linearized methods for oscillating-flap simulations.
AIAA J. 48 (4), 788–797.
Faller, W.E., Schreck, S.J., 1997. Unsteady fluid mechanics applications of neural networks. J. Aircr. 34 (1), 48–55. http://dx.doi.org/10.2514/2.2134.
Finn, C., Goodfellow, I., Levine, S., 2016. Unsupervised learning for physical interaction through video prediction. In: Lee, D.D., Sugiyama, M.,
Luxburg, U.V., Guyon, I., Garnett, R. (Eds.), Advances in Neural Information Processing Systems 29. Curran Associates, Inc., pp. 64–72, URL
http://papers.nips.cc/paper/6161-unsupervised-learning-for-physical-interaction-through-video-prediction.pdf.
Giesing, J., Kalman, T., 1975. Oscillatory supersonic lifting surface theory using a finite element doublet representation. In: 16th Structures, Structural
Dynamics, and Materials Conference, Denver, Colorado, USA. American Institute of Aeronautics and Astronautics, http://dx.doi.org/10.2514/6.1975-
761.
Giesing, J.P., Kalman, T.P., Rodden, W.P., 1971. Subsonic Unsteady Aerodynamics for General Configurations; Part I, Vol. I - Direct Application of the
Nonplanar Doublet-Lattice-Method, Technical Report No. AFFDL-TR-71-5, Part I, Vol. I.
Glaz, B., Liu, L., Friedmann, P.P., 2010. Reduced-order nonlinear unsteady aerodynamic modeling using a surrogate-based recurrence framework. AIAA
J. 48 (10), 2418–2429. http://dx.doi.org/10.2514/1.j050471.
Glorot, X., Bordes, A., Bengio, Y., 2011. Deep sparse rectifier neural networks. In: Proceedings of the Fourteenth International Conference on Artificial
Intelligence and Statistics, pp. 315–323.
Goodfellow, I., Bengio, Y., Courville, A., 2016. Deep Learning. MIT Press.
Hall, K.C., Crawley, E.F., 1989. Calculation of unsteady flows in turbomachinery using the linearized Euler equations. AIAA J. 27 (6), 777–787.
http://dx.doi.org/10.2514/3.10178.
Harder, R.L., Desmarais, R.N., 1972. Interpolation using surface splines.. J. Aircr. 9 (2), 189–191. http://dx.doi.org/10.2514/3.44330.
He, X., Asada, H., 1993. A new method for identifying orders of input-output models for nonlinear dynamic systems. In: 1993 American Control
Conference. IEEE, http://dx.doi.org/10.23919/acc.1993.4793346.
Kingma, D.P., Ba, J., 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Kou, J., Zhang, W., Yin, M., 2016. Novel Wiener models with a time-delayed nonlinear block and their identification. Nonlinear Dynam. 85 (4),
2389–2404. http://dx.doi.org/10.1007/s11071-016-2833-y.
Kreiselmaier, E., 1998. Berechnung Instationärer Tragflügelumströmungen auf der Basis der zeitlinearisierten Eulergleichungen (Dissertation).
Technische Universität München, München.
Kreiselmaier, E., Laschka, B., 2000. Small disturbance Euler equations: Efficient and accurate tool for unsteady load prediction. J. Aircr. 37 (5), 770–778.
LeVeque, R.J., 2007. Finite Volume Methods for Hyperbolic Problems. Cambridge Univ. Press, Cambridge.
Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J., Quillen, D., 2017. Learning hand-eye coordination for robotic grasping with deep learning and large-scale
data collection. Int. J. Robot. Res. 37 (4–5), 421–436. http://dx.doi.org/10.1177/0278364917710318.
Lindhorst, K., Haupt, M.C., Horst, P., 2014. Efficient surrogate modelling of nonlinear aerodynamics in aerostructural coupling schemes. AIAA J. 52
(9), 1952–1966. http://dx.doi.org/10.2514/1.j052725.
Lotter, W., Kreiman, G., Cox, D., 2016. Deep predictive coding networks for video prediction and unsupervised learning. arXiv preprint arXiv:
1605.08104.
Mannarino, A., Mantegazza, P., 2014. Nonlinear aeroelastic reduced order modeling by recurrent neural networks. J. Fluids Struct. 48, 103–121.
http://dx.doi.org/10.1016/j.jfluidstructs.2014.02.016.
Marques, F., Anderson, J., 2001. Identification and prediction of unsteady transonic aerodynamic loads by multi-layer functionals. J. Fluids Struct. 15
(1), 83–106. http://dx.doi.org/10.1006/jfls.2000.0321.
Mathieu, M., Couprie, C., LeCun, Y., 2016. Deep multi-scale video prediction beyond mean square error. In: 4th International Conference on Learning
Representations, ICLR 2016.
Montavon, G., 2012. Neural Networks: Tricks of the Trade. Springer, Berlin New York.
Murua, J., Palacios, R., Graham, J.M.R., 2012. Applications of the unsteady vortex-lattice method in aircraft aeroelasticity and flight dynamics. Prog.
Aerosp. Sci. 55, 46–72. http://dx.doi.org/10.1016/j.paerosci.2012.06.001.

19
V. Rozov and C. Breitsamter Journal of Fluids and Structures 104 (2021) 103316

Nelles, O., 2001. Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuzzy Models. Springer-Verlag Berlin Heidelberg,
Berlin, Heidelberg.
Park, K.H., Jun, S.O., Baek, S.M., Cho, M.H., Yee, K.J., Lee, D.H., 2013. Reduced-order model with an artificial neural network for aerostructural design
optimization. J. Aircr. 50 (4), 1106–1116. http://dx.doi.org/10.2514/1.c032062.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E.,
DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., Chintala, S., 2019. Pytorch: An imperative style, high-performance
deep learning library. In: Advances in Neural Information Processing Systems, Vol. 32. Curran Associates, Inc., pp. 8026–8037.
Pechloff, A., Laschka, B., 2006. Small disturbance Navier-Stokes method: Efficient tool for predicting unsteady air loads. J. Aircr. 43 (1), 17–29.
Pines, S., Dugundji, J., Neuringer, J., 1955. Aerodynamic flutter derivatives for a flexible wing with supersonic and subsonic edges. J. Aeronaut. Sci.
22 (10), 693–700. http://dx.doi.org/10.2514/8.3436.
Rabiner, L., 1975. Theory and Application of Digital Signal Processing. Prentice-Hall, Englewood Cliffs, N.J.
Rodden, W.P., Giesing, J.P., Kalman, T.P., 1972. Refinement of the nonplanar aspects of the subsonic doublet-lattice lifting surface method. J. Aircr. 9
(1), 69–73. http://dx.doi.org/10.2514/3.44322.
Roe, P., 1981. Approximate Riemann solvers, parameter vectors, and difference schemes. J. Comput. Phys. 43 (2), 357–372.
Ronneberger, O., Fischer, P., Brox, T., 2015. U-Net: Convolutional networks for biomedical image segmentation. In: Lecture Notes in Computer Science.
Springer International Publishing, pp. 234–241. http://dx.doi.org/10.1007/978-3-319-24574-4_28.
Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., Woo, W.-c., 2015. Convolutional LSTM network: A machine learning approach for precipitation
nowcasting. Adv. Neural Inf. Process. Syst. 28, 802–810.
Shi, X., Gao, Z., Lausen, L., Wang, H., Yeung, D.-Y., Wong, W.-k., Woo, W.-c., 2017. Deep learning for precipitation nowcasting: A benchmark and a
new model. In: Advances in Neural Information Processing Systems. pp. 5617–5627.
Silva, W.A., 1993. Application of nonlinear systems theory to transonic unsteady aerodynamic responses. J. Aircr. 30 (5), 660–668. http://dx.doi.org/
10.2514/3.46395.
Voitcu, O., Wong, Y.S., 2003. Neural network approach for nonlinear aeroelastic analysis. J. Guid. Control Dyn. 26 (1), 99–105. http://dx.doi.org/10.
2514/2.5019.
Vougioukas, K., Petridis, S., Pantic, M., 2018. End-to-end speech-driven facial animation with temporal GANs. arXiv preprint arXiv:1805.09313.
Wang, Q., Cesnik, C.E., Fidkowski, K., 2020. Multivariate recurrent neural network models for scalar and distribution predictions in unsteady
aerodynamics. In: AIAA Scitech 2020 Forum. American Institute of Aeronautics and Astronautics, http://dx.doi.org/10.2514/6.2020-1533.
Widhalm, M., Thormann, R., 2017. Efficient evaluation of dynamic response data with a linearized frequency domain solver at transonic separated
flow condition. In: 35th AIAA Applied Aerodynamics Conference. Denver, Colorado, United States.
Winter, M., Breitsamter, C., 2014. Reduced-order modeling of unsteady aerodynamic loads using radial basis function neural networks. In: Deutscher
Luft- Und Raumfahrtkongress 2014. Deutsche Gesellschaft für Luft-und Raumfahrt-Lilienthal-Oberth e. V., Bonn.
Winter, M., Breitsamter, C., 2016a. Efficient unsteady aerodynamic loads prediction based on nonlinear system identification and proper orthogonal
decomposition. J. Fluids Struct. 67, 1–21. http://dx.doi.org/10.1016/j.jfluidstructs.2016.08.009.
Winter, M., Breitsamter, C., 2016b. Neurofuzzy-model-based unsteady aerodynamic computations across varying freestream conditions. AIAA J.
2705–2720.
Winter, M., Breitsamter, C., 2018. Nonlinear identification via connected neural networks for unsteady aerodynamic analysis. Aerosp. Sci. Technol.
77, 802–818. http://dx.doi.org/10.1016/j.ast.2018.03.034.
Zhang, W., Kou, J., Wang, Z., 2016. Nonlinear aerodynamic reduced-order model for limit-cycle oscillation and flutter. AIAA J. 54 (10), 3304–3311.
http://dx.doi.org/10.2514/1.j054951.
Zhang, W., Wang, B., Ye, Z., Quan, J., 2012. Efficient method for limit cycle flutter analysis based on nonlinear aerodynamic reduced-order models.
AIAA J. 50 (5), 1019–1028. http://dx.doi.org/10.2514/1.j050581.

20