OpenFOAM Benchmark and Validation PDF

24070000
OpenFOAM Benchmark and Validation
Phase 1
Detection of Butterfly Effect in OpenFOAM
August 1st, 2012
Copyright © ESI Group, 2012.

2009. All rights reserved. 1
Executive Summary
In previous runs done by SL-RASCH with PAM-FLOW (FEM CFD solver) to extract
moments and forces on structures, it was observed that flow development process
and instantaneous results would change even with seemingly innocuous
modifications in the setup of the simulations. Items like number of CPUs and
partitioning had a significant impact onto these instantaneous results.
If there was no change at all in the simulation, the results did not deviate. However,
changing the number of CPUs (and hence partitioning) had a significant impact
onto results. Round-off errors were highlighted as the probable reason for such
behaviour.
This project was performed to check if the same phenomenon appeared in
OpenFOAM (FVM CFD solver).
Copyright © ESI Group, 2012. All rights reserved. 2

Executive Summary (cont’d)
Following steps were carried out for this:

• Good quality mesh was generated in VisCART, keeping feature lines.
Refinement boxes and cell sizes were defined to resemble the PAM-FLOW
model as much as possible.
• Simulations were performed in CentFOAM 2.1, at SL-RASCH cluster in 8, 32
and 64 CPUs:
• LES (Smagorinsky) simulation for 1800 seconds, with time step sizes of 5ms.
Results showed that the same effect was happening in OpenFOAM: instantaneous
results did change significantly between the different cases. Therefore the curves of
forces and moments along time were also different.
Regardless of this, the time-averaged flow field was very similar for all cases.

Executive Summary (cont’d)
In terms of average figures:

• Time-average forces onto umbrella were very similar.
• Time-average moments did vary depending on the number of CPUs even though
the flow fields were very similar.
In order to investigate this deviation, simulation was continued up to 3000s (from
initial 1800s), showing following output:
• RMS values of forces and moments seem to be quite consistent regardless of
the number of CPUs used.
• Time-averaged values of forces are also consistent (or core-independent).
• Time-averaged moments are not consistent regardless of the averaging time.
After investigation of this issue, no reason was identified as possible cause
other natural variability of the CFD simulation. Increasing simulation time or
changing the solver tolerances did not make results more consistent, and the
bugs identified in the code regarding communication between CPUs should
only have a minor impact (hence this was rejected by the code developers).
Model Description:
Domain
Model consisted of a umbrella geometry with

a building (30x30x30m3) upstream of it.
93m

Model Description:
Y Section
Total Mesh size:

910k cells
330m
4m cell size
2m cell
size

Model Description:
Y Section
2m cell
size
0.5m 0.25m
cell size cell size
Model Description:
X Section
4m cell
size
2m cell
size
60m
0.25m
cell size
94m
Model Description:
Inlet Profile
A wind-type velocity profile was applied

at the inlet, with a law of:
v(z) = 21.1 (z/10)^0.26, in m/s

Model Description:
Set-up and Simulation
With the model as described, simulation was initially performed for 1800s (30 min) as
follows:
• LES with Smagorinsky Subgrid Scale model.
• Flow field initialised from a static flow (0 m/s in all directions).
• Simulation was done with 8, 32 and 64 CPUs. Partitioning was similar in all cases:
simple partitioning method (coefficients 4-2-1, 16-2-1 and 32-2-1 in X-Y-Z). Set-up
files were the same for all 3 cases.
• Time step sizes of 5ms (5e-03s), which produced maximum courant numbers of
1.5-2.0 along time.
• Following results were extracted from the simulation:
• Forces and moments on the umbrella were extracted for each time step.
• Velocity and static pressure values on monitoring points in the domain were
extracted for each time step.
• Flow field (p and U) for every 5 seconds.
Model Description:
Set-up and Simulation
Point #3
11 points were monitored:

Point #8
• 3 points 5 meters upstream of the
cube, at +/-20m and 0 in Y, and at Point #4
30m in Z.
• 4 points 5m downstream of the
cube, at +/-5m and +/-15m in Y, and Point #1
at 30m in Z.
• 4 points upstream of umbrella at
60m in X, at +/-5m and +/-15m in Y,
and at 31m in Z.
Results:
Forces on Umbrella
As observed in the cases done with PAM-FLOW, the first 12 seconds of simulation
showed a very similar set of results for the cases with different CPUs.
However, after this time, the calculated instantaneous force progressed along
completely different curves.
(Only the first 30 seconds shown here)

Results:
Moments on Umbrella
Similarly with the moments, the first 10-13 seconds of simulation showed a similar set
of results for the cases with different CPUs. After these, the calculated instantaneous
moments progressed along completely different curves.
(Only the first 30 seconds shown here)

Results:
Distribution at Y=0m
Instantaneous Velocity
For the first 10 seconds of simulation, the flow is similar for all cases, regardless of
the number of CPUs used.
There are some small differences around the umbrella, but the rest of the domain
looks alike.
5 seconds 10 seconds

Results:
Instantaneous Velocity
20 s
15 s
After the first

10 seconds,
the cases
progress
with
significant
differences.
25 s 30 s
Results:
Velocity at points
Upstream of the cube, there are no

major differences between the runs,
since there have been no eddies
generated. Deviations appear
downstream of the leading edge of
the cube.
Looking at the velocity plots at the

points downstream of the cube,
deviations started to appear at
around 7-8 seconds, earlier than
observed at the forces&moments
curves, and also earlier than
observed in the section cuts.

Results:
Velocity at points
Between the cube and the umbrella,

flow behaves similar to the points just
downstream of the cube.
The main difference in these locations
is that whilst for the outer points (+/-
15m in Y) deviations start between
seconds 7-8, and after 10s the curves
are completely different, for the inner
points (+/-5 in Y), curves are similar
until 15 seconds of simulation.

Results:
Average Velocity
Although the instantaneous forces/moments as well as flow pictures are different, the
average flow fields are very similar.
Last ~17min
of averaging
Last ~18min
of averaging
Last ~10min
of averaging

Results:
Average Velocity
Looking deeper into the flow around, there are some slight differences underneath the
umbrella membranes. Regardless of this, the flow structure is extremely similar.
Differences could be due to different timings when starting the averaging.
Last ~17min
of averaging
Last ~18min
of averaging
Last ~10min (Isolines

generated every
of averaging 2.5m/s)

Results:
Average Static Pressure
Similar behaviour is observed from the average static pressure
(Isolines generated
every 50Pa)

Results:
Contours of Averaged Pressure Coefficients
Distribution of pressure coefficients on the umbrella are very similar regardless of the
number of CPUs used in the calculation (view from rear).
Pressure coefficient calculated with a reference velocity of 28.07m/s, (Isolines generated

corresponding to the inlet velocity at a height of 30m. every 0.125)
Results:
number of CPUs used in the calculation (view from front).

Results:
number of CPUs used in the calculation (view from top).

Results:
Averaged Forces and Moments on Umbrella
Forces and moments time-averaged over the last 1000 seconds for the different
cases were as follows:
# CPUs Fx Fy Fz
8 27838 390 -180251
32 30410 -189 -181709 Values in N
64 27885 392 -180201
# CPUs Mx My Mz
8 -15596 122808 526
32 14494 205487 -1031 Values in N*m
64 -13944 145562 1248
In terms of forces, time averaged values are very similar. However, for the moments
there is a strong variation depending on the number of cores used.
Since the average Cp values on the umbrella do not have any core dependency (as
observed in previous slides), moments should be similar in all 3 cases.
Investigation about this is shown in following slides.

Addendum A
Investigation on Average Moment Figures

Addendum A
Evolution of time-averaged moments and forces along time were evaluated.
Time refers to length of averaging in the 1800s of simulation time: 1000s means that only the last 1000s were included in the
averaging (from 800s to 1800s).
As the averaging time increases, the time-averaged values get closer and the relative
difference in the forces as defined by Max  Min reduce except for Fy, which is close to
Max
0N.
Concerning moments, there are important differences even with averaging of 1500s,
especially for My.
Addendum A
In order to check the impact of the amplitude of oscillations of the forces and
moments, RMS values of forces and moments were calculated, as follows:
RMS forces and moments showed a much more consistent behavior regardless of
the number of cores used in the simulation, with maximum deviations of around 5% in
Fx and 3% in My.

Addendum A
Actually, since the model was symmetric around Y axis, time-averaged value of Fy,
Mx and of Mz should be very close to 0Nm.
When comparing the time-averaged values to the RMS values, the actual
significance of the time-averaged values is put in the right place:
TimeAveraged
RMS
TimeAveraged
RMS

Addendum A
The values of forces and moments do get closer as the averaging time increases. But
the significance of the time-averaged moments is still around 20% of RMS value in
the best case, and are below 2-3% in the cases of Fy, Mx and Mz (virtually 0).
Since the time-averaged values seemed to converge to a similar values regardless of

the number of cores used, simulation was continued up to 3000s (48 minutes).

Addendum A
Results of this continuation up to 3000s were as follows:
With the increase in averaging time, the RMS values are (again) consistent regardless
of the number of CPUs used. All forces and moments RMS values are within around
3% (from max).
The time-averaged force figures converged to uniform values.

Addendum A
The time-averaged values of Mx and Mz are very close to 0Nm (below 3% of the
RMS value). However, the value of My (TimeAveraged/RMS) does not converge to a
uniform value and has some variability, as shown below:
Increasing the averaging time only makes the results slightly more uniform, but even
after 3000s there are important differences between cases.

Addendum A
As fluctuations of forces and moments are very important, sample standard deviation
was calculated for the different cases:
Time-
averaged
values
Sample
Standard
Deviation
Comparing the figures, the standard deviations are more consistent than the time-
averaged values, and they have much higher values than the time-averaged ones
(except Fz).
Actual importance of the time-averaged values should be relativized given the strong
variability along time.
Addendum B
Investigation on OpenFOAM solver
tolerances

Addendum B
Investigation on OpenFOAM solver tolerances
Simulations with 32 and 64 CPUs were re-run for 1800s with tighter solver
tolerances in fvSolutions file, to check if the variability still appeared:
solvers solvers
{ {
p { p {
[…] […]
relTol 0.01; relTol 0.005;
[…] […]
} }
pFinal { pFinal {
[…] […]
tolerance 1.0e-06; tolerance 1.0e-07;
[…] […]
} }
U { U {
[…] […]
tolerance 1e-07; tolerance 1e-08;
[…] […]
} }
Ufinal { Ufinal {
[…] […]
tolerance 1e-07; tolerance 1e-08;
[…] […]
} }
Original tolerances Tighter tolerances

Addendum B
Investigation on OpenFOAM solver tolerances
Results from the new runs showed that the variability does not reduce when
tightening up the solver tolerances.
* refers to simulations done with tighter solver tolerances.

OpenFOAM Benchmark and Validation PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

OpenFOAM Benchmark and Validation PDF

Uploaded by

Copyright:

Available Formats

24070000

OpenFOAM Benchmark and Validation

August 1st, 2012

Copyright © ESI Group, 2012.

Copyright © ESI Group, 2012. All rights reserved. 2

Following steps were carried out for this:

Copyright © ESI Group, 2012. All rights reserved. 3

In terms of average figures:

Model consisted of a umbrella geometry with

Copyright © ESI Group, 2012. All rights reserved. 5

Total Mesh size:

Copyright © ESI Group, 2012. All rights reserved. 6

A wind-type velocity profile was applied

Copyright © ESI Group, 2012. All rights reserved. 9

11 points were monitored:

(Only the first 30 seconds shown here)

(Only the first 30 seconds shown here)

Copyright © ESI Group, 2012. All rights reserved. 13

Copyright © ESI Group, 2012. All rights reserved. 14

After the first

Upstream of the cube, there are no

Looking at the velocity plots at the

Copyright © ESI Group, 2012. All rights reserved. 16

Between the cube and the umbrella,

Copyright © ESI Group, 2012. All rights reserved. 17

Copyright © ESI Group, 2012. All rights reserved. 18

Last ~10min (Isolines

Copyright © ESI Group, 2012. All rights reserved. 19

Similar behaviour is observed from the average static pressure

Copyright © ESI Group, 2012. All rights reserved. 20

Pressure coefficient calculated with a reference velocity of 28.07m/s, (Isolines generated

Pressure coefficient calculated with a reference velocity of 28.07m/s, (Isolines generated

Pressure coefficient calculated with a reference velocity of 28.07m/s, (Isolines generated

Copyright © ESI Group, 2012. All rights reserved. 24

Copyright © ESI Group, 2012. All rights reserved. 25

Evolution of time-averaged moments and forces along time were evaluated.

Copyright © ESI Group, 2012. All rights reserved. 27

Copyright © ESI Group, 2012. All rights reserved. 28

Since the time-averaged values seemed to converge to a similar values regardless of

Copyright © ESI Group, 2012. All rights reserved. 29

Results of this continuation up to 3000s were as follows:

The time-averaged force figures converged to uniform values.

Copyright © ESI Group, 2012. All rights reserved. 30

Copyright © ESI Group, 2012. All rights reserved. 31

Copyright © ESI Group, 2012. All rights reserved. 33

Original tolerances Tighter tolerances

* refers to simulations done with tighter solver tolerances.

Copyright © ESI Group, 2012. All rights reserved. 35

You might also like