Textbook High Performance Computing in Science and Engineering 16 Transactions of The High Performance Computing Center Stuttgart Hlrs 2016 1St Edition Wolfgang E Nagel Ebook All Chapter PDF

High Performance Computing in
Science and Engineering 16

Transactions of the High Performance
Computing Center Stuttgart HLRS 2016
1st Edition Wolfgang E. Nagel
Visit to download the full and correct content document:
https://textbookfull.com/product/high-performance-computing-in-science-and-engineer
ing-16-transactions-of-the-high-performance-computing-center-stuttgart-hlrs-2016-1st
-edition-wolfgang-e-nagel/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...
High Performance Computing in Science and Engineering '

18: Transactions of the High Performance Computing
Center, Stuttgart (HLRS) 2018 Wolfgang E. Nagel
https://textbookfull.com/product/high-performance-computing-in-
science-and-engineering-18-transactions-of-the-high-performance-
computing-center-stuttgart-hlrs-2018-wolfgang-e-nagel/
High Performance Computing in Science and Engineering

15 Transactions of the High Performance Computing
Center Stuttgart HLRS 2015 1st Edition Wolfgang E.
Nagel
https://textbookfull.com/product/high-performance-computing-in-
science-and-engineering-15-transactions-of-the-high-performance-
computing-center-stuttgart-hlrs-2015-1st-edition-wolfgang-e-
nagel/
Advances in High Performance Computing: Results of the

International Conference on “High Performance
Computing” Borovets, Bulgaria, 2019 Ivan Dimov
https://textbookfull.com/product/advances-in-high-performance-
computing-results-of-the-international-conference-on-high-
performance-computing-borovets-bulgaria-2019-ivan-dimov/
High Performance Computing: 34th International

Conference, ISC High Performance 2019, Frankfurt/Main,
Germany, June 16–20, 2019, Proceedings Michèle Weiland
https://textbookfull.com/product/high-performance-computing-34th-
international-conference-isc-high-performance-2019-frankfurt-
main-germany-june-16-20-2019-proceedings-michele-weiland/
High Performance Computing: ISC High Performance 2019
International Workshops, Frankfurt, Germany, June
16-20, 2019, Revised Selected Papers Michèle Weiland
https://textbookfull.com/product/high-performance-computing-isc-
high-performance-2019-international-workshops-frankfurt-germany-
june-16-20-2019-revised-selected-papers-michele-weiland/
Fair Scheduling in High Performance Computing

Environments Art Sedighi
https://textbookfull.com/product/fair-scheduling-in-high-
performance-computing-environments-art-sedighi/
High Performance Computing for Geospatial Applications

Wenwu Tang
https://textbookfull.com/product/high-performance-computing-for-
geospatial-applications-wenwu-tang/
Parallel and High Performance Computing MEAP V09

Robert Robey
https://textbookfull.com/product/parallel-and-high-performance-
computing-meap-v09-robert-robey/
High-Performance Computing for Structural Mechanics and

Earthquake/Tsunami Engineering 1st Edition Shinobu
Yoshimura
https://textbookfull.com/product/high-performance-computing-for-
structural-mechanics-and-earthquake-tsunami-engineering-1st-
edition-shinobu-yoshimura/
Wolfgang E. Nagel
Dietmar H. Kröner
Michael M. Resch Editors
High Performance
Computing
in Science and
Engineering ’16
123
High Performance Computing in Science
and Engineering ’16
Wolfgang E. Nagel • Dietmar H. KrRoner •
Michael M. Resch
Editors
High Performance
Computing in Science
and Engineering ’16
Transactions of the High Performance
Computing Center, Stuttgart (HLRS) 2016
123
Editors
Wolfgang E. Nagel Dietmar H. Kröner
Zentrum für Informationsdienste Abteilung für Angewandte Mathematik
und Hochleistungsrechnen (ZIH) Universität Freiburg
Technische Universität Dresden Freiburg
Dresden Germany
Germany
Michael M. Resch
Höchstleistungsrechenzentrum
Stuttgart (HLRS)
Universität Stuttgart
Stuttgart
Germany
Front cover figure: Bag breakup event during the air-assisted atomization of a liquid fuel. The air
flow field is colored by particle IDs which depend on the creation time and their respective release
position at the inlet. Details can be found in “Smoothed Particle Hydrodynamics for Numerical
Predictions of Primary Atomization”, by S. Braun, R. Koch and H.-J. Bauer, Institut für Thermische
Strömungsmaschinen (ITS), Karlsruher Institut für Technologie (KIT), Karlsruhe, Germany on page
321ff.
ISBN 978-3-319-47065-8 ISBN 978-3-319-47066-5 (eBook)

DOI 10.1007/978-3-319-47066-5
Library of Congress Control Number: 2016963434
Mathematics Subject Classification (2010): 65Cxx, 65C99, 68U20
© Springer International Publishing AG 2016

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, express or implied, with respect to the material contained herein or for any
errors or omissions that may have been made.
Printed on acid-free paper
This Springer imprint is published by Springer Nature

The registered company is Springer International Publishing AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Contents
Part I Physics
The Illustris++ Project: The Next Generation of Cosmological
Hydrodynamical Simulations of Galaxy Formation . . . . .. . . . . . . . . . . . . . . . . . . . 5
Volker Springel, Annalisa Pillepich, Rainer Weinberger,
Rüdiger Pakmor, Lars Hernquist, Dylan Nelson, Shy Genel,
Mark Vogelsberger, Federico Marinacci, Jill Naiman,
and Paul Torrey
Hydrangea: Simulating a Representative Population
of Massive Galaxy Clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 21
Yannick M. Bahé, for the C-EAGLE collaboration
PAMOP Project: Computations in Support of Experiments
and Astrophysical Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 33
B.M. McLaughlin, C.P. Ballance, M.S. Pindzola, P.C. Stancil,
S. Schippers, and A. Müller
Estimation of Nucleation Barriers from Simulations of Crystal
Nuclei Surrounded by Fluid in Equilibrium . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 49
Antonia Statt, Peter Koß, Peter Virnau, and Kurt Binder
The Internal Dynamics and Early Adsorption Stages of
Fibrinogen Investigated by Molecular Dynamics Simulations .. . . . . . . . . . . . . 61
Stephan Köhler, Friederike Schmid, and Giovanni Settanni
Vorticity, Variance, and the Vigor of Many-Body Phenomena
in Ultracold Quantum Systems: MCTDHB and MCTDH-X . . . . . . . . . . . . . . . 79
Ofir E. Alon, Raphael Beinke, Lorenz S. Cederbaum,
Matthew J. Edmonds, Elke Fasshauer, Mark A. Kasevich,
Shachar Klaiman, Axel U.J. Lode, Nick G. Parker,
Kaspar Sakmann, Marios C. Tsatsos, and Alexej I. Streltsov
v
vi Contents
Nucleon Observables as Probes for Physics Beyond the

Standard Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 97
Constantia Alexandrou, Karl Jansen, Giannis Koutsou,
and Carsten Urbach
Numerical Evaluation of Multi-loop Feynman Integrals . . . . . . . . . . . . . . . . . . . . 107
Peter Marquard and Matthias Steinhauser
Part II Molecules, Interfaces, and Solids

Mechanochemistry of Ring-Opening Reactions: From
Cyclopropane in the Gas Phase to Thiotic Acid on Gold in the
Liquid Phase .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 117
Martin Zoloff Michoff, Miriam Wollenhaupt, and Dominik Marx
Microscopic Insights into the Fluorite/Water Interfaces from
Vibrational Sum Frequency Generation Spectroscopy . .. . . . . . . . . . . . . . . . . . . . 131
Rémi Khatib and Marialore Sulpizi
Growth, Structural and Electronic Properties of Functional
Semiconductors Studied by First Principles . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 145
Andreas Stegmüller, Phil Rosenow, Josua Pecher, Nikolay Zaitsev,
and Ralf Tonner
Submonolayer Rare Earth Silicide Thin Films on the Si(111) Surface . . . . 163
S. Sanna, C. Dues, U. Gerstmann, E. Rauls, D. Nozaki, A. Riefer,
M. Landmann, M. Rohrmüller, N.J. Vollmers, R. Hölscher,
A. Lücke, C. Braun, S. Neufeld, K. Holtgrewe, and W.G. Schmidt
Computational Analysis of Li Diffusion in NZP-Type Materials
by Atomistic Simulation and Compositional Screening .. . . . . . . . . . . . . . . . . . . . 177
Daniel Mutter, Britta Lang, Benedikt Ziebarth, Daniel Urban,
and Christian Elsässer
Molecular Dynamics Simulations of Silicon: The Influence of
Electron-Temperature Dependent Interactions .. . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 189
Alexander Kiselev, Johannes Roth, and Hans-Rainer Trebin
Non-linear Quantum Transport in Interacting Nanostructures . . . . . . . . . . . . 203
Benedikt Schoenauer and Peter Schmitteckert
Part III Reactive Flows

A DNS Analysis of the Correlation of Heat Release Rate with
Chemiluminescence Emissions in Turbulent Combustion . . . . . . . . . . . . . . . . . . . 229
Feichi Zhang, Thorsten Zirwes, Peter Habisreuther,
and Henning Bockhorn
Contents vii
Direct Numerical Simulation of Non-premixed Syngas

Combustion Using OpenFOAM . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 245
Son Vo, Andreas Kronenburg, Oliver T. Stein, and Evatt R. Hawkes
Numerical Simulations of Rocket Combustion Chambers with
Supercritical Injection .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 259
Martin Seidl, Roman Keller, Peter Gerlinger, and Manfred Aigner
Two-Zone Fluidized Bed Reactors for Butadiene Production:
A Multiphysical Approach with Solver Coupling for
Supercomputing Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 269
Matthias Hettel, Jordan A. Denev, and Olaf Deutschmann
Part IV Computational Fluid Dynamics

High-Pressure Real-Gas Jet and Throttle Flow as a Simplified
Gas Injector Model Using a Discontinuous Galerkin Method . . . . . . . . . . . . . . 289
Fabian Hempert, Sebastian Boblest, Malte Hoffmann, Philipp
Offenhäuser, Filip Sadlo, Colin W. Glass, Claus-Dieter Munz,
Thomas Ertl, and Uwe Iben
Modeling of the Deformation Dynamics of Single and Twin
Fluid Droplets Exposed to Aerodynamic Loads . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 301
Lars Wieth, Samuel Braun, Geoffroy Chaussonnet, Thilo F. Dauch,
Marc Keller, Corina Höfler, Rainer Koch, and Hans-Jörg Bauer
Smoothed Particle Hydrodynamics for Numerical Predictions
of Primary Atomization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 321
Samuel Braun, Rainer Koch, and Hans-Jörg Bauer
Towards Solving Fluid Flow Domain Identification Problems
with Adjoint Lattice Boltzmann Methods . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 337
Mathias J. Krause, Benjamin Förster, Albert Mink,
and Hermann Nirschl
Investigation on Air Entrapment in Paint Drops Under Impact
onto Dry Solid Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 355
Qiaoyan Ye and Oliver Tiedje
Numerical Study of the Impact of Praestol® Droplets on Solid Walls.. . . . . 375
Martin Reitzle, Norbert Roth, and Bernhard Weigand
Turbulent Skin-Friction Drag Reduction at High Reynolds Numbers . . . . . 389
Davide Gatti
Control of Spatially Developing Turbulent Boundary Layers
for Skin Friction Drag Reduction . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 399
Alexander Stroh
viii Contents
Scalability of OpenFOAM with Large Eddy Simulations and

DNS on High-Performance Systems . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 413
Gabriel Axtmann and Ulrich Rist
Numerical Simulation of Subsonic and Supersonic Impinging
Jets II . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 425
Robert Wilke and Jörn Sesterhenn
Aeroacoustic Simulations of Ducted Axial Fan and Helicopter
Engine Nozzle Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 443
Alexej Pogorelov, Mehmet Onur Cetin, Seyed Mohsen Alavi
Moghadam, Matthias Meinke, and Wolfgang Schröder
Adding Hybrid Mesh Capability to a CFD-Solver
for Helicopter Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 461
Ulrich Kowarsch, Timo Hofmann, Manuel Keßler,
and Ewald Krämer
Direct Numerical Simulation of Heated Pipe Flow with Strong
Property Variation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 473
Xu Chu, Eckart Laurien, and Sandeep Pandey
CFD Analysis of Fast Transition from Pump Mode to
Generating Mode in a Reversible Pump Turbine. . . . . . . . .. . . . . . . . . . . . . . . . . . . . 487
Christine Stens and Stefan Riedelbauch
Scale Resolving Flow Simulations of a Francis Turbine Using
Highly Parallel CFD Simulations.. . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 499
Timo Krappel and Stefan Riedelbauch
CFD Simulations of Thermal-Hydraulic Flows in a Model
Containment: Phase Change Model and Verification of Grid
Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 511
Abdennaceur Mansour, Christian Kaltenbach, and Eckart Laurien
Simulations of Unsteady Aerodynamic Effects on Innovative
Wind Turbine Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 529
Annette Fischer, Levin Klein, Thorsten Lutz, and Ewald Krämer
Part V Transport and Climate

Simulation of the Rain Belt of the West African Monsoon
(WAM) in High Resolution CCLM Simulation . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 547
Diarra Dieng, Gerhard Smiatek, Dominikus Heinzeller,
and Harald Kunstmann
Anthropogenic Aerosol Emissions and Rainfall Decline in
South-West Australia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 559
Dominikus Heinzeller, Wolfgang Junkermann,
and Harald Kunstmann
Contents ix
High-Resolution Climate Projections Using the WRF Model on

the HLRS. . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 577
Viktoria Mohr, Kirsten Warrach-Sagi, Thomas Schwitalla,
Hans-Stefan Bauer, and Volker Wulfmeyer
Biogeophysical Impacts of Land Surface on Regional Climate
in Central Vietnam .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 589
Ngoc Bich Phuong Nguyen, Harald Kunstmann, Patrick Laux,
and Johannes Cullmann
Reducing the Uncertainties of Climate Projections:
High-Resolution Climate Modeling of Aerosol and Climate
Interactions on the Regional Scale Using COSMO-ART:
Interaction of Mineral Dust with Atmospheric Radiation over
West-Africa .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 601
Bernhard Vogel, Hans-Juergen Panitz, and Heike Vogel
Part VI Miscellaneous Topics

Molecular Simulation Study of Transport Properties for 20
Binary Liquid Mixtures and New Force Fields for Benzene,
Toluene and CCl4 .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 613
Gabriela Guevara-Carrion, Tatjana Janzen,
Y. Mauricio Muñoz-Muñoz, and Jadran Vrabec
Large-Scale Phase-Field Simulations of Directional Solidified
Ternary Eutectics Using High-Performance Computing . . . . . . . . . . . . . . . . . . . . 635
J. Hötzer, M. Kellner, P. Steinmetz, J. Dietze, and B. Nestler
Seismic Applications of Full Waveform Inversion . . . . . . . .. . . . . . . . . . . . . . . . . . . . 647
A. Kurzmann, L. Gaßner, N. Thiel, M. Kunert, R. Shigapov,
F. Wittkamp, T. Bohlen, and T. Metz
A Massively Parallel Multigrid Method with Level Dependent
Smoothers for Problems with High Anisotropies .. . . . . . . .. . . . . . . . . . . . . . . . . . . . 667
Sebastian Reiter, Andreas Vogel, Arne Nägel, and Gabriel Wittum
Part I
Physics
Peter Nielaba
In this section, eight physics projects are described, which achieved important
scientific results by using the CRAY XC40 (Hornet and Hazel Hen) of the HLRS.
Fascinating new results are being presented in the following pages on astrophysical
systems (simulations of galaxy formation, of massive galaxy clusters, and of
photodissociation), soft matter systems (simulations of nucleation in colloidal
systems and of dynamics and adsorption of fibrinogen), many body quantum
systems (simulations of ultracold quantum systems) and elementary particle systems
(simulations of nucleon observables and of the anomalous magnetic moment of the
muon).
The studies of the astrophysical systems have focused on the galaxy formation,
massive galaxy clusters, and on photodissociation of certain molecules.
V. Springel, A. Pillepich, R. Weinberger, R. Pakmor, L. Hernquist, J. Naiman,
D. Nelson, M. Vogelsberger, and F. Marinacci from Heidelberg (V.S., A.P., R.W.,
R.P.), Cambridge USA (L.H., J.N., M.V., F.M.) and Garching (D.N.), in their
project GCS-ILLU present results from a new generation of hydrodynamical
simulations (“Illustris++” project, AREPO code), including new black hole physics
and chemical enrichment models, using more accurate techniques and an enlarged
dynamical range. The authors reproduced the appearance of a red sequence of
galaxies, quenched by accreting supermassive black holes and computed disk galax-
ies populations with properties closely matching observational data. In addition,
the authors predicted magnetic field amplifications through small-scale dynamo
processes for galaxies of different sizes and types.
Yannick M. Bahé and the C-EAGLE collaboration from Garching used in their
HLRS project GCS-HYDA the GADGET-3 code to simulate 25 galaxy clusters with
high resolution (“Hydrangea” project). By the ongoing data analysis new insights
P. Nielaba ()
Fachbereich Physik, Universität Konstanz, 78457 Konstanz, Germany
e-mail: peter.nielaba@uni-konstanz.de
2 P. Nielaba
into the physics of galaxy formation in an extreme environment and on the growth
of the massive haloes, in which cluster galaxies are embedded, are achieved.
B M McLaughlin, C P Ballance, M S Pindzola, P C Stancil, S Schippers and A
Müller from the Universities of Belfast (B.M.M., C.P.B.), Auburn (M.S.P.), Georgia
(P.C.S.), and Giessen (S.S., A.M.) investigated in their project PAMOP atomic,
molecular and optical collisions on petaflop machines in order to support measure-
ments at synchrotron radiation facilities and to study photodissociation effects for
astrophysical applications. The Schrödinger and Dirac equations have been solved
with the R-matrix or R-matrix with pseudo-states approach, and the time dependent
close-coupling method has been used. Various systems and phenomena have been
investigated, ranging from X-ray and inner-shell photoionization in atomic oxygen
and argon ions, as well as in tungsten ions, the single-photon double ionization in
helium, to the photodissociation in SHC .
The studies of the soft matter systems have focused on nucleation barriers in
colloidal systems and on the dynamics and adsorption of fibrinogen.
A. Statt, P. Koß, P. Virnau and K. Binder from the University of Mainz present in
their project colloid a method to study the free energy barriers for homogeneous
nucleation of crystals from a fluid phase, which is not hampered by the fact
that the fluid-crystal interface tension in general is anisotropic. By Monte Carlo
simulations in the NpT ensemble, using the softEAO model for colloidal systems,
and by analyzing the equilibrium of a crystal nucleus surrounded by fluid in a small
simulation box in thermal equilibrium, the fluid pressure, chemical potential and
the volume of the nucleus have been computed to obtain the nucleation barrier.
Interesting deviations from the classical nucleation theory with spherical nucleus
assumptions have been discovered and analysed.
S. Köhler, F. Schmid and G. Settanni from the University of Mainz investigated
in their project Flexadfg dynamical properties of fibrinogen and of the initial adsorp-
tion stages of fibrinogen on mica and graphite surfaces by atomistic Molecular
Dynamics simulations. The adsorption simulations on mica showed a preferred
adsorption orientation in a reversible process without large deformations of the
protein, and the adsorption simulations on graphite showed an irreversible character
and a formation of a large quantity of protein-surface contacts which eventually lead
to deformations of the protein and the onset of denaturation.
In the last granting period, quantum mechanical properties of elementary particle
systems have been investigated as well as the quantum many body dynamics of
trapped bosonic systems.
O.E. Alon, R. Beinke, L.S. Cederbaum, M.J. Edmonds, E. Fasshauer, M.A.
Kasevich, S. Klaiman, A.U.J. Lode, N.G. Parker, K. Sakman, M.C. Tsatsos,
A.I. Streltsov from the Universities of Haifa (O.E.A.), Heidelberg (R.B., L.S.C.,
S.K., A.I.S.), Newcastle (M.J.E., N.G.P.), Tromso (E.F.), Stanford (M.A.K.), Basel
(A.U.J.L.), Wien (K.S.), Sao Paulo (M.C.T.) studied in their project MCTDHB ultra-
cold atomic systems by their method termed multiconfigurational time-dependent
Hartree for bosons (MCTDHB). The principal investigators have focused on
seven topics: (a) single shots imaging of dynamically created quantum many-
body vortices, (b) many-body tunneling dynamics of Bose-Einstein condensates
I Physics 3
and vortex states in 2D, (c) transition from vortices to solitonic vortices in 2D
trapped Bose-Einstein condensates, (d) variance as a sensitive probe of correla-
tions and uncertainty product of an out-of-equilibrium many-particle system, (e)
development of a multiconfigurational time-dependent Hartree method for fermions
(“MCTDH-X”) (f) trapped fermions escape, (g) composite fragmentation of multi-
component Bose-Einstein condensates.
C. Alexandrou, K. Jansen, G. Koutsou and C. Urbach from Nicosia (C.A., G.K.),
Zeuthen (K.J.) and Bonn (C.U.) investigated in their project GCS-Nops the inner
structure of the proton and other hadrons by lattice chromodynamics. By generating
the ensemble using directly the physical value of the pion and nucleon masses, the
principal investigators were able to compute the hadron spectrum, the axial and
tensor charges moments of parton distribution functions and the quark contents of
the nucleons.
P. Marquard and M. Steinhauser from Zeuthen (P.M.) and Karlsruhe (M.S.)
computed in their project NumFeyn multi-loop Feynman integrals for the electron
contribution to the anomalous magnetic moment of the muon, using the FIESTA
package.
The Illustris++ Project: The Next Generation
of Cosmological Hydrodynamical Simulations
of Galaxy Formation
Volker Springel, Annalisa Pillepich, Rainer Weinberger, Rüdiger Pakmor,

Lars Hernquist, Dylan Nelson, Shy Genel, Mark Vogelsberger,
Federico Marinacci, Jill Naiman, and Paul Torrey
Abstract Cosmological simulations of galaxy formation provide the most powerful

technique for calculating the non-linear evolution of cosmic structure formation.
This approach starts from initial conditions determined during the Big Bang – which
are precisely specified in the cosmological standard model – and evolves them
forward in time to the present epoch, thereby providing detailed predictions that
test the cosmological paradigm. Here we report first preliminary results from a new
V. Springel ()
Astronomisches Recheninstitut, Zentrum für Astronomie der Universität Heidelberg,
Mönchhofstr. 12–14, 69120, Heidelberg, Germany
Heidelberg Institute for Theoretical Studies, Schloss-Wolfsbrunnenweg 35, 69118, Heidelberg,
Germany
e-mail: volker.springel@h-its.org
A. Pillepich
Max-Planck Institute for Astronomy, Königstuhl 17, 69117, Heidelberg, Germany
e-mail: pillepich@mpia-hd.mpg.de
R. Weinberger • R. Pakmor
Heidelberg Institute for Theoretical Studies, Schloss-Wolfsbrunnenweg 35, 69118, Heidelberg,
Germany
e-mail: rainer.weinberger@h-its.org; ruediger.pakmor@h-its.org
L. Hernquist • J. Naiman
Center for Astrophysics, Harvard University, 60 Garden Street, 02138, Cambridge, MA, USA
e-mail: lars.hernquist@cfa.harvard.edu; jill.naiman@cfa.harvard.edu
D. Nelson
Max-Planck Institute for Astrophysics, Karl-Schwarzschild-Str. 1, 85740, Garching, Germany
e-mail: dnelson@mpa-garching.mpg.de
S. Genel
Department of Astronomy, Columbia University, 550 W. 120th St., 10027, New York, NY, USA
e-mail: shygenelastro@gmail.com
M. Vogelsberger • F. Marinacci • P. Torrey
Kavli Institute for Astrophysics and Space Research, MIT, 02139, Cambridge, MA, USA
e-mail: mvogelsb@mit.edu; fmarinac@mit.edu; ptorrey@mit.edu
© Springer International Publishing AG 2016 5

W.E. Nagel et al. (eds.), High Performance Computing in Science
and Engineering ’16, DOI 10.1007/978-3-319-47066-5_1
6 V. Springel et al.
generation of hydrodynamical simulations that excel with new physics, enlarged

dynamic range and more accurate numerical techniques. The simulations of our
ongoing Illustris++ project on HazelHen successfully reproduce the appearance of
a red sequence of galaxies that are quenched by accreting supermassive black holes,
while at the same time yielding a population of disk galaxies with properties that
closely match observational data. Also, we are able to predict the amplification of
magnetic fields through small-scale dynamo processes in realistic simulations of
large galaxy populations, thereby providing novel predictions for the field strength
and topology expected for galaxies of different size and type.
1 Introduction
In principle, simulations of cosmic structure formation are well-specified initial

value problems that ought to be able to predict galaxy formation in an ab-
initio manner. However, the enormous dynamic range and the complex baryonic
processes in galaxy formation make this an extremely challenging multi-scale,
multi-physics problem whose full understanding is still a distant goal. Nevertheless,
earlier simulations have already proven instrumental for developing our current
understanding of structure formation, even given their underlying simplifications.
Indeed, cosmological simulations have played a crucial role in establishing CDM
as the leading cosmological theory, despite our present ignorance of the true physical
nature of dark matter and dark energy.
In particular, dark matter only simulations such as the Millennium simulations
[1–3] have led to significant physical insight and reached a high degree of maturity
and accuracy. However, such DM-only simulations do not provide predictions
regarding the galaxies themselves, and an extra step is required in order to bridge
the gap with observations. Primarily two approaches have been used to establish
this link: (1) the technique of “semi-analytical modeling”, whereby baryonic
physics is modeled coarsely at the scale of an entire galaxy and applied in post-
processing on top of DM simulations [4, 5], and (2) hydrodynamic simulations,
where the evolution of the gaseous component of the Universe is treated using
the methods of computational fluid dynamics. The latter approach, together with
subgrid prescriptions that provide numerical closure and that take into account
astrophysical processes related to star formation, enables the complex interaction
of the different baryonic components (gas, stars, black holes) to be treated on a
much smaller scale, ideally yielding a self-consistent and powerfully predictive
calculation.
Our group has published in 2014 the presently largest hydrodynamic simulations
of galaxy formation [6–8]. This simulation suite – dubbed “Illustris” – used a
different approach than the ones so far commonly adopted in astrophysics to
simulate gas on a computer (“smoothed particle hydrodynamics”, SPH, and “Eule-
rian” mesh-based methods, typically utilizing adaptive mesh refinement, AMR).
Illustris employed a moving, unstructured mesh as it has been implemented in
Galaxy Formation in Illustris++ 7
our code AREPO [9]: like in AMR, the volume of space is discretized into many
individual cells, but as in SPH, these cells move with time, adapting to the flow
of gas in their vicinity. As a result, the mesh itself, constructed through a Voronoi
tessellation of space, has no preferred directions or regular grid-like structure. Over
the past few years, we have shown that this new type of approach for simulating
gas has significant advantages over the other two methods, particularly for large
cosmological simulations like Illustris [10–13].
One of the major achievements of the Illustris simulation is its ability to track the
small-scale evolution of gas and stars within a representative portion of the Universe.
The calculation yielded a population of thousands of well-resolved elliptical and
spiral galaxies, reproduced the observed radial distribution of galaxies in clusters
and the characteristics of hydrogen on large scales, and at the same time it matches
the metal and hydrogen content of galaxies on small scales. However, the analysis
of Illustris has also revealed a number of tensions with observational data. In
particular, it has become clear that the physical model used for the so-called radio-
mode feedback [14] of accreting supermassive black holes has been too strong and
violent at the scale of galaxy groups and low-mass clusters, causing a depletion of
their baryon content. At the same time, this physical model still proved insufficient
to quench the central galaxies in these systems to the required degree, causing
these galaxies to become too massive. Other problems we identified were the
normalization of the faint-end of the galaxy luminosity function, too large galaxy
sizes, and a lack of a pronounced bimodality in the galaxy color distribution. In
addition, important physical ingredients such as magnetic fields were still missing.
This provides the motivation for the ‘Illustris++’ project that we currently carry
out as a GCS large-scale project on HazelHen at HLRS. The primary scientific goal
of our project is to calculate new, unprecedentedly large hydrodynamic simulation
models of the universe that improve upon the earlier Illustris project in several
important respects. Most importantly, we aim to improve the feedback models by
using a newly developed model for black hole accretion and its associated energy
release, by employing a considerably improved multi-species chemical enrichment
model, by adjusting the treatment of galaxy winds, and last but not least, by adding
magnetic fields to our simulations, opening up a rich new area of predictions that are
still poorly explored, given that the body of cosmological magneto-hydrodynamic
(MHD) simulations is still very small [15–22]. In particular, we aim to study the
strength of magnetic field amplification through structure formation as a function of
halo and galaxy size. We will also be able to quantify for the first time the expected
distribution of magnetic field properties for galaxies of a given size, and to explore
the role of winds and strong feedback events in “polluting” the intergalactic-medium
with magnetic fields. In addition, we aim for a larger number of resolution elements,
and a larger simulation volume than realized previously. This is necessary to study
the regime of galaxy clusters better (which are rare and can only be found in a
sufficiently large volume), and to allow a sampling of the massive end of the galaxy
and black hole mass functions.
At the time of this writing, our project is still running, and only a subset of the
production calculations have finished, with some of the main runs presently being
computed. In this article, we describe some of the developments undertaken for the
project, detail practical and technical aspects of our work, and the status of our runs.
We also describe a few preliminary results in an exemplary fashion.
2 Physics and Code Developments for Illustris++
2.1 New Blackhole Physics Model
As discussed above, we replaced the so-called ‘radio-mode’ of supermassive black

hole accretion and feedback in our AREPO code with a novel implementation. The
physics of active galactic nuclei (AGN) is crucial for quenching star formation in
large galaxies, particularly the central galaxies in groups and clusters of galaxies.
While our previous model for blackhole growth in Illustris worked well in certain
respects [23], it also showed significant deficits, in particular, it reduced the
Sunyaev-Zeldovich decrement in galaxy groups as a result of excessive gas loss, and
led to a still insufficient reduction of the star formation rate in the largest clusters,
making the central galaxies not red enough.
We have therefore adopted a new kinetic feedback model for AGN driven winds,
motivated by recent theoretical suggestions that conjecture advection dominated
inflow-outflow solutions for the accretion flows onto the black holes in this regime
[24]. For a detailed description of the new model we refer to our recent preprint
[25]. In brief, our approach estimates the gas accretion rate through the Bondi-
Hoyle-Lyttleton model. However, unlike in previous work, we have eliminated
any artificial boost factor to the accretion rate in favor of starting with a slightly
higher seed mass of 5 105 Mˇ . In terms of feedback, we distinguish between a
quasar mode for high accretion rates where the feedback is purely thermal, and a
kinetic mode for low accretion rate states where the feedback is purely kinetic. The
latter replaces the old radio-mode. Instead, we now inject kinetic energy directly
at the position of the black hole, in random directions, so that the time-averaged
momentum injection vanishes. The distinction between the two feedback modes
is based on the Eddington ratio of the black hole accretion. For Eddington ratios
above 0.1, the black hole is assumed to be always in the quasar mode, but for lower
Eddington ratios we make the threshold dependent on the black hole mass, such that
it becomes progressively easier for low mass black holes to stay in the quasar mode.
Or expressed differently, large black hole masses will eventually transition to the
kinetic mode, and as this has a higher impact on the host system than the thermal
feedback of the quasar mode, this will tend to reduce the accretion rate further, such
that the system will then typically stay in the kinetic mode. In the kinetic feedback
state, strong quenching of cooling flows and star formation in the halo is possible,
such that the corresponding galaxy quickly reddens.
2.2 Hierarchical Time Integration
The simulations carried out in Illustris++ represent a significant challenge not only
in terms of size and spatial dynamic range, but also in terms of the dynamic
range in timescales. In particular, the strong kinetic AGN feedback, which couples
to the densest gas in galaxies, induces very small timesteps for a small fraction
of the mass. Over the course of 13 billion years of cosmic evolution, we need
up to 107 timesteps in total. This would be completely infeasible with time
integration schemes that employ global timesteps, but even for the individual
timestepping we use in AREPO, this represents a formidable problem. It can
only be tackled if the computation of sparsely populated timesteps can be made
extremely fast so that they do not dominate the total CPU time budget. This in turn
requires elimination of overheads that touch the full particle/cell system on such
timesteps.
For Illustris++, we have developed a novel hierarchical timestepping scheme
in our AREPO code that solves this in a mathematically clean fashion. This
is done by recursively splitting the Hamiltonian describing the dynamics into
a ‘slow’ and a ‘fast’ system (similar to [26]). One important feature of this
time integration scheme is that the split-off fast system is self-contained, i.e. its
evolution does not rely on any residual coupling with the “slow” part. This means
that our goal, namely that poorly populated short timesteps can be computed
without touching any parts of the system living on longer timesteps, can be
realized.
2.3 Chemical Enrichment Model
For Illustris++, we have also improved our modelling of chemical enrichment, both
by using updated yield tables that account for the most recent results of stellar
evolution calculations, and by making the tracking of different chemical elements
more accurate and informative. The most important technical measure to achieve
this has been the introduction of a fiducial “other chemical elements” mass bin, such
that together with the explicit tracking of 9 chemical elements (H, He, C, N, O, Ne,
Mg, Si, Fe), the metal abundance vector accounts for the full mass content of every
cell or star. Since we do a spatial reconstruction for every element individually,
the previous code could arrive at extrapolated flux vectors at cell interfaces with
an unphysical sum of the 9 explicitly tracked elements, leaving for the other
elements a negative contribution. In our new treatment, the density of these other
elements is reconstructed as well, and the abundance pattern is renormalized after
extrapolation, thereby always leading to physically viable chemical compositions at
flux exchanges.
The other significant change we made is that we now account with special
chemical tagging fields separately for metals produced by asymptotic giant branch
(AGB) stars, type-II supernovae, and type-Ia supernovae. This has not been done
before in such hydrodynamical simulations, and opens up a rich additional set of
analysis possibilities which are largely unexplored thus far. Given that the metallic-
ity patterns in the circumgalactic medium are emerging as a critical observational
diagnostic and constraint for the feedback physics, this is a very timely extension of
our modelling capabilities.
2.4 Hydrodynamical Accuracy Improvements
For certain problems, the original implementation of AREPO reached only first-
order convergence in the L1 norm. In [27] we have shown that this can be rectified
by simple modifications in the time integration scheme and the spatial gradient
estimates of the code, both acting together to improve the accuracy of the code. As a
result, the new formulation used for Illustris++ is now second-order accurate under
the L1 norm even in unfavorable situations. As a welcome side effect, conservation
of angular momentum is substantially improved, too. We have found that these
changes can significantly improve the results of smooth test problems. On the
other hand, we also showed that cosmological simulations of galaxy formation
are unaffected for well resolved galaxies, demonstrating that the numerical errors
eliminated by the new formulation do not impact these simulations significantly.
Nevertheless, the improved accuracy of the new formulation is clearly to be
preferred, and we expect that small, poorly resolved galaxies are rendered more
accurately in Illustris++ than before, corroborating the advantage of our moving-
mesh technique compared to SPH or AMR codes in this regime.
We have also made important improvements to the ideal magnetodynamics
(MHD) solver in our code [28], primarily in the form of an additional timestep
criterion that controls the size of the Powell source terms applied for divergence
control. Previously, this was not checked explicitly, instead the timestep of a
cell was determined only by the Courant condition and a kinematical timestep
constraint. It could thus happen under rare conditions that the source terms would
apply order unity corrections to the magnetic field over the course of a timestep,
leading to a relatively large local error. In our new code used for Illustris++, this
is now safely prevented, increasing both the accuracy and robustness of our MHD
implementation.
2.5 Elimination of All-to-All Communication Steps
In the AREPO code, we need to carry out, at multiple places, parallel, distributed
tree walks that serve to calculate, e.g., the short-range gravitational field, the
local enrichment region of stellar populations, or the zone of accretion around a

supermassive black hole. Algorithmically, each MPI rank first does a range-search
on its local domain, during which it also detects if the search region overlaps with
other domains. In the latter case, a search request to the foreign domain is registered.
These are exchanged to the corresponding target rank and processed in a second
phase of the distributed tree walk. The number of these search requests for each of
the other possible MPI ranks is stored in a table. After the first tree walk phase,
the table is communicated with an MPI_Alltoall such that each rank knows how
many items it needs to import from each other processor. As the domain layout is
highly irregular as a result of the work-load and memory-balancing, the detailed
communication pattern arising here is irregular and sparse, and cannot be predicted
ahead of time.
The sparseness however implies that for a large number of MPI ranks mostly
zero entries are communicated in the MPI_Alltoall. This has not been a critical
source of overhead thus far if the dynamic range of the simulation is limited, but
becomes an issue in simulations with 104 MPI ranks and beyond, where the smallest
timesteps (which are carried out most frequently) need to be very fast so as to
not start dominating the total CPU budget. Illustris++ runs on HazelHen are our
first large production simulations where this source of overhead plays a sizable
role.
To mitigate this, we have developed during the first project phase a relatively
involved rewrite of our communication patterns in the parallel tree walks, which can
completely eliminate the need for an MPI_Alltoall. Instead, we can now employ a
sparse communication pattern that makes use of an asynchronous collective barrier.
The latter is in principle available as part of MPI-3.0, but for compatibility reasons
with older systems that do not yet support MPI-3.0 we have implemented an efficient
sparse asynchronous communication pattern for the distributed tree walks ourselves,
relying only on MPI-2 features.
3 Simulation Set and Production Runs
After obtaining access to HORNET/HazelHen, we have first carried out a limited

number of test simulations plus a number of science runs of zooms into the
Illustris volume targeting individual galaxies (these also served to test our new
physics implementations, and several publications about them are currently in
preparation). These had confirmed that our simulation code AREPO runs effectively
and without technical issues on the new Cray XC40 machine. This also helped us to
establish the precise performance for our simulation work-load, both with respect to
computational throughput, communication bandwidth and I/O speed. In all three
areas, the high expectations we had for the XC40 were met (module some I/O
issues that initially surfaced, but which could be resolved by switching our project
to a different filesystem). Also, our tests did not reveal any technical obstacles
against carrying out our simulations, aside from a surprisingly low memory ceiling
for the application code on the compute nodes. We could initially not use more
than 3900 MB per core in large partition runs without sometimes falling victim
to OOMs, caused by memory needs of the I/O subsystem and MPI buffers. How
to work around this reliably required a lot of experimenting and testing on our
end.
After finalizing our modification of the physical model of Illustris++, we
adjusted our plans for the primary science runs in the project by first carrying out
“IllustrisPrime”, a very demanding simulation with 18 billion resolution elements
in a 75 h1 Mpc box similar to the original Illustris run, but now using the new
full physics model of Illustris++ with all its improvements, as well as including
magneto-hydrodynamics (MHD). We also now adopted the newest cosmological
models as determined by the PLANCK Satellite. This cosmological simulation is the
first that includes MHD and resolves galaxy formation at high resolution, opening
up many possibilities for novel predictions. Also, IllustrisPrime will be ideal to
convincingly demonstrate that we can solve the problems at the bright end of the
galaxy luminosity functions that have troubled all previous simulations in the field,
including our older Illustris simulation that presently defines the state-of-the-art in
this area.
In Table 1, we give an overview of our primary production simulations, omitting
smaller test calculations. We are currently still in the process of finalizing one of
Table 1 Overview of primary production runs carried out by the Illustris++ project. All simu-
lations use PLANCK cosmological parameters and are carried out with a tracer particle method
that is faithful with respect to the mass flux in the system between all baryonic components. We
typically use two Monte Carlo tracers per Voronoi cell, i.e. Ntracers D 2 Ncells . All simulations
follow more than 13 billion years of cosmic evolution, with smallest timesteps of order a few
thousand years
Symbolic name Boxsize Ndm Ncells MPI ranks Physics Run status
L75n1820TNG 75 h1 Mpc 18203 18203 10;752 Final full Advanced
physics model
L75n1820MF 75 h1 Mpc 18203 18203 12;000 Alternative Finished
AGN feedback
L75n910TNG 75 h1 Mpc 9103 9103 2688 Final full Finished
physics model
physics model
L205n2500TNG 205 h1 Mpc 25003 25003 24;000 Final full Started
physics model
L35n2160TNG 35 h1 Mpc 21603 21603 16;320 Final full Started
physics model
physics model
L12.5n512TNG 12:5 h1 Mpc 5123 5123 1200 Final full Finished
physics model
our main production runs using 10,752 cores on HazelHen, while IllustrisPrime
has already finished. In addition, we are carrying out two further large calculations
which just have been started. They are substantially larger and either excel in volume
or mass resolution, respectively. We have already transferred more than 240 TB of
production data to the Heidelberg Institute of Theoretical Studies, in part by using
fast gridftp services offered by HLRS. From the ongoing runs, we expect about
200 TB of additional data, which we will also transfer to Heidelberg for the scientific
analysis in the coming years.
4 Selected Preliminary Results
In Fig. 1, we illustrate the large-scale distribution of different quantities in the

IllustrisPrime simulation (L75n1820MF). From top to bottom, we show projections
of the gas density field, the mean mass-weighted metallicity, the mean magnetic
field strength (field energy weighted), the dark matter density, and the stellar density.
The displayed regions are 75 h1 Mpc from left to right, and 3:75 h1 Mpc deep. We
can nicely see the cosmic web on large-scales, formed both in the dark matter and
the diffuse gas. The color hue in the gas distribution shown on top encodes the
mass-weighted temperature across the slice. We see that the largest halos are filled
with hot plasma, and in addition, there is clearly evidence for very strong outflows
in the largest halos impinging on the intergalactic medium, leading to relatively
widespread heating.
The bottom panel in Fig. 1 displays the stellar mass density. Clearly, on the scales
shown in this image, the individual galaxies appear as very small dots, illustrating
that the stellar component fills only a tiny fraction of the volume. However, our
simulations have enough resolution and dynamic range to actually resolve the
internal structure of these galaxies in remarkable detail. This is shown in Fig. 2,
which zooms in on two disk galaxies formed in our simulations. The one on the
right hand panel is in a more massive halo and has a more massive black hole. This
in fact has made it start to transition into the quenched regime, which here begins
by a reduced star formation in the center as a result of kinetic AGN feedback. The
outskirts of the galaxy still support some level of star formation, causing blue spiral
arms.
Of particular interest in our new simulations is the magnetic field that builds
up in halos and galaxies. In Fig. 3, we show a typical disk galaxy in a face-on
orientation, plotting the magnetic vector field overlaid on a rendering of the gas
density in the background. We see that the field is ordered in the plane of the disk,
where it has been amplified by shearing motions to sizable strength. Interestingly,
there are multiple field reversals and a complicated topology of the field surrounding
the disk. The magnetic field not only provides additional pressure for the gas, but
also is of critical importance for transport processes of heat energy and cosmic rays.
Our realistic field topologies should be very useful for studying the propagation of
Fig. 1 Thin projections through the L75n1820MF simulation, showing the gas density field, the
metallicity, the magnetic field strength, the dark matter density, and the stellar density
Fig. 2 Stellar mass distributions of two disk galaxies in halos of mass 8:3 1011 Mˇ (left) and
2:0 1012 Mˇ (right), respectively, in face-on (top) and edge-on projections (bottom). The stellar
colors are assigned according to their age and metallicity
cosmic rays in the Milky Way, and for analyzing the deflections of ultra-high energy
cosmic rays of extra-galactic origin.
In Fig. 4, we show an analysis of the typical magnetic field strengths reached in
halos of different size. We here plot radial profiles for four different halo masses,
stacking up to 50 halos in a narrow mass range around the virial masses 1010 ,
1011 , 1012 , and 1013 Mˇ . We see that field strengths of several G are reached
in the centres of galaxies in halos of masses 1011 1012 Mˇ , in good agreement
with typical observed fields. In smaller halos, the fields are still notably weaker,
presumably because here they have not yet been amplified as efficiently. In larger
halos, 1013 Mˇ and beyond, they are also weaker in the centers, but for a different
reason. Here some of the magnetic flux is expelled by strong nuclear outflows
driven by AGN feedback. In any case, the strength of the simulated fields implies a
remarkable amplification relative to the primordial fields that we seeded in the initial
conditions. This initial field strength is empirically largely unconstrained, but our
results reached for the field strength in galactic centres do not depend on the value
we used (which in our case was 1011 Gauss) over a wide dynamic range, because
the magnetic amplification processes stop once the dynamo processes responsible
for the exponential amplification saturate. This happens when the magnetic pressure
becomes comparable to the thermal pressure.
Fig. 3 Magnetic field structure in a disk galaxy (the one displayed in the left-hand panel of Fig. 2),
overlaid over a rendering of the gas density structure (color-scale in the background). The length
of the drawn vectors is made only weakly dependent on the magnetic field strength (as / jBj1=4 )
in order to see more of the field structure in the regions with weaker fields
On large scales, however, the magnetic field strengths reached in voids still reflect
the initial field. This is clearly seen in Fig. 5, which gives a phase-space diagram of
gas density versus magnetic field strength. The correlation B / 2=3 (indicated as
a solid line) reflects adiabatic expansion/compression of the initial field set at the
starting redshift z D 127. At baryonic overdensities of around 100, we see that
much larger fields are created. This is in part due to the amplification of the field
through large-scale shearing flows and in part due to a small-scale dynamo driven
by star formation and galactic wind feedback on small scales.
In sum, our calculations demonstrate that already an extremely tiny magnetic
field left behind by the Big Bang is sufficient to explain orders of magnitude larger
field strengths observed today. Interestingly, the magnetic field strength found in
the simulation agrees very well with the values measured for the Milky Way and
neighboring galaxies. This is remarkable given that there are no free parameters
influencing the magnetic field amplification that could be tuned to modify the final
field strength reached in our simulated galaxies.
10.00 10.00
log(M ) = 10.0 log(M ) = 11.0
200 200
1.00 1.00
B [ μG ]
B [ μG ]
0.10 0.10
0.01 0.01
0.01 0.10 1.00 0.01 0.10 1.00

R/ R200 R/ R200
10.00 10.00
log(M ) = 12.0 log(M ) = 13.0
200 200
1.00 1.00
B [ μG ]
B [ μG ]
0.10 0.10
0.01 0.01
0.01 0.10 1.00 0.01 0.10 1.00

R/ R200 R/ R200
Fig. 4 Spherically averaged profiles of the mean magnetic field strength in halos of different mass.
Each panel shows a stacked set of up to 50 halos in a narrow mass bin around a different virial mass,
as labeled in each panel
Another powerful application of our simulations lies in studies of the metal

enrichment in the universe. We track 9 chemical elements explicitly, and all other
elements are lumped together in a 10-th fiducial component so that the advected
metallicity vectors always correspond to physically meaningful abundance patterns
in all situations. In addition, we use a newly developed metal tagging technique,
allowing us to characterize which fraction of metals in every cell or star originated
from AGB stars, supernova type II explosions, or type-Ia explosions.
In Fig. 6, we show a break down of the total metal content in the gas phase of
the simulated universe at different times as a function of gas density. The individual
histograms are normalized to the total metal content in the gas at the corresponding
epoch. The distributions can hence answer the question at which gas densities the
majority of the metals can be found. Interestingly, we see that most of the metals
are actually stored at gas densities that correspond to the circumgalactic medium,
whereas only a smaller fraction is contained in the star-forming interstellar medium,
and very little in the low-density intergalactic medium. The relative shares between
these phases are time-dependent, with more metals found in low density gas towards
late times. This is most likely a result of the feedback that expelled these metals from
the galaxies.
Fig. 5 Phase-space diagram of the magnetic field strength versus gas overdensity at z D 0 in one
of our Illustris++ simulations. The line shows the locus corresponding to adiabatic compression or
expansion of the initial field strength
5 Conclusions
Understanding the feedback processes in galaxy formation and evolution is the

principal challenge in theoretical extragalactic astronomy. This question is also of
critical relevance for cosmology, as baryonic processes impact the distribution of
dark matter, and hence in turn affect cosmological probes that aim to constrain, for
example, the physical properties of dark energy. Solving the feedback conundrum
is unthinkable without further refining the simulation models and the employed
numerical methods. This is due to the multi-scale and multi-physics nature of the
problem, which tends to limit analytic approaches for studying the problem to highly
schematic and correspondingly uncertain models.
The Illustris++ project aims to redefine the state-of-the-art of cosmological
hydrodynamical simulations of galaxy formation. In particular, our new simulations
make significant progress on predicting the bright end of the galaxy luminosity
function through the use of a new model for AGN feedback. Also, realistic galaxy
sizes, morphologies and colors are obtained at the same time. The scientific analysis
of the simulation promises a rich harvest and will primarily focus on testing the
0.4
z = 0.0
z = 1.0
z = 2.0
0.3 z = 4.0
z = 7.0
met,tot
M
0.2
dM / dlog
met
0.1
0.0
10-2 100 102 104 106 108
ρ/ <
Fig. 6 Distribution of the gas phase metal content with respect to baryonic overdensity at different
epochs, as labelled. Each distribution is normalized to the total metal mass in the gas at the
corresponding time
models further. Our simulation predictions for the gas around galaxies, the so-
called circum-galactic medium (CGM) are particular timely, as the Cosmic Origins
Spectrograph (COS) on board of the Hubble Space Telescope (HST) has provided a
wealth of absorption line data probing this phase. In addition, our simulations allow
us to make novel, testable predictions for the magnetic field strength in different
environments, and its correlation with other galaxy properties.
Acknowledgements The authors gratefully acknowledge computer time through the project
GCS-ILLU on Hornet/HazelHen at HLRS. We acknowledge financial support through subproject
EXAMAG of the Priority Programme 1648 ‘SPPEXA’ of the German Science Foundation, and
through the European Research Council through ERC-StG grant EXAGAL-308037, and we would
like to thank the Klaus Tschira Foundation.
References
1. Springel, V., White, S.D.M., Jenkins, A., Frenk, C.S., Yoshida, N., Gao, L., Navarro, J.,
Thacker, R., Croton, D., Helly, J., Peacock, J.A., Cole, S., Thomas, P., Couchman, H., Evrard,
A., Colberg, J., Pearce, F.: Nature 435, 629 (2005). doi:10.1038/nature03597
2. Boylan-Kolchin, M., Springel, V., White, S.D.M., Jenkins, A., Lemson, G.: MNRAS 398, 1150
(2009). doi:10.1111/j.1365-2966.2009.15191.x
3. Angulo, R.E., Springel, V., White, S.D.M., Jenkins, A., Baugh, C.M., Frenk, C.S.: MNRAS
426, 2046 (2012). doi:10.1111/j.1365-2966.2012.21830.x
4. Kauffmann, G., Colberg, J.M., Diaferio, A., White, S.D.M.: MNRAS 303, 188 (1999).
doi:10.1046/j.1365-8711.1999.02202.x
5. Cole, S., Lacey, C.G., Baugh, C.M., Frenk, C.S.: MNRAS 319, 168 (2000).
doi:10.1046/j.1365-8711.2000.03879.x
6. Vogelsberger, M., Genel, S., Springel, V., Torrey, P., Sijacki, D., Xu, D., Snyder, G., Bird, S.,
Nelson, D., Hernquist, L.: Nature 509, 177 (2014). doi:10.1038/nature13316
7. Vogelsberger, M., Genel, S., Springel, V., Torrey, P., Sijacki, D., Xu, D., Snyder, G., Nelson,
D., Hernquist, L.: MNRAS 444, 1518 (2014). doi:10.1093/mnras/stu1536
8. Genel, S., Vogelsberger, M., Springel, V., Sijacki, D., Nelson, D., Snyder, G., Rodriguez-
Gomez, V., Torrey, P., Hernquist, L.: MNRAS 445, 175 (2014). doi:10.1093/mnras/stu1654
9. Springel, V.: MNRAS 401, 791 (2010). doi:10.1111/j.1365-2966.2009.15715.x
10. Vogelsberger, M., Sijacki, D., Kereš, D., Springel, V., Hernquist, L.: MNRAS 425, 3024
(2012). doi:10.1111/j.1365-2966.2012.21590.x
11. Sijacki, D., Vogelsberger, M., Kereš, D., Springel, V., Hernquist, L.: MNRAS 424, 2999
(2012). doi:10.1111/j.1365-2966.2012.21466.x
12. Torrey, P., Vogelsberger, M., Sijacki, D., Springel, V., Hernquist, L.: MNRAS 427, 2224
(2012). doi:10.1111/j.1365-2966.2012.22082.x
13. Bauer, A., Springel, V.: MNRAS 423, 2558 (2012). doi:10.1111/j.1365-2966.2012.21058.x
14. Sijacki, D., Springel, V., Di Matteo, T., Hernquist, L.: MNRAS 380, 877 (2007).
doi:10.1111/j.1365-2966.2007.12153.x
15. Dolag, K., Bartelmann, M., Lesch, M.: A&A 348, 351 (1999)
16. Dolag, K., Bartelmann, M., Lesch, H.: A&A 387, 383 (2002). doi:10.1051/0004-
6361:20020241
17. Dolag, K., Grasso, D., Springel, V., Tkachev, I.: J. Cosmol. Astropart. Phys. 1, 009 (2005).
doi:10.1088/1475-7516/2005/01/009
18. Donnert, J., Dolag, K., Lesch, H., Müller, E.: MNRAS 392, 1008 (2009). doi:10.1111/j.1365-
2966.2008.14132.x
19. Bonafede, A., Dolag, K., Stasyszyn, F., Murante, G., Borgani, S.: MNRAS 418, 2234 (2011).
doi:10.1111/j.1365-2966.2011.19523.x
20. Kotarba, H., Lesch, H., Dolag, K., Naab, T., Johansson, P.H., Donnert, J., Stasyszyn, F.A.:
MNRAS 415, 3189 (2011). doi:10.1111/j.1365-2966.2011.18932.x
21. Beck, A.M., Lesch, H., Dolag, K., Kotarba, H., Geng, A., Stasyszyn, F.A.: MNRAS 422, 2152
(2012). doi:10.1111/j.1365-2966.2012.20759.x
22. Marinacci, F., Vogelsberger, M., Mocz, P., Pakmor, R.: MNRAS 453, 3999 (2015).
doi:10.1093/mnras/stv1692
23. Sijacki, D., Vogelsberger, M., Genel, S., Springel, V., Torrey, P., Snyder, G.F., Nelson, D.,
Hernquist, L.: MNRAS 452, 575 (2015). doi:10.1093/mnras/stv1340
24. Yuan, F., Narayan, R.: ARA&A 52, 529 (2014). doi:10.1146/annurev-astro-082812-141003
25. Weinberger, R., Springel, V., Hernquist, L., Pillepich, A., Marinacci, F., Pakmor, R., Nelson,
D., Genel, S., Vogelsberger, M., Naiman, J., Torrey, P.: ArXiv e-prints (2016)
26. Pelupessy, F.I., Jänes, J., Portegies Zwart, S.: New A 17, 711 (2012).
doi:10.1016/j.newast.2012.05.009
27. Pakmor, R., Springel, V., Bauer, A., Mocz, P., Munoz, D.J., Ohlmann, S.T., Schaal, K., Zhu,
C.: MNRAS 455, 1134 (2016). doi:10.1093/mnras/stv2380
28. Pakmor, R., Bauer, A., Springel, V.: MNRAS 418, 1392 (2011). doi:10.1111/j.1365-
2966.2011.19591.x
Hydrangea: Simulating a Representative
Population of Massive Galaxy Clusters
Yannick M. Bahé, for the C-EAGLE collaboration
Abstract Galaxy clusters are the most massive bound structures in the Universe,
and contain not only up to several thousand galaxies, but also extended haloes of
dark matter and hot gas. Observations show that galaxies in clusters differ from
those living in more isolated parts of the Universe, but the physics of how clusters
shape their galaxies is at present not well understood. Not only does this constitute
a major gap in our understanding of galaxy formation, but it also limits the use of
galaxy clusters as cosmological probes. In the Hydrangea project, we have created
a suite of 24 simulated galaxy clusters at unprecedented resolution, using a state
of the art galaxy formation model developed for the EAGLE project. Detailed
scientific analysis of the simulation outputs, which has only just begun, is expected
to lead to major new insight into the physics of both galaxy formation in an extreme
environment and the growth of the massive haloes in which cluster galaxies are
embedded.
1 Introduction
Galaxy clusters are collections of large numbers of galaxies – up to several

thousands in the most extreme cases – occupying a region in our Universe that
is typically a few megaparsec (Mpc1 ) in size. Moreover, observations have firmly
established that the galaxies which are seen in optical light are in fact only a minor
component of these objects: the dominant constituents in terms of mass are instead
extended, diffuse ‘haloes’ of dark matter (DM) and very hot gas (the ‘intra-cluster
medium’ or ICM) that together account for typically more than 90 % of the mass
of a galaxy cluster [1]. Including these optically invisible components, the mass
of the largest such objects exceeds 1015 times the mass of our Sun .Mˇ /, making
1
The parsec (pc) is the standard unit of length in astronomy, with 1 pc = 3:08 1016 m.
Y.M. Bahé ()
Max Planck Institut für Astrophysik, Karl-Schwarzschild-Str. 1, 85748, Garching, Germany
e-mail: ybahe@mpa-garching.mpg.de
© Springer International Publishing AG 2016 21

W.E. Nagel et al. (eds.), High Performance Computing in Science
and Engineering ’16, DOI 10.1007/978-3-319-47066-5_2
Another random document with
no related content on Scribd:
Cuareim River with Quarahim, Brazil, and its Great Southern
Railway running northward. At Salto there is considerable
interchange of traffic with Argentina through Concordia opposite, an
important railway junction and city. At Cabellos connection is made
with the Uruguay Northern, another road running to the Brazilian
Boundary, the terminus San Eugenio.
The Uruguay East Coast Railway with 78 miles of road, starting
from Olmas on the Central Uruguay runs to Maldonado. Much of the
traffic is to the seaside resorts, Puente del Este and Piriápolis; there
is also considerable freight for the Montevideo market, of agricultural
and pastoral products and fish. An extension northward from San
Carlos to Rocha is authorized. The road has been purchased by the
Government. A line from Durazno on the Central to Trinidad, begun
by the Farquhar-Pearson Syndicate as part of a line designed to
cross the country diagonally from Colonia to the Brazilian border,
was taken over by the Government, which has in view the securing
of a system of State railways. Besides building the 31-mile line from
San Carlos on the East Coast Railway it intends purchasing the 23-
mile line from Rocha to the port Paloma. Further needed
construction is planned by the Government as soon as may be
practicable.
Aeroplane service is to be installed by a British company from
Montevideo to Rio de Janeiro and Pernambuco, and aerial postal
service is planned for the interior.
Resources and Industries
Stock raising is by far the leading industry, as is evident from the

fact that in 1917 nearly 98 per cent of the exports were of stock
products. Of the 44 million acres devoted to livestock and agriculture
the latter occupies hardly 5 per cent. In 1603, 100 cattle and two
herds of horses were brought into the country; the cattle increased
so rapidly that at one time they were killed for their hides, as more
recently in Paraguay. Since 1860 when Durham bulls were imported
and stock breeding began, much advance has been made in quality.
Herefords, Devon, a few Polled Angus and others have also been
imported; some for dairy purposes, as Swiss and Flemish. In 1917
the cattle numbered about 8,000,000. The best estancias have
sheds to house pedigreed stock, they plant trees and have cattle
dips. Societies encourage scientific breeding and the Government
subsidizes agricultural shows. One estancia of 60,000 acres has
15,000 cattle, 20,000 sheep, some horses, and pays dividends of
16-25 per cent on a capital of £120,000. Another company with
40,000 acres and a capital of £40,000 pays dividends of 14-20 per
cent. At least 20,000 acres are devoted to dairy farming; 50,000
pounds of butter are made monthly, and both butter and cheese are
exported. The best of apparatus is employed.
Sheep, imported from Argentina in 1608, flourish to the number of
11,700,000; the varieties of Merino, Lincoln, Romney Marsh are
found among others, the English breeds being preferred.
Horses of good quality are raised, 570,000, both light and heavy,
but few in comparison to the cattle. There are 300,000 pigs, a few
mules, 16,000, and 12,000 goats.
Meat Packing. In 1754 the first meat salting plant was established
but the true pioneer dates from 1786. Others followed. There are
now 13 besides seven factories for preserving meat and two
frigorificos. The slaughter season is from November to January. The
meat for saladeros is separated from the bones, dried 4-6 days in
the sun, and then salted. It is arranged in four grades according to
fat or lean; the fat meat is sent to Brazil, the lean to Cuba and
elsewhere.
Of frigorificos the Swifts own one, and exported to Europe in 1915
and 1916, each, over 700,000 frozen quarters of beef and 100,000
chilled; also mutton and lamb. The Frigorifica Uruguaya shipped
nearly 44,000,000 pounds of beef to Europe in 1916 and over
2,000,000 pounds of mutton. The total export of animal products
shipped in 1916 was worth $73,000,000. A model slaughter house
and cold storage plant is in prospect. A new one for wool washing
has a capacity of 132,000 pounds daily. Saving in freight cost and
immunity from deterioration are thereby attained.
The Liebig Extract of Beef Company, with extensive holdings in
and near Fray Bentos and with a total capitalization of £2,000,000
usually pays 20 per cent dividends on the ordinary shares; 5 per cent
in 1916. They use the best of meat, and their Oxo capsules and
Lemco have a world-wide reputation. They own and rent in Uruguay,
Argentina, and Paraguay 1,120,000 acres.
In spite of strikes and labor troubles the livestock industry has
brought prosperity to the country, with record prices for beef, mutton,
hides, and wool, thus greatly increasing land values. The cattle are
pastured on the natural long thick grass, very little alfalfa being
cultivated. Hogs, hens, bees, and silkworms are raised. The seal
industry and fisheries are important.
Agriculture
The agricultural products are insufficient for the use of the country
although 2,000,000 acres are in cultivation. About 900,000 acres are
in wheat, 700,000 in corn, 128,000 in flax, 100,000 oats, some
barley and canary seed. In 1916 agricultural exports were valued at
$1,500,000. Among other crops are tobacco, which is especially
promising, linseed, alfalfa, sugar cane, some cotton, potatoes, etc.
Viticulture is quite extensive, American grapes growing better in the
south, and French and Italian in the north. Other fruits, oranges,
olives, apples, pears, cherries, peaches, and melons flourish.
Forestry
Forestry is encouraged so far as planting is concerned; about

1,000,000 acres are in natural forest land. Millions of trees have
been planted on land not otherwise useful. The supply of wood in
future will be greatly increased and there may even be export.
Minerals
Minerals are of some importance and may become of more. The

country is believed to contain considerable wealth in gold, silver,
coal, marble, jasper; and in other minerals and semi-precious
stones, including amethyst and topaz. There is little export save
sand, stone for paving, and similar articles.
Gold. The chief gold fields are in the Department Rivera near
Cuñapiru not far from the Brazilian border. A district 35-40 miles long
and 7 wide contains auriferous reef with gold 5 ounces to the ton; if
deep the prospect is limitless. A modern English plant is now getting
out gold. Bars worth $4000 were exported in 1915. Four hundred
mines have been denounced in the Department. Enormous
quantities of manganese are in the neighborhood. Gold is found also
in Minas, Salto, and Tacuarembó.
Copper exists in quantity in Cerro Largo, Maldonado, Minas,
Paysandú, and Salto. Iron, silver, slate, gypsum, asbestos, lead, etc.
may be exploited later. Even greatly needed coal of fair quality has
been found in Montevideo, Santa Lucía, and especially in Cerro
Largo where it seems promising, though no working of minerals is
sufficiently developed to present decisive results. Indications of
petroleum have been noted at the north, the strata coming in from
Brazil.
Manufactures
Naturally manufacturing save for home consumption is of slight

importance, except of products of the pastoral industry, as of dairies
and of meat extract. For home use there are 115 flour mills, 45
others, as of hardware, soap, macaroni; 1 sugar factory, 3 starch, 1
cement, 4 breweries. Many of these are in Montevideo. The
Government proposes the construction of chemical factories for the
production of sulphuric, nitric, carbolic, and acetic acid, glycerine,
benzol, alcohol, sulphuric ether, etc., and a powder and explosive
factory; these to cost over $2,000,000, material and machinery to
enter free of duty.
An important project of the Government is the development of
water power from the cataracts of the Uruguay River, which will be
equivalent to 3,000,000 tons of coal per annum. Two dams are
planned, one movable and one fixed, with canals by which 419 miles
of river will be open to navigation from the lower section. Irrigation is
included in the project, and 37,000 acres near Montevideo are to be
irrigated as an illustration. Fifteen cities have authorized work in
connection with this project.
Investments
Aside from the development of hydro-electric power and the

construction of public works of various kinds including railways, it is
probable that agriculture and fisheries present the most favorable
openings, with good possibilities also in manufacturing industries,
stone cutting, and mining. Stock raising is already pretty well
developed.
CHAPTER XLV
BRAZIL: AREA, HISTORY, GOVERNMENT, POPULATION, ETC.
The country of Brazil, largest of the South American Republics, has also
a greater area than the United States without Alaska, and is more than
three-fourths the size of all Europe. It cannot therefore be considered as a
whole so easily as the other Republics. It is essential to differentiate
between the various regions and States; for the dissimilarity is not confined
to climate and productions; or to the character of the people, by reason of
some being indigenous and others of European descent. It arises in part
from the long coast line and the difficulty of land communication; in part
from the fact that in some districts the population is almost entirely of
European descent while in others there is a large percentage of negro
blood; as well as from differences in physical and climatic conditions. Thus
the Capital is not so markedly the centre of the Republic as in Argentina,
and the States are more loosely bound together than in the other
Republics. The States and the character of the people may be said to
differ as much among themselves as the countries of the West Coast from
each other, a point of importance to notice in commercial relations.
Area, Population, Boundary
Area. Brazil covers a surface of 3,112,453 square miles. Its length, 2750
miles, is about that of Chile; its extreme width, 2560 miles, is ten times at
great. The coast-line is much longer, 4140 miles. A considerable portion of
this immense area is still but superficially explored.
Population. According to the cabled report of the census of 1920, Brazil
has 30,553,509 inhabitants. Its population, therefore, exceeds that of any
other South American Republic even more largely than its area.
Boundary. The boundary of the country, though familiar from that of the
others, may be rehearsed. On the north we find Colombia, Venezuela, and
the three Guianas with the Atlantic beyond; on the east and southeast the
Atlantic only, on the south Uruguay, a speck of Argentina, Paraguay,
Bolivia, a bit of Peru; on the west a small corner of Argentina, Paraguay,
Bolivia, and Peru. The only countries of the continent not touching its
border are Ecuador and Chile. However, a few writers mention Ecuador on
the west, as the southeast boundary line of that country is still
undetermined.
History
The first of the South American countries to be discovered after

Colombia and Venezuela, it was to the region of Brazil that the name
America was first applied. It is therefore especially unbecoming for us to
appropriate to ourselves in any exclusive sense the title of Americans;
though having no other name, with apologies to the others, it may be
pardonable for us to employ it when necessary.
In the year 1500 the first landing on this part of the continent was made
by Pedro Alvares Cabral, then on his way from Portugal to the West
Indies. In commemoration of that event, May 3rd is a Brazilian national
holiday and the date of the assembling of Congress. As soon as the news
was received in the home country, an expedition was sent out under
Amerigo Vespucci, who explored the coast from its eastern extremity
almost to La Plata, nearly 2000 miles. Fifteen Captaincies, each 150 miles
along the coast, were later allotted and settlements were begun. The
earliest of these which rose to importance were São Vicente in the
neighborhood of Santos, and Pernambuco; a little later, Bahia and Rio de
Janeiro. These were the first agricultural colonies to be founded in South
America, gold and silver being the attraction elsewhere. The French also
had an eye to this country, making a settlement at Rio de Janeiro; the
Dutch as well, who about a century later captured Bahia and Pernambuco;
but both were ultimately expelled, the whole country remaining in the
hands of the Portuguese. Conflicts with the Indians took place, at first with
some who were unfriendly, and afterwards through attacks made by the
invaders upon those Indians who had been christianized by the Jesuits.
Their settlements were destroyed, 300,000 are said to have been
slaughtered, and the rest were driven by the Paulistas from the region of
the upper Paraná.
As Philip II of Spain in 1581 became ruler of Portugal, during the 60
years following, the expansion of Brazil to the west in territory which had
been assigned to Spain was permitted, and such possession remained
permanent. At other times conflicts occurred with the Spaniards at the
south, but in 1777 peace was made with the boundaries as at present.
In 1807, Prince João, fleeing from Napoleon, came with his court to
Brazil. He soon opened the five chief ports to commerce, he encouraged
science, education, literature, art, and the immigration of foreigners, thus
inaugurating a development of permanent value. On his return to Europe
in 1821, the Prince, in view of the revolts of the Spanish colonies, hinted to
his son whom he left in charge the advisability of himself assuming the
crown, if a disposition towards independence became manifest.
Accordingly in 1822, the son was crowned Emperor of Brazil; but having
alienated his supporters, in 1831 he abdicated in favor of his infant son,
Pedro. In 1889, the old Emperor, Dom Pedro II, who for many years had
ruled wisely and well, was expelled on 24 hours’ notice; after a brief interim
a Republic was established in 1891. Extravagance, insurrections, and
financial distress followed, but since 1900 the country has made rapid
advancement in wealth and in varied lines of development.
Government
The government is a federalized republic with the usual branches, the

States more loosely bound together than with us, or than in any other
South American Republic. They may even fix export taxes, and levy stamp
duties. The President, with a Vice President, is elected for four years and
is ineligible for a succeeding term. He has a Cabinet of seven Ministers.
Congress is composed of a Senate with 63 members and a Chamber of
212 Deputies. There are 20 States, a Federal District, and the Territory of
Acre. The last is composed of three Prefectures, with capital cities where
government is administered by Government appointees. The States have
their own administrative bodies, some with one house of legislation, some
with two; and with a Governor or President as chief executive, a slight
confusion possibly arising at times where the latter term is employed. All
male citizens over 21 may vote except illiterates, soldiers, beggars, and
members of monastic orders subject to vows of obedience, a wise
prescription. The list of States precedes, with the usual figures, as
accurate as obtainable, the areas from the latest Government survey. The
list begins at the northwest, goes down the coast, and follows with the
interior.
States Area, Population Capitals Population Altitude,
in in feet
square
miles
Amazonas 645,940 435,000 Manaos 60,000 131
Pará 399,000 992,300 Belem 250,000 23
Maranhão 150,830 853,000 Maranhão 40,000 198
Piauhy 89,850 548,250 Therezina 35,000
Ceará 62,160 1,436,300 Fortaleza 65,000
Rio Grande 15,925 552,000 Natal 20,000 25
do Norte
Parahyba 22,548 785,100 Parahyba 20,000
Pernambuco 38,570 1,975,440 Recife 200,000
Alagôas 10,400 990,000 Maceió 40,000
Sergipe 8,983 535,000 Aracajú 30,000
Bahia 206,990 3,373,000 São 300,000 147
Salvador
Espirito 16,860 479,200 Victoria 20,000
Santo
Rio de 16,408 1,502,000 Nictheroy 30,000
Janeiro
São Paulo 101,890 4,823,000 São Paulo 510,000 2,510
Paraná 73,465 674,300 Curityba 50,000 2,980
Santa 43,168 633,000 Florianopolis 30,000
Catharina
Rio Grande 92,350 2,138,800 Porto Alegre 125,000
do Sul
Minas 227,238 5,789,000 Bello 35,000 3,081
Geraes Horizonte
Goyaz 284,000 529,000 Goyaz 18,000 1,577
Matto 554,400 274,100 Cuyabá 32,000 953
Grosso
Territory of 67,712 104,000
Acre
Federal 450 1,150,080 Rio de 1,150,080
District Janeiro
Population
The population, by the recent census 30,553,509, is of a more varied
character than that of Uruguay and Argentina at the south. Some figures
given are 52 per cent white, 26 mixed, 13 Indian, and 9 per cent negro.
The original settlers were Portuguese, and at first immigration was from
the mother country. In the early days many negroes were imported from
Africa as slaves, yet there was little color prejudice so that the number of
mulattoes and lighter as well as of negroes in some sections is very large.
During the last hundred years over 3,000,000 immigrants have arrived,
of whom the Italians formed the larger proportion; next in number were the
Portuguese, half as many Spaniards; those of other nationalities included
100,000 Germans, and a small colony from our Southern States, who left
in disgust in 1867. The negroes, freed in 1888 and endowed with suffrage,
were less qualified for it than in the United States. While some have made
good advancement others have relapsed into a worse condition, being
able in many parts of the country to exist on almost nothing. Indolence is a
failing among many of all complexions, as is natural in tropical regions; on
the other hand many Brazilians even in the warmer sections are
characterized by great activity and industry. In the large cities culture and
elegance are noticeable and aristocracy of birth is cherished. In some
regions the inhabitants are less pretentious, live more simply, and practise
the homely virtues; the most primitive section according to Oakenfull is
between the São Francisco River and Maranhão. Women in general are
more secluded than in some of the other Republics. The Brazilians have
much literary and artistic taste and as a rule are punctilious in courtesy,
though exceptions may be noted.
Education
Education is highly regarded in Brazil. Primary and secondary are free

and secular, generally provided by the States and Municipalities. The
Federal Government administers several Professional Schools as of
Medicine, Law, Engineering, etc., and many of Agriculture in various parts
of the country. With some of these, local schools are affiliated, as Schools
of Law, of Applied Science, at Rio of Social Science, at Recife
Engineering, etc. The different States spend 4-21 per cent of their revenue
on primary education, averaging above ten. The Federal District spends 28
per cent. There are also private schools in different cities, several English
or American; the American Mackenzie College at São Paulo is affiliated
with the University of the State of New York.
Press, Religion, etc.
The Press is influential and of high quality, the leading papers of Rio,
São Paulo, and other cities comparing well with those of cities of
corresponding size in other parts of the world.
Religion. In Brazil there is entire separation of Church and State and
absolute religious freedom. Civil marriage alone is recognized.
Post and Telegraph. Brazil has 3700 Post Offices.
Telegraph wires (over 26,000 miles) are in part national; other lines
belong to the railways; there are submarine cables, and one up the
Amazon. Wireless stations have been installed at many points on the
coast and in the interior, including the Amazon district, as at Manaos and
beyond. Telephones are to be found in all cities of any considerable size,
about 80 systems.
Money is more complicated and bothersome than in any of the other
countries, the unit being of 1000 instead of 100 parts, as is usual. Thus the
milreis, written 1$000, equals 1000 reis as the name indicates. The milreis
of gold is equal to 54.6 cents, but exchange varies, and the paper in
common use varies from its ordinary value, 33.3 cents, to half that or even
less in disturbed conditions. A conto of reis, a term often used, is 1000
milreis, and is written with a colon, thus: 5 contos, 20 milreis, and 300 reis
would be written 5:20$300.
The Metric System is legal and compulsory, but in some places, the old
Portuguese measures persist; these differ from the Spanish. A vara in
Peru is less than a yard, but in Brazil it is 1.111 metre, or 1.215 yard. A
libra is 4.695 kilos; an alquiere varies from 24 to 160 litres. Other variety
exists in the same or in different places.
CHAPTER XLVI
BRAZIL: PHYSICAL CHARACTERISTICS
Brazil presents in physical characteristics more variety than is

generally supposed. As the great Amazon Basin is in striking
contrast to the immense Andean Range, the entire country is thought
of as hot. Since it extends from 5° 10′ N. Lat. to 33° 45′ S. Lat. with
the widest part near the equator, the greater portion of the territory is
evidently in the torrid zone, 11° only in the temperate, with more than
twice that in the tropics. However, in this comparatively low country,
there is happily a variation in altitude sufficient to affect the climate
and to give rise to variety in productions; to which diversity the 11° in
the temperate zone also contribute. The territory may be considered
as in four general sections: the Amazon Basin, the Plata (the two
almost connected over low elevations), the Coastal Belt, and a mass
of mountains and highlands along the coastal states, extending also
at a lower level across to Bolivia. In addition there are the Guiana
Highlands at the north.
The Coastal Belt
The coastal section is largely a low-lying sandy tract, varied by

swamp lands overgrown with palms or other verdure, and slopes
covered with dense tropical vegetation. Without deep indentations in
the form of gulfs and bays there is a considerable number of good
harbors. In the far south two large lakes have been created which
are connected with the ocean. The coastal strip varies in width from
one to 100 miles.
The Guiana Highlands
Of the mountainous regions, the range forming the boundary line

with Venezuela and Guiana on the north with its offshoots and the
country between has attracted little interest and been but slightly
explored. From Mt. Roraima, 8500 feet, at the corner of Venezuela
and British Guiana, the range lowers toward the East, the highest
point on the French frontier being about 2600 feet. South of this are
broken ranges and deep river courses on the Brazilian plateau,
which with an altitude of 2000 feet slopes south and east. Excepting
the part near the coast, this section called Brazilian Guiana is semi-
arid, on account of the mountains extracting the moisture from the
northeast trade winds. Inhabited by a few roving bands of Indians
and in the east visited by white mining adventurers, it has been
deemed one of the least attractive parts of the Republic. Recent
exploration, however, has reported an extent of valuable forest lands
and immense areas of open country suited to cattle ranges. A
railway from British Guiana is talked of to render this district
accessible to the outside world.
The Plateau and Mountain Region
An important part of Brazil is the plateau region (altitude 1000-

3000 feet) south of the Amazon, especially that portion extending
along the coastal states. The greater part of the central section was
once covered with a thick sandstone sheet, now deeply eroded by
numerous rivers which have left high flat ridges between the lower
basins. The true mountain systems which rise from the plateau are
parallel ranges following the coast, and the Central or Goyana
system. An almost continuous range, the Serra do Mar, stretches
from Rio Grande do Sul to Cape Frio, just east of Rio de Janeiro;
beyond this, farther from the coast, broken sections extend well
towards Cape St. Roque in Rio Grande do Norte. The highest point
in the Serra do Mar or Coast Range is 7223 feet in the Organ
Mountains near Rio, as the capital city is often called for short and
may be so understood when here used.
A second parallel range runs from Eastern São Paulo northeast
and north to the point where the São Francisco River turns east in
Bahia; Itatiaiá, 9823 feet, in Rio de Janeiro State, is its culminating
point in Brazil. West of the São Francisco River is the Central or
Goyaz Range, also in two branches, running from the southern part
of Goyaz northeast, one branch across western Minas Geraes, the
other in Goyaz with highlands extending far north separating the
Tocantins Basin from the São Francisco, and in the south separating
it from the Paraná. The highest point is near the city of Goyaz, Mt.
Pyraneus, 7750 feet. How the great table-land has been broken
down by rivers is shown by the Tocantins-Araguaya Basin excavated
northward for 700 miles. From 100 to 500 miles wide, it is from 10 to
1500 feet deep.
The eastern margin of the plateau is near the Atlantic where it
breaks off suddenly with an average height of 2600-3200 feet. This
plateau is the best part of the country, the Atlantic slope heavily
wooded, the more gradual west slope with many grass covered
plains. A fertile soil, rich forests, and regular rainfall make this a
favored region. Farther north than Minas Geraes, the section has
less rain, is thinly wooded in places, with large areas suited to
grazing but subject to drought. The plateau extends across Brazil
north of Paraguay abruptly breaking down from a height of 2600-
3000 feet towards the Paraná and Paraguay Basins. It has a gradual
slope towards the Amazon, the parts excavated by the rivers having
lowland characteristics.
The Amazon Basin
The section of the Amazon Basin is indeed immense. The whole

drainage area with that of the Tocantins, generally included though
not strictly a component part, spreads over 2,700,000 square miles,
much of it as we have already seen outside of Brazil, and occupying
two-fifths of South America. This area is greater than the basins of
the Mississippi, Missouri, Danube, and Nile, all combined. The
course of one tributary to the sea is 3200 miles. Of the 55 largest
rivers in the world, it is said that 33 are mainly if not wholly in Brazil.
Twenty-seven thousand miles of navigable rivers are found in the
country, the greater part connected with the Amazon, which is itself
navigable for ocean steamers to Iquitos, Peru, a distance of 2300
miles, and 486 miles farther for vessels of 14 feet draft. The true
mouth of the Amazon is west and north of the Marajos Island which
is greater in area than Holland or Belgium, while the Pará River at
the south and east is the mouth of the Tocantins. This, however, is
the usual entrance for Amazon steamers even when they do not
wish to call at Pará, for this river is connected by natural channels
among islands with the greater stream, and the northern entrance
has too strong a current to be desirable for navigation. As a matter of
convenience the Tocantins is generally included with the Amazon in
descriptions. The depth of the estuary is 90-900 feet, averaging 150.
The Tocantins River is a great artery of Brazil flowing from south
to north a distance of 1600 miles, with a west affluent, the Araguaya,
almost its equal in size and 1000 miles long above the union. Both
rivers receive many tributaries. The flat broad valleys are overlooked
by steep bluffs. The cataracts and rapids which occur on nearly all of
the rivers as they come down from the plateau greatly interfere with
navigation, but in their lower courses many are navigable for
hundreds of miles, the Tocantins for 130.
The Xingú, the next river, with a length of 1260 miles is the first
true tributary of the Amazon on the south. This also flows nearly
north with many rapids and at last a fall, Itamaracá, at the head of
navigation, 105 miles from its mouth. Near this the river expands into
a great lake which communicates with the Amazon by many natural
channels.
The Tapajós, 1158 miles long, entering the Amazon about 500
miles above Pará, rises near Diamantino on the plateau, and flows
through a long, hot, unhealthful valley. One hundred and eighty-eight
miles are navigable, the lower 100 of these being 4-9 miles wide and
throughout much of its course very deep. Along the lower river valley
are bluffs 300-400 feet high.
The Madeira, entering 870 miles above Pará, almost rivals the
Amazon in volume. In the rainy season, during which it rises 50 feet,
the largest ocean steamers may ascend 665 miles to the falls of
Santo Antonio and the Madeira-Mamoré Railway; from June to
December it is navigable for vessels of 5-6 foot draft only. Tributary
to the Madeira on the east is the river formerly called the Duvida, first
completely traversed by Colonel Roosevelt and his party and now
named in his honor Rio Roosevelt. The Madeira which has 90
tributaries and a basin of over 600,000 square miles is formed a little
higher up by the junction of the Beni and Mamoré, both of which
streams rise close to the head waters of the Paraguay. From Santo
Antonio south, the Madeira-Mamoré is obstructed for 263 miles by a
series of rapids and cataracts, the Madeira Falls, where the Beni
comes in, presenting a vast display of whirlpool and boiling torrents.
Above Santo Antonio the drainage is received of the southwest slope
of the State of Matto Grosso, the north slope of the Chiquitos
Sierras, and the northeast slope of the Andes from Santa Cruz de la
Sierra in Bolivia to Cuzco in Peru. The most important of the
affluents are the Guaporé, Baurés, Itonamas, Mamoré, Beni, and
Madre de Dios. The almost level Mojos and Beni plains are said to
rival if not to exceed in fertility the Nile Valley; they are the most
healthful and most inviting grazing and agricultural regions in the
Amazon Basin, which has an area about equal to that of France and
Spain. However, 35,000 square miles are flooded 2-3 feet, three or
four months every year.
The Purús, over 2000 miles in length, is a very sluggish stream
parallel to the Madeira, in the great depression between the Brazilian
plateau and the Andes. A peculiar feature is five parallel canals
coming into it from the Amazon northwest at almost regular intervals,
making five low islands; and nearer to the great river are three more.
The Purús is navigable for light steamers 1648 miles five months in
the year; for 800 miles its depth is not less than 45 feet. The lands
are subject to inundation, the river at times rising 50 feet.
The Juruá is a similar river, navigable for 1133 miles.
The Javary, the boundary line between Brazil and Peru, is
navigable for craft 260 miles. The region is occupied by savages.
The Trombetes. On the north side of the Amazon there are fewer
important rivers, the Trombetes, the first from the east, which is
navigable 135 miles, comes from the Guiana Highlands.
The Negro, 900 miles from Pará, 1500 miles long, is navigable for
450 miles forming midway a succession of lagoons, and overflowing
its banks, often for a width of 20 miles. The rivers farther west have
been sufficiently described.
The average depth of the Amazon is 50 feet, the current is three
miles an hour. Beginning to rise in November the river is fullest in
June, then falling to November. The Madeira, which rises and falls
two months earlier, is in places 4-6 miles wide. The width of the
Amazon is 20-60 miles, while in periods of inundation the forest is
partly submerged for a width of 400 miles.
The Plata Basin
A much smaller portion of Brazil lies in the basin of La Plata; this,

at least for the moment, is the best and richest part of the entire
country, containing the greatest population outside of the coastal
fringe.
The Uruguay. At the south the several rivers forming the Uruguay,
which rise in the Serra do Mar, drain Rio Grande do Sul and part of
Santa Catharina, while from there up to the north end of São Paulo
and into Goyaz only a narrow coast region is outside the Paraná
Basin.
The Paraná. The most remote source of the Paraná, that of the
branch Paranahyba, is in the Serra Pyreneos in Goyaz, while the Rio
Grande branch rises in the Serra da Mantiqueira near the peak
Itatiaiá, so to say, in sight of Rio. Many affluents are received from
the States of São Paulo and Paraná, these generally flowing
northwest or west; the Paraná itself flows southwest forming the
western boundary of those States. A branch, the Tiété, in São Paulo,
700 miles long, is broken by 54 rapids and two falls. The
Paranapanema in Paraná, 600 miles long, is navigable for 30 miles.
The Iguassú, rising in the Serra do Mar in Santa Catharina and
flowing west is hardly navigable for canoes.
Twenty-eight miles above the mouth of the Tiété the course of the
Paraná is interrupted by the Falls of Urubupungá. From here to the
Guaira or Sete Quedas Falls, 400 miles, there is unobstructed
navigation. At this point the river forms a lake 4¹⁄₂ miles long and 2¹⁄₂
wide before cutting through the Serra de Maracajú. Then after
rushing through a deep and narrow gorge for two miles, it plunges
down a long cañon hardly 200 feet wide in a series of rapids or falls
called the Sete Quedas, Seven Falls. It is reported as able to supply
over a million horse power, probably the most of any cataract in the
world. Again the river is navigable from a little below the falls, and
with regular service it forms an outlet for the State of Paraná to the
ocean.
The Paraguay. The River Paraguay rises near Diamantino in
Matto Grosso receiving a number of tributaries from that State, one
of which, the Cuyabá, called the São Lourenço lower down, has its
source close to that of the Tapajós branch of the Amazon.
Coast Rivers and Lakes
Other rivers flow directly into the Atlantic, several of some

importance. North of the Amazon a few small rivers are called
auriferous, the Araguary of considerable length. South of the
Amazon Delta, some rivers flow north and northeast among the
mountain ridges, others directly to the ocean. Of the former the most
important are the Parnahyba, 900 miles long, navigable in sections,
and farther south the São Francisco, the largest river of the coastal
region, navigable 192 miles from the coast to the fine Paulo Affonso
Falls, and above these for a much greater distance. The
Jequitinhonha, 500 miles long, has 84 miles navigable. The most
important river south of the São Francisco is the Parahyba do Sul,
658 miles long, rising in the State of São Paulo and flowing across
Rio. It is navigable from its mouth a distance of 57 miles and 90 in its
upper course.
The Ribeira de Iguapé. The only coast river of economic
importance south of Cape Frio is the Ribeira de Iguapé which rises
on the table-land of Paraná and after receiving several affluents
breaks through the mountains near the boundary of São Paulo.
Besides a navigable channel of 118 miles it communicates with an
inland waterway called the Iguapé or Mar Pequeno, extending many
miles along the coast. In Rio Grande do Sul, where the coastal plain
extends half across the State, several rivers partly navigable flow
into the important Lagôa dos Patos, with which is connected by a
navigable channel, 61 miles long, the Lagôa Mirím.
Lakes. The Lakes of Brazil are few in comparison with the rivers.
Those in the Amazon Basin are reservoirs from the overflow of the
rivers and rise and fall with these. The coastal section has lagoons
and inland channels formed by uplifted beaches; they are usually
shallow and some, as in Bahia, are associated with swamps; but on
the Alagôas coast the lakes are long, narrow, and deep. The largest
coastal lakes are the two in Rio Grande do Sul already mentioned,
separated by broad sand beaches from the ocean, with which they
communicate by a channel 42 miles long at the south end of the
Lagôa dos Patos. This lake is 140 miles long and 30 wide; the Mirím
is 78 miles long and at the most 25 wide. Both are navigable, though
shallow and with many sand bars.
Climate
This great country of Brazil presents considerable diversity of

climate, as already observed. The forest covered Amazon Basin is
hot, with slight variation throughout the year, and with heavy rainfalls;
but while the average temperature is over 80°, ranging from 65° to
95°, some locations are decidedly preferable to others. The regular
rainy season is from November to March, a second of less degree
from August to October; also the rainy season is said to last nine
months. On the upper Amazon there is a short dry season in
January and February. There is a flood time November, December,
and higher water from March to June. The average rainfall is about
78 inches, the rise of the river 45 feet. There are east winds, warm
and moisture laden, the deflected trades, and colder winds from the
west and southwest.
The coast region as far south as Santos is generally warm and
humid (except for a stretch at the northeast), with a wet and dry
season, though it rains also in the latter, except on the arid northeast
coast. A variation occurs in Pernambuco where it rains from March
to August, the dry and cooler season in Rio. Here on the edge of the
tropics the annual temperature is 5° lower than on the Amazon. At
Santos rainfall is heavy and the place is hot, but farther south it is
cooler with a wider range of temperature and a more even
distribution of rain. In Rio Grande do Sul the mercury ranges from
20° to 80°. Cold southwest storms from the Argentine pampa,
occasionally as far as Rio, create discomfort for two or three days at
a time. The highest temperature in Rio is 98.7°.
The Brazilian plateau is very different. As a rule the days are
hotter and the nights cooler, the air drier, than on the coast at the
same latitude. With mean temperature 68° there is occasional frost.
Brazilian Guiana is hot and arid, though with more rain at the east
and west than in the centre. South of the Amazon from Piauhy to
southern Bahia is another semi-arid section with a rainless season
from June to December, when streams are dry and fields are burned
bare. With heavy rains from January to May, the country is covered
with verdure; when these fail, sometimes for successive years, the
droughts are destructive to agriculture and cattle. The plateaus of
Minas, São Paulo, and Rio have a climate which is modified by
luxuriant vegetation, south winds, and their altitude; though Minas
Geraes, having forests only along the rivers and at the south, is
hotter by day, but with always cool nights. The open lands of São
Paulo also have higher daily temperature; the annual mean is
68°-77°. In south Minas and São Paulo frosts occasionally occur. In
the State of Rio there is a delightful climate in the high valleys of the
Serra do Mar, temperature 45°-90°. The table-lands farther south,
Paraná, Santa Catharina, and Rio Grande do Sul enjoy a temperate
climate with abundant rain, occasional frost, but no snow. The
northern valleys of the Paraná River are sub-tropical, the mean
annual temperature of Goyaz City being 77°.
The country over a large extent may be considered healthful; the
yellow fever which once afflicted Rio and some other cities has been
exterminated by rigid sanitary measures, and the conditions of life on
the eastern plateau and in the temperate south are agreeable. The
Amazon Valley is of course malarial, though some sections are far
better than others. It would seem that the lower parts which are
subject to inundation can hardly be made suitable for residence
whatever precautions are taken; but the higher lands may be to

Textbook High Performance Computing in Science and Engineering 16 Transactions of The High Performance Computing Center Stuttgart Hlrs 2016 1St Edition Wolfgang E Nagel Ebook All Chapter PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Textbook High Performance Computing in Science and Engineering 16 Transactions of The High Performance Computing Center Stuttgart Hlrs 2016 1St Edition Wolfgang E Nagel Ebook All Chapter PDF

Uploaded by

Copyright:

Available Formats

High Performance Computing in

Science and Engineering 16

High Performance Computing in Science and Engineering '

High Performance Computing in Science and Engineering

Advances in High Performance Computing: Results of the

High Performance Computing: 34th International

Fair Scheduling in High Performance Computing

High Performance Computing for Geospatial Applications

Parallel and High Performance Computing MEAP V09

High-Performance Computing for Structural Mechanics and

ISBN 978-3-319-47065-8 ISBN 978-3-319-47066-5 (eBook)

Library of Congress Control Number: 2016963434

Mathematics Subject Classification (2010): 65Cxx, 65C99, 68U20

© Springer International Publishing AG 2016

Printed on acid-free paper

This Springer imprint is published by Springer Nature

Nucleon Observables as Probes for Physics Beyond the

Part II Molecules, Interfaces, and Solids

Part III Reactive Flows

Direct Numerical Simulation of Non-premixed Syngas

Part IV Computational Fluid Dynamics

Scalability of OpenFOAM with Large Eddy Simulations and

Part V Transport and Climate

High-Resolution Climate Projections Using the WRF Model on

Part VI Miscellaneous Topics

Volker Springel, Annalisa Pillepich, Rainer Weinberger, Rüdiger Pakmor,

Abstract Cosmological simulations of galaxy formation provide the most powerful

© Springer International Publishing AG 2016 5

generation of hydrodynamical simulations that excel with new physics, enlarged

In principle, simulations of cosmic structure formation are well-specified initial

2 Physics and Code Developments for Illustris++

2.1 New Blackhole Physics Model

As discussed above, we replaced the so-called ‘radio-mode’ of supermassive black

2.2 Hierarchical Time Integration

2.3 Chemical Enrichment Model

2.4 Hydrodynamical Accuracy Improvements

2.5 Elimination of All-to-All Communication Steps

local enrichment region of stellar populations, or the zone of accretion around a

3 Simulation Set and Production Runs

After obtaining access to HORNET/HazelHen, we have first carried out a limited

4 Selected Preliminary Results

In Fig. 1, we illustrate the large-scale distribution of different quantities in the

0.01 0.10 1.00 0.01 0.10 1.00

0.01 0.10 1.00 0.01 0.10 1.00

Another powerful application of our simulations lies in studies of the metal

Understanding the feedback processes in galaxy formation and evolution is the

Yannick M. Bahé, for the C-EAGLE collaboration

Galaxy clusters are collections of large numbers of galaxies – up to several

© Springer International Publishing AG 2016 21

Resources and Industries

Stock raising is by far the leading industry, as is evident from the

Forestry is encouraged so far as planting is concerned; about

Minerals are of some importance and may become of more. The

Naturally manufacturing save for home consumption is of slight

Aside from the development of hydro-electric power and the

Area, Population, Boundary

The first of the South American countries to be discovered after

The government is a federalized republic with the usual branches, the

Education is highly regarded in Brazil. Primary and secondary are free

Brazil presents in physical characteristics more variety than is

The Coastal Belt

The coastal section is largely a low-lying sandy tract, varied by

The Guiana Highlands