Professional Documents
Culture Documents
EMCEurope2014
EMCEurope2014
EMCEurope2014
net/publication/271263872
CITATIONS READS
3 198
3 authors, including:
Frank Sabath
Bundeswehr Research Institute for Protective Technologies and CBRN Protection
184 PUBLICATIONS 972 CITATIONS
SEE PROFILE
All content following this page was uploaded by Frank Sabath on 10 December 2016.
Abstract—During the last two decades scientific computing has the emergence of high performance computing. Scientific
been developed as an additional form of science and engineering. computing consists of mathematical modelling, computing
Scientific computing links theory and laboratory experiment in a and numerical approximation and scientific discipline.
kind of theory based virtual testing environment. The approach
is to gain understanding based on computer implemented mathe-
matical models. Common quality assurance systems like ISO 9001 Scientific computing changed the common qualification
demand validation and verification during product development needs in engineering to a focus on design engineering and
and manufacturing. Digital MockUp (DMU) based computer numerical modelling in the corresponding discipline, here in
aided engineering (CAE) is the main computational engineering
service in industrial applications. computational electromagnetics (CEM).
A verification and validation of the computed results is usu- The basis of all scientific and engineering work is confidence
ally not possible, because CAE is used as an alternative to in the originated results. Achieving confidence in theoretical
measurements or the device under test is not measurable or results is possible through mathematical proofs. This is not
does not exist in the early stage of design. For that reason a possible in the case of experimental work. An accreditation as
validation and verification of the CAE based engineering service
is necessary. An approach for quality assurance in computational a testing lab in accordance with ISO 17025 [10] is confidence-
electromagnetics is given in this paper. building and the common way for laboratory results. Scientific
Numerical analysis became a standard procedure in engineering computing is based on mathematical models. Most CEM
and science. Achieving a traceable and confidence-building proof methods are deterministic. Calibration of mathematics and
of quality becomes more important in industry-sector-specific deterministic system seems meaningless, because numerical
computational engineering.
results do not depend on calibration checks of soft- or hard-
Index Terms—Quality Assurance; Computational Electromag- ware. There are fundamentally different issues between virtual
netics; Quality Management; Numerical Computation and laboratory testing in order to achieve high accuracy which
I. I NTRODUCTION is the basis of high confidence.
Traceability to international norms in order to proof confidence
Science and engineering has changed over the last decades.
in the methods of scientific computing and the resulting
Numerical computations allow a description of physical
service is the aim of this paper. CEM is used as an virtual
phenomena based upon mathematical methods and theoretical
testing environment to describe complex real-world behaviour
assumptions. This yields an virtual test environment which
of electromagnetic systems, but it must not be understood as
uses Digital MockUp and allows a mapping between theory
a virtual lab. Assuming that a “testing laboratory approach” is
and experiment. This scientific analysis approach is often
not possible a different solution is needed. In order to classify
called ’Scientific Computing’ (cp. 1).
the work of simulation analysts and their results there are some
fundamental questions:
• Is the exclusive inspection of the personnel competence
sufficient?
• Is the exclusive validation and verification of the software
sufficient?
• What kind of quality management system is needed?
• Is a software certification needed?
An understanding of CEM as a service (cp. [6]) is proposed in
this paper. The advantage of this approach is the consideration
of the personnel qualification and the software as a tool.
Fig. 1. Simulation Results Therefore it is possible to verify and validate the simulation
software as a black box. This functional testing ignores
The methods of scientific computing are well developed the internal mechanism of simulation software and focuses
and still under development. Much of these methods became solely on the numerical results generated in response to the
possible due to the development in computer technology and excitation depending to boundary conditions. For that reason
the sufficiency of software-tools may be proofed without identical in both QM-norms. Figure 3 shows the interaction
the need of certification. This is important, because most between ISO 9001 and NAFEMS QSS 001 exemplary for the
numerical packages are closed source without a sufficient ’Ressource Management’. General requirements are made by
documentation of the exact implementation. Furthermore the
simulation analysts environment and other requirements with
respect to ISO 9000 series [3] have to be considered. Based
on this QM-system a service certification scheme is proposed
here.
The paper is organised as follows: Section 2 gives a re-
view of ISO/IEC 9000 “Quality Management System” and
the NAFEMS QSS 001 supplement as and augmented QM-
system. A Validation & Verification (V& V) scheme for com-
putational results is given in section 3. A resulting certification
scheme based on ISO 17000 [6] is proposed in section 4.
Section 5 provides a conclusion and an outlook on future work.
The used terminology and vocabulary refers to [2], [6], [21].
Fig. 3. Augmentation of ISO 9000 by NAFEMS QSS 001
II. Q UALTIY M ANAGEMENT IN CEM
ISO 9001, whereas NAFEMS QSS 001 specifies this general
The achievement of implemented international QM stan- formulation with respect to scientifc computing needs. In ISO
dards is the demonstration of technical competence and the 9001 [3, 6.2.2] the following requirements are given:
ability of consistence in generating technically valid results. “The organization shall:
A QM system ensures that quality principles are built into the
a) determine the necessary competence for per-
simulation design process at a fundamental level. This facil-
sonnel performing work affecting conformity
itates quality control in all aspects of the simulation process
to product requirements,
(modelling, meshing, computing, validation, etc.) enabling
b) where applicable, provide training or take other
consistent results on a professional standard. Figure 2 shows
actions to achieve the necessary competence,
the interaction between ISO 9000 series norms and NAFEMS
c) evaluate the effectiveness of the actions taken,
QSS 001. The terminology is defined in [2], the requirements
d) ensure that its personnel are aware of the
are given by [3] and a guideline to a total quality system is
relevance and importance of their activities and
given by [4]. ISO 9004 is a kind of design guideline for a QM-
how they contribute to the achievement of the
system without own requirements, it gives recommendations
quality objectives, and
for quality culture for organisations. NAFEMS QSS 001 is a
e) maintain appropriate records of education,
QM-system supplement for scientific computing. It conforms
training, skills and experience.”
to ISO 9001 and elaborates augmented requirements. ISO 9001
These requirements are not specific enough to describe
specifies general requirements, whereas NAFEMS QSS 001
the competence needs of computational analysts. One of the
specifies requirements particular to scientific computing. The
specifications is given in NAFEMS QSS 001 [24, 6.2.2.2]:
specifications aim at qualification of personnel, software, and
category product risks etc. In this manner the NAFEMS QSS “Analysts must have an understanding of the
assumptions inherent in the computational methods
used, and the relationship between the simulation
and the engineering application. Analysts shall have
adequate expertise in the computational methods,
and an understanding of software and hardware
system employed, especially with regard to its lim-
itations.”
A comparison of both requirements provides an idea about
the fundamental need for the use of NAFEMS for quality
assurance in computational electromagnetics or in scientific
computing in general.
Nowadays modern and high performance simulation pack-
Fig. 2. Supplement of ISO 9000 Series ages have powerful graphical post-processing and persuasively
present the computed results. This results, when used by
001 norm augments the whole ISO 9000 family with additional untrained personnel, in a tendency to assume correct results
requirements for the needs of scientific computing. Therefore and a lack of validation. Furthermore there is a tendency
ISO 9000 [2] and ISO 9004 [4] are still valid and usable to over-specify the DMU approximation as a computational
for a NAFEMS based QM-system. The chapter taxonomy is model. This is also an issue due to the analysts lack of
241
Proc. of the 2014 International Symposium on Electromagnetic Compatibility (EMC Europe 2014), Gothenburg, Sweden, September 1-4, 2014
skills. Operating a specified quality management system will The post-processing is the phase of evaluation and assessment
minimise this issue and allows computing under controlled of the computed results. A plausibility check with respect
condition and demonstrates reliability in the numerical results. to the initial assumptions must be done. This demands a
high understanding of the engineering problem and of the
III. CEM D ESIGN P ROZESS & P ITFALLS numerical method used.
A computational engineering design process requires the
following initial informations:
TABLE I
• a clear and accurate definition of the problem, E RRORS IN S CIENTIFIC C OMPUTING
• prediction about the anticipated physical behaviour of the
device system, Error Source
• assessment criteria for the results. Analyst Software Usage
Figure 4 shows the main steps in computational Pre- fringe elements no element check
electromagnetics. The definition of the numerical model Processor unexact material data corrupt meshing
corrupt idealisation
demands an accurate understanding and definition of the Processor wrong boundary conditions ignored warnings
problem. This is needed in order to get the right approximation wrong solver imprecise solver
Post scaling wrong smoothing
Prozcessor wrong choice wrong presentation
242
Proc. of the 2014 International Symposium on Electromagnetic Compatibility (EMC Europe 2014), Gothenburg, Sweden, September 1-4, 2014
measured physical phenomena. with two independent numerical methods. The International
Numerical analysis involves three different types of models. Compumag Society presents the well known TEAM problem
The conceptual model is an abstraction of the real-world. [36]. The IEEE standard [22] gives an overview on the
The conceptual model describes the physical phenomenon examples from the Electromagnetic Code Consortium. In order
of interest of the device under test (DuT), like antenna to increase the number of usable examples more cases are
performance, radar cross section (RCS), electromagnetic actually under construction.
compatibility (EMC) etc. All electromagnetic assumptions In order to ensure correctness in computing, the analyst must
and description about the device under test (DuT) are not know the real solution when computing numerical results.
considered in this level of modelling. The next model in the
sequence is the mathematical model. This model describes C. An Example - Mie Series
the scientific problem with equations like the wave equation The Mie-series is an analytical solution to Maxwell’s equa-
of electromagnetic propagation. After the implementation of tions and describes the scattering of electromagnetic radiation
this mathematical model the resulting code becomes under by a sphere. It is the most simple radar-cross section example.
consideration with discretization and physical parameters This section presents the functionality of the FSV tool with
of the computational model. The first verification variant is an exact numerical solution and a erroneus variant. It aims to
called “Code Verification”. Here the computational code is present the grading sensitivity of the FSV routine.
verified with respect to its mathematical background. This Figure 5 shows the Mie-scattering for the analytical, exact
verification is done by software manufacturers, as in most numerical and erroneus numerical solution.
cases the source code is not accessible. The most popular
method is to compare code results with analytical solutions.
“Calculation Verification” estimates the computational errors
due to discretization. One of the most popular methods is the
estimation of the amount of error on different discretizations
i.e. finer or coarser meshes. The responsibility on “Calculation
Verification” lies with the simulation analyst, because it is
not possible for the code developer to assure correct user-
developed meshes (cp. [26], [27]).
Both code and calculation verification may be important
while operating CEM software and should be considered and
documented in a QM-system. In particular if self-developed
code is used. Fig. 5. Mie Scattering
243
Proc. of the 2014 International Symposium on Electromagnetic Compatibility (EMC Europe 2014), Gothenburg, Sweden, September 1-4, 2014
difference of both classes of methods should be considered in putational Fluiddynamics yields lots of analogies, especially
the definition confidence levels. This paper aim at the exact the mathematical background of the methods used. For that
methods. reason quality assurance strategies like NAFEMS may be used
The FSV routine uses a decomposition of the results and analogously. CEM is one of the latest numerical engineer-
compares two measures. After this a recombination of the ing disciplines which develops formalised quality assurance.
results is done in order to compute a “global goodness of Wide parts of quality assurance were done from the earliest
fit measure”. The comparison of amplitudes and “trends” beginning, but there is no formalised, traceable, and universal
yields the Amplitude Difference Measure (ADM). The rapidly procedure for the whole CEM service. A service certification
changing features result in the Feature Difference Measure scheme arises from a combination of ISO, NAFEMS and
(FDM). A combination of ADM and FDM yields the Global IEEE.
Difference Measure (GDM). The ADM, FDM and GDM are
usable as a point-by-point analysis algorithms or as a single A. Fundamentals on ISO 17065
overall measurement. Figure 6, 7 and 8 depicts the measures The overall aim of certifying numerical analyst services is
respecitevely. to give confidence in the numerical results. The value of certi-
fication is the high level of confidence that is established by an
impartial certification body (CB) and demonstrates competend
fulfillment of third-party requirements. Possible parties that
have an interest in CEM-service certifcation are govermental
authorities, non govermental organisations, consumers, CEM-
service customers, clients of CBs, etc.
244
Proc. of the 2014 International Symposium on Electromagnetic Compatibility (EMC Europe 2014), Gothenburg, Sweden, September 1-4, 2014
245
View publication stats