Ref.

Ares(2019)7922089 - 27/12/2019

D4.3: MOD EL SELECTION

Work Package WP4 (D4.3)

Lead Authors (Org) S. Belouettar (LIST) & Carlos Kavka (ESTECO)
Contributing Author(s) (Org) Ahmed Makradi (LIST)
Reviewers (Org) Ali Daouadji (INSA) and Ahmed Makradi (LIST)
Due Date 31-12-2019
Date 27-12-2019
Version Final version

Dissemination level

PU: Public X
PP: Restricted to other programme participants
RE: Restricted to a group specified by the consortium
CO: Confidential, only for members of the consortium

D4.2: Model Selection 1

Versioning and contribution history

Version Date Author Notes

v1.0 06-12-2019 S. Belouettar Initial draft

v1.1 26-12-2019 C. Kavka Corrected draft
V1.2 27-12-2019 Daoaudji A. Review

V1.3 27-12-2019 Ahmed Makradi Review

V1.4 28-12-2019 S. Belouettar Final Version

Disclaimer:
This document’s contents are not intended to replace consultation of any applicable legal
sources or the necessary advice of a legal expert, where appropriate. All information in
this document is provided “as is” and no guarantee or warranty is given that the information
is fit for any particular purpose. The user, therefore, uses the information at its
sole risk and liability. For the avoidance of all doubts, the European Commission has no
liability in respect of this document, which is merely representing the authors’ view.

D4.2: Model Selection 2



TABLE OF CONTENTS

Versioning and contribution history .................................................................................................................. 2
Disclaimer: ......................................................................................................................................................... 2
About COMPOSELECTOR ................................................................................................................................... 5
Task and Deliverable descriptions from the project proposal .......................................................................... 5
Compliance with the work-programme NMBP-23-2016 ................................................................................... 6
Executive Summary ........................................................................................................................................... 6
Introduction ...................................................................................................................................................... 6
Terminology related to Modelling and Simulation ........................................................................................... 7
Modelling Strategy Selection ............................................................................................................................ 8
Workflow Construction .............................................................................................................................. 9
Workflow Selection .................................................................................................................................. 10
Technical KPIs for Model Selection ................................................................................................................. 11
Model Robustness .................................................................................................................................... 12
Sensitivity ................................................................................................................................................. 12
Model Complexity .................................................................................................................................... 13
Model Uncertainty ................................................................................................................................... 14
Model selection using graph theory ................................................................................................................ 16
Business KPIs for model selection process ...................................................................................................... 18
Modelling workflow selection with DMN. ....................................................................................................... 19
The advantages of modelling business decisions with DMN ................................................................... 19
A general operative example in material modelling ................................................................................ 21
WORKLFOW SELECTION IN END USER’S APPLICATIONS .......................................................................... 23
Conclusions ..................................................................................................................................................... 25

D4.2: Model Selection 3

LIST of Figures and Tables

Figure 1: Top level architecture of BDSS platform used in COMPOSELECTOR project and its components ..... 5
Figure 2: Material model (PE and MR) as defined by the EMMC. ..................................................................... 7
Figure 3: The model selection is part of the translation process taking into account validation and
verification of models. The determination of the models, or “model selection” process is driven by the
the analysis of the user case by a translator based on experience (translation) and on available
validation and verification data. ................................................................................................................ 9
Figure 4: Business decision should influence how the modelling should be done based on cost, time, fidelity,
for example. ............................................................................................................................................... 9
Figure 5: Each edge in the graph represents a coupling between two partial models at two different scales.
A parameter based on the model complexity and number of input parameters obtained and used for
hierarchal model definition and selection. .............................................................................................. 11
Figure 6: Technical KPIs for model selection: Robustness, Sensitivity, Complexity and Uncertainty ............. 12
Figure 7: Based on the sensitivity indices, it is possible to determine the most important uncertain input-
parameters, their correlation and to quantify their impact on the predicted output (Choi-2005). ........ 13
Figure 8: Model complexity and data needs. Less complex models are used for the trends analysis where
more complex model would be used for very specific analysis. The factors that measure model and
software complexities are: ....................................................................................................................... 13
Figure 9. Definition (selection) of Model Granularity/Complexity. M1: number of the individual models. M2:
number of the connections among the workflow. M3: number of attributes (parameters). M4:
indicates the number of the manually changed attributes. ..................................................................... 14
Figure 10: Computational complexity determine the required time duration to fulfil a simulation run.
Attributes for computational complexity could include i) the l algorithmic complexity of the model and
ii) the total number of lines in all the program blocks ............................................................................. 14
Figure 11: Computational framework: Non-deterministic. ............................................................................. 15
Figure 12: Work flow with ModeFrontier: All inputs and outputs are connected with APIs Design variable is
defined as input Objective and constraints are connected to outputs Optimizer is selected Run the
optimization process ................................................................................................................................ 15
Figure 13: A sketch of a graph ......................................................................................................................... 16
Figure 14: The tool developed for mode selection. The source codes are available on Project GitHub folder.
................................................................................................................................................................. 18
Figure 15. Business Attributes and KPIs for model selection .......................................................................... 18
Figure 16: Decision Modelling and Notations (DMN) process ........................................................................ 19
Figure 17: A process where decision logic is modeled with process flow elements ....................................... 20
Figure 18: A process where decision logic is implemented in a DMN decision task ....................................... 20
Figure 19: A decision table for the simulation location example .................................................................... 21
Figure 20: DMN decision table for model selection ........................................................................................ 21
Figure 21: Business process for model selection between atomistic and mesoscopic ................................... 23
Figure 22: Workflow selection by using a DMN decision table ....................................................................... 23
Figure 23: DMN table ...................................................................................................................................... 24
Figure 24: The enhanced DMN table with default output. ............................................................................. 25
Figure 25: Workflow selection ........................................................................................................................ 25

D4.2: Model Selection 4

About COMPOSELECTOR
The mission of COMPOSELECTOR is to develop a Business Decision Support System (BDSS), which
integrates materials modelling, business tools and databases into a single end-user’s workflow to
support the complex decision process involved in the selection and design of polymer-matrix
composites1.

Material Simulation
Layer Layer

Figure 1: Top level architecture of BDSS platform used in COMPOSELECTOR project and its
components
An overview of the Composelector BDSS platform is presented in Figure 1. The Business Layer
(BPMN based tool) interacts with Material Layer (database and workflow manager system). The
Simulation Layer (MuPIF) provides an infrastructure for defining and executing distributed
simulation workflows, consisting of several linked/coupled models designed for defined Key
Performance Indicators (KPIs). The individual models are connected to the platform by
implementing standardized APIs, allowing for model steering, data/metadata exchange, distributed
and remote computation, monitoring, etc. The metadata structure is defined by a schema where
the metadata can be attached to any component and validated against this schema. Since the
metadata are encapsulated in the individual components, they can be passed together with the data
in one consistent package. This novel development is also key for the management of the metadata
linked to modelling and simulation tasks.

Task and Deliverable descriptions from the project proposal

WP definition: In the WP4, an optimisation approach will be implemented in a goal-oriented
manner to satisfy the essential industrial and market requirements. The material selection process
will be performed via several mappings among the desired performances of the final component,
the manufacturing process inputs, the component configuration, cost, and industrial requirements.
A process selection methodology integrating materials modelling, optimisation and control will be
developed. The multidisciplinary evaluation/selection will consider the production, use and end-of-
life of the composite material, as well as explore their potential future market penetration among
other constraints in order to develop relevant scenarios.

1
https://doi.org/10.1016/j.compstruct.2018.06.121

D4.2: Model Selection 5

Task 4.2-Model Selection: Based on the sensitivity measures and model complexity, we will provide
an approach based on graph theory for the model selection based on graph theory. The
computation of the parameter at the node of the graph could be obtained using a BNC mapping
considering the model complexity and number of input parameters. The underlying material model
consists of many partial models. The partial model will be selected from a large range of nonlinear,
time-dependent, multi-physics behaviour and data driven models of PMCs, which may vary from
group to group. Particularly, model error estimates can be used to drive an automatic multiscale
model selection. Moreover, in the case of top-down approaches, goal oriented error estimators can
be used to automatically select the local features that need to be analysed at a finer-scale. For a
bottom-up approach, the model quality and also the quality of the partial models will be assessed
by a framework for model assessment. Based on the sensitivity measures, the model robustness
and model complexity, we will provide an approach based on graph theory for the model selection.
Investigated partial models are those reported in Task 4.1. M12-M36.
D4.3: Full report on model selection, LIST: This deliverable presents the work made on model
selection for modelling, simulation and business decision making.

Compliance with the work-programme NMBP-23-2016

Selecting the appropriate model is of key importance for a reliable business decision process. As a
matter of fact, too simplistic model may fail to capture much of the useful information available and
an overly complex model may describe many irrelevant details. For example, “which atomistic,
electronic or mesoscopic models are available?”, and “which can be further developed to augment
and integrate with other existing solutions?”. Furthermore, there is a lack of systematic information
regarding the validity, i.e., the limits of applicability of a particular method to a particular problem.
The definition of the KPIs and the specific process that defines the model selection and modelling
workflows (including simulation tools) needs to be specified.

Executive Summary
In this deliverable, we describe the developed approach and tool for model selection. The idea
proposed in COMPOSELECTOR is to consider graph theory and Decision Model and Notations (DMN)
standard to implement a practical model selection in the BDSS. Indeed, using DMN, decisions can
be represented in a business process just as a sequence of logical steps, which guide the execution
based on KPIs and input values.

Introduction
Selecting the appropriate model is of key importance for a reliable decision process. A good model
should capture all necessary effects and should contain as few input parameters as possible. As a
matter of fact, too simplistic model may fail to capture much of the useful information available and
an overly complex model may describe many irrelevant details. In addition, the large number of
existing material models makes it difficult to select the most relevant one. In other words, which
atomistic, electronic or mesoscopic models are available, and which model is appropriate.
Particularly, the following features should be analyzed and answered: i) matching modelling
strategies (predictive capability of a Model) with business decision needs; ii) translation of process
modelling value (cost/benefits), cost and triggers (part complexity/size/materials/process) iii)
evaluation of optimization strategies for trade-offs in decisions and effect of model resolution
(uncertainty) on business decisions: Select a model for decision making.

D4.2: Model Selection 6

Terminology related to Modelling and Simulation
This section reports common terminology accepted by industrial end-users of continuum modelling
but is not meant to be exhaustive.
Material Model11 is the combination of an approximated Physics/chemistry Equation (PE)
describing generic, widely accepted physics and its Materials Relations (MR) describing a specific
material and its behavior (Figure 2). Neither the PE nor MR can be solved in isolation, but it is the
application of the PE to a specific case ‘documented’ by the Materials Relation which can be solved.
MRs include the constitutive equation used in continuum models. When a new MR is found, the
combination PE + MR is called a new model, or Governing Equations. For a full explanation of PE +
MR with examples refer to the RoMM11. Note that ‘model’ here refers to the combination of PE
and MR. In some cases, the selection described below only is across different MR for the same PE,
in other cases it may also include different PE. In particular, the model error estimates described
previously can be used to drive an automatic model selection. Moreover, in the case of top-down
approaches, goal oriented error estimators can be used to automatically select the local features
that need to be analyzed at a finer-scale. For a bottom-up approach, the model quality and also the
quality of the partial models will be assessed by a framework for model assessment.

PHYSICS BASED MODEL

PHYSICS EQUATION MATERIAL RELATIONS

PE MR
Equation based on a physics/chemistry PHYSICS Information on the material needed to
theory which describes the spatial and close the PE and to make the system of
QUANTITIES
temporal evolution of physics Governing Equations solvable
quantities of the entity

Figure 2: Material model (PE and MR) as defined by the EMMC.

Simulation is the ensemble of models—deterministic, load, boundary, material, performance, and

uncertainty—that are exercised to produce a simulation outcome. Simulation is the act of
modelling, or the execution of the model on a user case via a numerical code [ROMM]. The
Simulation output is the result generated by the computer code (or multiple codes) that reflects
both the deterministic and nondeterministic response of the model.
Prediction is the use of simulation models to forecast what occurs under conditions for which there
are no relevant experiment data. Predictions come with errors and uncertainties because of
limitations in numerical accuracy, approximations and deficiencies in the model, uncertain initial
and boundary conditions, and uncertainties in the system and /or the environment.
Verification is the process of determining that a model numerical implementation accurately
represents the developer’s conceptual description of the model and the solution to the model.
Validation is the process of determining the degree to which a model (approximation of the physics)
is an accurate representation of the real world from the perspective of the intended uses of the
model,
Uncertainty Quantification (UQ) is the process of characterizing all uncertainties in the model and
experiment and quantifying their effect on the simulation and experimental outcomes. There are
established methods (e.g. nondeterministic methods) for UQ.
Accuracy is the degree to which the result of a measurement, calculation, or specification conforms
D4.2: Model Selection 7
to the correct or specified value or a standard. See also Validation Metric. Specifically, Fidelity is a
term often used to indicate the difference between simulation and experimental outcomes.
Precision is the closeness of the measurements to each other. It may also refer to the refinement in
a measurement, calculation, or specification, especially as represented by the number of digits
given.
Error is a recognizable deficiency in any phase or activity of modelling and simulation that is not due
to lack of knowledge. Margin of Error refers to the accuracy of the model in percent as specified in
the post-processing reporting requirements of the RoMM.
Convergence (Stability) is the successive refinement of simulation parameters, e.g. mesh, time step,
size of simulation box etc., until a target level of accuracy is obtained..

Modelling Strategy Selection

We use a "modelling strategy selection" instead of "model selection". The reason is that individual
models may have different dependencies, while strategies can have dependencies more or less
defined. In addition, the following statements and remarks should be added:
§ The business decision will influence how the modelling should be done based on cost, time,
fidelity, etc. The Translator creates a description of a user case. This is done using the
information provided by the end-user.
§ The global simulations workflows will be decomposed into individual tasks (determination
of single KPI, process simulation, etc.). These tasks together with their input/output
requirements will be identified for each case study.
§ Alternative implementation of individual tasks (represented by so called task workflows)
with different performance indicators (time requirements, cost, fidelity of results, etc.) will
be implemented. The availability of alternative task workflows (yielding the specific action)
would allow the BDSS to explore different modelling options based on different business
and/or technical requirements (performance indicators). This single workflow will allow
accounting for the complete production chain involving all the major processes and, at the
same time, all possible parameters affecting the costs and other important factors under
consideration.
§ Determine all possible model performance indicators that are of relevance and importance
to the whole process. This is done independently of the BDSS end user requirements, as it
must address the technical attributes as well, that pertain to the materials, process and
possible candidate models that can be utilised.

D4.2: Model Selection 8



Figure 3: The model2 selection is part of the translation process taking into account validation and
verification of models. The determination of the models, or “model selection” process is driven by
the the analysis of the user case by a translator based on experience (translation) and on available
validation and verification data.

Identify potentially
needed tasks
BDSS

Select Task Workflow Select Task Workflow

based on performance Based on perf. criteria
criteria

Decide if
additional tasks
to be analyzed

Task workflow 1.1

Task workflow 2.1
Task workflow 1.2
MuPIF

Task workflow 2.2

Task workflow 1.3

Figure 4: Business decision should influence how the modelling should be done based on cost,
time, fidelity, for example.

Workflow Construction
Typically, a modelling workflow is composed of a number of predefined models. Figure 4 illustrates
the material modelling strategy. At the top level, the properties acting as key performance indicators
2 Model means both the PE and the MR, which is related to specific applications and materials choice. Furthermore,
the choice of the PE is related to the required granularity level, i.e., whether one needs to describe the system on
electronic, atomistic, particles, or continuum levels.

D4.2: Model Selection 9

can be modelled using models with varying physical complexity. Thus, a choice has to be made
among the several models enabling the determination of one or more KPIs. The material input
parameters required for this top modelling layer will either be provided by modelling at a lower
scale, or by databases fed by literature or experiments. At this scale, databases or several modelling
techniques and models with varying levels of complexity may be used for obtaining one or more
material parameters required by the modelling level above. For example, some material properties
may be obtained by a simple micromechanical model of the composite structure, or by more
complex modelling of the production process like thermoforming or resin transfer moulding, for
example. The material parameters required for modelling at this scale may then be obtained by
more or less complex models, for example molecular dynamics, coarse-grained models, etc., or by
database lookup. The number of modelling layers may be increased by including layers with more
refined methods, like atomistic models or others. It should be noted that the last scale of modelling
will entirely depend on material or structure properties obtained from database lookup. As
illustrated in Figure 5, at each level, various models exist for obtaining the required input for the
next levels, or the KPI at the top level, respectively. This choice has to be made by considerations of
availability, computing time and cost, as well as the precision required in the application case. At
each scale, there can be a choice of several models with diverse qualities and sensitivities.
Considering that data, at a given scale, are only known to a certain degree (uncertainties), we can
afford to choose less accurate models that are faster to evaluate, provided that the resulting error
is of the order of the global uncertainties. In order to formalize this idea, key quantities will be
considered uncertain at the finest scale, where the material is described, and at the coarsest scale,
where the engineering problem is described. The chain of models will be allowed to transport these
uncertainties through the scales, using robust uncertainty quantification or probabilistic analysis by
sampling methods.

Workflow Selection
Each workflow should define its performance indicators (as workflow metadata). The task
workflows should have some general common inputs-outputs defined. The individual workflows will
depend on additional parameters-properties, which can be obtained by simulation or from
database, however this logic must be implemented into specific workflow. This way the BDSS is able
to freely combine different alternatives without being concerned by i/o dependencies. In parallel, a
business decision mechanism based on a balance between investment (complexity and number of
inputs) and return will be implemented to decide on the type of models (in a chain) namely
electronic, atomistic, mesoscopic and/or continuum

D4.2: Model Selection 10

 P1 P2 Pi Pn

Level n
Model Selec1on Strategy
DB
P: Property
Mn.1 Mn.2 Mn.i
M: Model
DB: Database

Level i

Level 2

M2.1 M2.2 M2.i DB

Level 1

M2.1 M2.2 M2.i

DB
Figure 5: Each edge in the graph represents a coupling between two partial models at two
different scales. A parameter based on the model complexity and number of input parameters
obtained and used for hierarchal model definition and selection.

Technical KPIs for Model Selection

KPI’s are used in COMPOSELECTOT primarily as measurable and quantifiable indicators for the
specific stakeholder enabling them to take actionable decisions in a business context. The decision
regarding model selection might include theoretical and computational considerations. Once the
KPI’s are determined, specific Uncertainty Estimates (UE) limits or requirements to each KPIs are
provided addressing the end user application requirements (which include aping the end user

D4.2: Model Selection 11

requirements to KPIs). This then enables determining the model granularity (e/a/m/c) and choice of
materials relations and limits of accuracy on each KPI. These KPIs determine the required granularity
of the model and the set of data used/needed to create a model workflow (Figure 6).

• User cases KPIs and • User cases &

Physics KPIs

DMN
Robustness Uncertainty

DECISION TABLES
KPIs
THE TRANSLATOR
Complexity Sensitivity
• Error and
• Model and Software Uncertainty
complexity

Figure 6: Technical KPIs for model selection: Robustness, Sensitivity, Complexity and Uncertainty

Model Robustness
The robustness is defined as the degree that model is physics based. This represents the capability
of the model to predict the physical properties and represent the physical reality. The following
attributes could be used to “estimate” qualitatively the robustness of a model:
§ The degree to which the model is calibrated: The number of modelling assumptions that can
be relaxed without overturning the conclusions (outputs)
§ The number of modelling assumptions that can be relaxed without overturning the
conclusions (outputs)
§ Documented verification and experimental validations of the model: the physics fidelity
basis with which the model is being extrapolated from its validation and calibration database
to the conditions of the application of interest. The Translator and Modeler have then to
refer to available literature, data bases and on own experience to give an estimate of the
model robustness.

Sensitivity
Based on the sensitivity analysis (estimation of sensitivity indices), it is possible to determine the
most important uncertain input-parameters, their correlation and to quantify their impact on the
predicted output (Figure 7).

D4.2: Model Selection 12


Figure 7: Based on the sensitivity indices, it is possible to determine the most important uncertain
input-parameters, their correlation and to quantify their impact on the predicted output (Choi-
2005).

The following information for model selection could be obtained from sensitivity analysis:
§ Identification of trends in the model response
§ Qualitative or quantitative ranking of inputs
§ Discrimination of the importance among different inputs
§ Grouping of inputs that are of comparable importance
§ Identification of inputs that are not important
§ Identification of critical limits
§ Identification of inputs and ranges that produce high exposure or risk

Model Complexity
The complexity of model might combine both model and software implementation complexities
(Figure 8 and Figure 9). The following attributes define the complexity of a model:
§ Structure and the level of detail in the model workflow.
§ The number of parameters and state variables
§ The sophistication of the mathematical relationships that describe each process
§ Number of processes in the model
Model complexity measure.
Software complexity measure.

Less Complex Complexity level More Complex

Less data intensive More intensive
Less sensitive More sensitive
Trends only Specific analysis

Figure 8: Model complexity and data needs. Less complex models are used for the trends analysis
where more complex model would be used for very specific analysis. The factors that measure

D4.2: Model Selection 13

model and software complexities are:
§ Factor 1: Structure and the level of detail in the model workflow (Figure 9).
§ Factor 2: The number of parameters and state variables
§ Factor 3: The sophistication of the mathematical relationships that describe each process
§ Factor 4: Number of processes in the model

Model MQ1 Model MN1

Model MQ2 Model MN2 Model MS2

Model MS3

Figure 9. Definition (selection) of Model Granularity/Complexity. M1: number of the individual
models. M2: number of the connections among the workflow. M3: number of attributes
(parameters). M4: indicates the number of the manually changed attributes.

Figure 10: Computational complexity determine the required time duration to fulfil a simulation
run. Attributes for computational complexity could include i) the l algorithmic complexity of the
model and ii) the total number of lines in all the program blocks

Model Uncertainty
Uncertainties are associated to the selection of the models and to the input parameters (operational
and geometrical) of the physical model considered for analysis. For example, in computational fluid
modelling, the physical properties of the fluid or the boundary conditions may not be known exactly.
Geometrical uncertainties mainly arise due to manufacturing tolerances and imprecise geometrical
D4.2: Model Selection 14
definitions in the model. The uncertainties associated to the system of consideration can have
different origins and can be classified as epistemic and aleatory. The epistemic uncertainties are
related to the properties of the model considered. While, the aleatory uncertainties are related to
the properties of the system being considered for the analysis. Epistemic uncertainties are due to
any lack of information in any phase of the modelling process and can be reduced as updated
information is acquired. These uncertainties may arise from mathematical assumptions or
simplifications used in the derivation process of the governing equations. The parameters with
epistemic uncertainty are not associated to any probability density functions and are not well
characterized by stochastic approaches. Typically, they are specified as an interval. In uncertainty
quantification of epistemic uncertainty, the output result is also specified as an interval. Epistemic
uncertainties can be quantified using sampling-based methods (such as random sampling, Latin
hypercube sampling, interval analysis, fuzzy set theory, etc). Aleatory uncertainties are associated
with the physical system or the environment under consideration. Material properties, operating
conditions, geometrical variability, initial conditions, boundary conditions etc. are some of the
examples of aleatory uncertainties. Uncertain parameters of aleatory uncertainty are always
associated to a particular probability distribution function. Using appropriate uncertainty
quantification (UQ) methods, the probability distribution function of the output quantity can be
determined (Figure 11).

Figure 11: Computational framework: Non-deterministic.

A unique set of methods are required to quantify each form of uncertainty. Epistemic uncertainties
are considered as reducible uncertainties. They can be reduced by research advancement. However,
aleatory uncertainties are unavoidable.

Figure 12: Work flow with ModeFrontier: All inputs and outputs are connected with APIs Design
variable is defined as input Objective and constraints are connected to outputs Optimizer is
selected Run the optimization process

D4.2: Model Selection 15

Model selection using graph theory
Graphs are powerful abstractions for capturing complex relationships in diverse application settings
(Figure 13). The idea proposed in COMPOSELECTOR is to consider various partial models as nodes
in a graph and a combination of partial models as a path in the graph. Each edge in the graph
represents a coupling between two partial models at different scales within the bottom-up
approach indicated by the unidirectional arrows. The final model quality is a combination of the
qualities and sensitivities of the partial models. Based on the sensitivity measures, the model
robustness and model complexity, we will provide an approach based on graph theory for the model
selection.

Figure 13: A sketch of a graph

• Based on the sensitivity measures, the model robustness and model complexity, we provide
an approach for the model selection.
• The idea is to consider various partial models as nodes in a graph and a combination of partial
models as a path in the graph.
• Each edge in the graph represents a coupling between two partial models at different scales
and/or processing options within the bottom-up approach indicated by the unidirectional
arrows.
• The chain of models will be allowed to transport these uncertainties through the scales, using
robust uncertainty quantification or probabilistic analysis by sampling methods.
• The final model quality is a combination of the qualities and sensitivities of the partial models.
It is important to notice that the data of the problem are only known to a certain degree. In the
best case, we have some knowledge of the uncertainties that we need to take into account (fine
scale features of the structure, material parameters).
• The uncertainties are the key drivers of model adaptivity. Indeed, the accuracy that we require
in our chain of models does not need to exceed the accuracy in the data. In other words, we can
afford to choose less accurate models and that are faster to evaluate, provided that the resulting
error is of the order of the global uncertainties
• we can afford to choose less accurate models and that are faster to evaluate, provided that the

D4.2: Model Selection 16

 resulting error is of the order of the global uncertainties. In order to formalize this idea, key
quantities will be considered uncertain at the finest scale, where the material is described, and
at the coarsest scale, where the engineering problem is described
• A decision tables is constructed based on robustness, sensitivity and complexity. A weight (0 <
W < 1) is assigned based on the quality (i.e. robustness, sensitivity, complexity) of the individual
partial models.

D4.2: Model Selection 17


Figure 14: The tool developed for mode selection. The source codes are available on Project
GitHub folder.

Business KPIs for model selection process

Creating a decision-making strategy for model selection starts with the identification of appropriate
business needs or objectives for a given use case and relating these objectives to KPIs (performance
indicators that have a target value and are declared to be of importance to the modelling and
simulation activity).

Cost

Model
Resources Expertise
selction

Utility

Figure 15. Business Attributes and KPIs for model selection

The utility of a model is defined considering the problem on hand i.e. the user case and the
attributes to be estimated. The utility of a model is determined according to the information we
have reflecting the specific objective at hand. The decision might include theoretical and
computational considerations. Notice that each application case is translated into a workflow
consisting of materials modelling and materials data components using the MODA (Modelling Data)

D4.2: Model Selection 18

materials modelling data templates 3 as a starting point. The utility of a model is determined
according to the information we have reflecting the specific objective at hand: i) Is the model able
to extract quantitative useful to estimate the KPI or KPI attributes; ii) did we have access to hardware
and software resources to execute the simulation? Ii) how much is the simulation using such model
is costly? iv) is the predicted results obtained using the model with enough confidence? The
availability and/or access to the appropriate modelling and simulation resources is essential for
delivering quality results. The right resources must be applied to the right modelling and simulation
tasks. The modelling cost includes human resources, direct and indirect costs linked to the
development of software or tools acquisition, licensing fees, etc.

Modelling workflow selection with DMN.

DMN is an industry standard decision modelling language, which consents business individuals to
model and maintain the business decision logic involved in their processes, supporting operational
decisions (Figure 16). In the DMN specification, a decision is formally defined as “the act of
determining an output value from a number of input values using logic defining how the output is
determined from the inputs”. In other words, a decision is not the result of an action selection
process based on some business logic, but instead is the action selection process by itself. A decision
then is the application of business logic to select an output value from a combination of input values.
The action that could eventually be performed after the decision is outside the scope of the decision
model itself. DMN provides a standard graphical representation for the relation between decisions
and input data, a tabular representation for decision logic and also an XML format for execution and
interchange of decision models. The DMN tabular representation will be used to represent
COMPOSELECTOR business logic in the BDSS business processes, and its associated XML
representation will be used for the execution in the BPMN engine that implements the BDSS
business layer.

Figure 16: Decision Modelling and Notations (DMN) process

The advantages of modelling business decisions with DMN

In principle, decisions can be represented in a business process just as a sequence of logical steps,
which guide the execution based on the value of some input values. Figure 17 shows a simple
business process example, where a simulation task is performed based on the availability of a
required software license and enough CPU power in a local computer or in a cluster, with preference
for local execution if possible. Once information is obtained in the first task, a check is performed
3
https://emmc.info/moda-workflow-templates/

D4.2: Model Selection 19

on the local availability of the license required to run the simulation. If it is available, a check is
performed to determine if there are enough CPU resources locally available. If yes, simulation is
performed. If there is no local license, or if there are no locally available resources even if a local
license is available, a check on cluster license availability is performed. If there is a cluster license
available and there are enough cluster CPU resources, then simulation is performed. In any other
case, simulation execution does not take place. Please note that the “decision logic” is represented
in term of “process logic”, or in other words, the decision aspects are represented by process flow
elements (gateways) in the diagram.

Figure 17: A process where decision logic is modeled with process flow elements

The example presented in Figure 18 shows an example where decision logic is separated from the
process logic. The process starts with a task that obtains the required information as in the previous
example. The following task node is a decision logic node, which determines a single output value
from a given set of input values. The whole decision logic is implemented inside this node, hiding
the supporting logic decision internally.

Figure 18: A process where decision logic is implemented in a DMN decision task

As mentioned before, one of the elements from the DMN standard used to represent decision logic
is the so-called decision table. A decision table is a two-dimensional tabular structure in which inputs
and outputs are represented in columns with each row corresponding to a single decision rule. Each
cell has a Boolean condition which evaluates to true or false. If all input conditions are satisfied, the
rule is activated and the output value is selected from the corresponding column.
In our simple example, there are four input variables which guide the decision process: availability
of local license (LocalLicense), availability of cluster license (ClusterLicense), availability of enough
CPU resources in local computer (LocalResources) and availability of enough CPU resources in
cluster computer (ClusterResources), all of them of type Boolean. The decision logic is expected to
produce two output values: an indication if the simulation can be executed (canBeExecuted) and

D4.2: Model Selection 20

the location for the execution (executionPlace). This first output variable is of type Boolean while
the second one of type String.
DMN decision table for execution
F Input Output
localLicense clusterLicense localResources clusterResources canBeExecuted executionLocation
boolean boolean boolean boolean boolean string
1 true - true - true local
2 - true - true true cluster
3 - - - - false nowhere
Figure 19: A decision table for the simulation location example

Figure 19 presents the DMN table that can be used to model the decision logic described before.
There are four columns, one for each of the input variables, and two columns for the output
variables. Each row represents a decision rule. The table hit policy is “First”, identified by the letter
“F” on the top left corner. With this policy, rules are evaluated in the order listed in the table,
selecting the first one that matches, avoiding possible overlap between the rules. The first rule
establishes that if local license is available and local CPU resources are available, execution takes
place locally. Only if the first rule does not match, the second rule is considered. It establishes that
execution will take place in the cluster if a cluster license is available and enough CPU resources are
available in the cluster. In any other case, the simulation cannot be executed. The solution with
decision logic implemented as a decision task (Figure 18) is preferred due to a number of reasons.
In particular, decision logic implemented as a sequence of gateways (as in Figure 17) complicates
unnecessarily the process. But even worse, a small change in decision logic will correspond to a
structural change in the process. With the decision logic extracted from the process and fully
detailed in a separate model, the resulting BPMN model is more clear, simpler and maintainable.
DMN can break down decision logic in maintainable units and reusable modules, with better support
to changes than other approaches. In fact, keeping separated decisions logic from process logic
provides betters consistency across the whole enterprise, making also decision logic auditable and
traceable.

A general operative example in material modelling

While the example presented in the previous section introduces the advantages of using DMN
instead of standard process logic to represent decisions, this section presents a realistic scenario
that was elaborated in COMPOSELECTOR to select between a mesoscopic and an atomistic model
for execution. The scenario discussed in this section is the following: depending on various attributes
of the requested simulation, the end-to-end business decision indicates if there is a valid alternative
for simulation, and in this case, also an indication of the type of model selected. The DMN table
defined for this example is presented in Figure 20.
DMN selection table for atomistic and mesoscopic model
U Input Output
expertise cost (€) time (hours) reliability atomistic mesoscopic feasible
boolean integer integer integer boolean boolean boolean
1 false - - - false false false
2 - <= 100 - - false false false
3 - - <= 6 - false false false
4 - - - <= 1 false false false
5 true ]100…1000[ > 6 >= 5 true false true
6 true ]100…1000[ > 6 ]1…5[ false false false
7 true >= 1000 ]6…24[ >= 5 true false true
8 true >= 1000 ]6…24[ ]1…5[ false false false
9 true >= 1000 >= 24 ]1…5[ false true true
10 true >= 1000 >= 24 >= 5 true false true
Figure 20: DMN decision table for model selection

D4.2: Model Selection 21

Four input variables are considered for the decision:
• Expertise: A Boolean variable that indicates with the value “true” that expertise is currently
available to support the requested simulation and “false” otherwise.
• Cost: An Integer variable that represents the cost in Euros required to perform the requested
simulation, including the personnel costs to setup the environment.
• Time: An Integer variable that represents the estimated time in hours requested to perform
the simulation.
• Reliability: An Integer variable that represents with a value greater than 0 the reliability of
the solution that is required from the simulation.
Based on these four attributes, the DMN decision model produces three output results:
• Feasible: A Boolean value that indicates with “true” that all conditions are fulfilled to
perform a simulation and “false” otherwise.
• Atomistic: A Boolean value that indicates with “true” that an atomistic model can be used to
perform the simulation in the selected conditions and “false” otherwise.
• Mesoscopic: A Boolean value that indicates with “true” that a mesoscopic model can be used
to perform the simulation in the selected conditions and “false” otherwise.
As discussed before, the DMN table is defined as a set of rows, each one of them representing a
decision rule. This table follows two important properties:
• Completeness: all possible combinations of input values are covered by a rule.
• Consistency: the table provides a single conclusion for each combination of input values.
The four initial rules cover situations where it is clear that no simulation can be performed. The first
rule establishes that if no expertise is available, independently of the value of the other variables, it
is not possible to perform a simulation. The second rule establishes that with less than 100 € it is
not possible to perform a simulation, whether or not expertise is available. The third rule indicates
that with less than 6 hours of CPU time available, no simulation can be performed, while the fourth
rule precludes simulations where the required reliability is smaller than 1. The other rules cover
more specific situations. Just as an example, rule 5 indicates that an atomistic model can be used if
expertise is available, the budget is between 100 and 1000 Euros, at least 6 hours of CPU time are
available and the required reliability is greater than or equal to 5. This DMN table can be included
into a business process in a DMN task node. Figure 21 presents an example of a business process in
which this DMN task node can be used for model selection. The first task prepares the simulation
request getting all information required. Based on that, the DMN task, which includes the DMN
table presented in Figure 20, provides the answers which are used for model selection. If the
simulation is feasible, then atomistic or mesoscopic simulation is automatically performed and
results are stored in a database. If simulation is not feasible, a human actor is contacted in order to
take a decision about the possible solution paths supporting the decision-making process. That
person is provided with all the information, allowing him to modify the request, may be by
increasing the budget, increasing the amount of CPU hours available or changing the reliability
required. Once these attributes are changed, the DMN decision task is evaluated again, providing

D4.2: Model Selection 22

possibly a successful selection of the best model.

Figure 21: Business process for model selection between atomistic and mesoscopic

Clearly, the process could have been more complex, involving even different human actors with
different level of responsibility (technical or business), supporting even a more complex decision
path.

WORKLFOW SELECTION IN END USER’S APPLICATIONS

In COMPOSELECTOR, it is assumed that MuPIF workflows area available to the business layer and
stored in a database. These workflows can be executed from the BPMN processes implemented in
the business layer. The BPMN workflow depicted in Figure 25 shows how a MuPIF workflow can be
selected and executed in the “Simulation layer”, the lowest layer in the diagram presented in Error!
eference source not found.. The task “DMN task for workflow selection” selects a MuPIF workflow
by evaluating a DMN table using as inputs the data in the Data Object “Input for workflow selection”,
i.e., the set of requirements that guides the selection of a particular workflow.

Figure 22: Workflow selection by using a DMN decision table

The selection of the workflow can be implemented with a DMN table like the one shown in Figure
23, a decision table with seven inputs and a single output.

D4.2: Model Selection 23

 DMN model selection
U Input Output
expertise cost need complexity sensitivity reliability outputs workflowID
string string string string string string string integer
ϵ {rheology, process,
1 very high very high very high 1
mechanical}
no very detailed analysis
2 high high high ? 2
high
3 medium ? 3

4 limited detailed analysis medium ? 4

low low
5 ? 5
low very high
6 preliminary ϵ {stiffness, strength} 6
yes very low
7 conceptual very low very low ϵ {stiffness, strength} 7

Figure 23: DMN table

The seven input variables considered for the decision are:

• Expertise: A String variable that represent the expertise that is available. The possible values
are “no”, “limited” and “yes”.
• Cost: A String variable that represents the cost required to perform the requested
simulation, including the personnel costs to setup the environment. The possible values are
“very low”, “low”, “medium”, “high” and “very high”.
• Need: A String variable that represents the type of analysis requested. The possible values
are “conceptual”, “preliminary”, “detailed” and “very detailed”.
• Complexity: A String variable that represents the complexity of the model that is required.
The possible values are “very low”, “low”, “medium”, “high” and “very high”.
• Sensitivity: A String variable that represents the sensibility of the model that is required. The
possible values are “very low”, “low”, “medium”, “high” and “very high”.
• Reliability: A String variable that represents the reliability of the results expected using the
model. The possible values are “high” and “very high”.
• Outputs: A String variable that identifies the name of the required KPI for estimation. The
possible values depend on the model, for example “stiffness”, “strength”, etc.
Based on these seven attributes, the DMN decision model produces one output result:
• WorkflowID: An Integer variable that identifies the workflow that is selected.
Each rule in the table shown in Figure 23 models a relation between a MuPIF workflow and a set of
requirements represented by the input variables. For example, the rule 7 models that when no
expertise is available, we want to keep costs very low, we need a conceptual analysis for stiffness,
or strength, with a model of very low complexity, very low sensitivity and very high reliability, then
we can use the MuPIF workflow with WorkflowID 7. Please note that this decision table is
incomplete, i.e., there are combinations of value for the input variables not covered by any rule. For
example, there is no rule to fulfill a request for the analysis of stiffness with a very low cost and very
high sensitivity model. In such a situation, the table cannot compute an output value, and a
simulation cannot be run.
Figure 24 shows a DMN decision table that can handle these situations. Its last rule covers all the
input value combinations not handled by the other rules and return a default value “-1” as output.
The “-1” value represents the absence of a workflow fulfilling the input request. It can be used to

D4.2: Model Selection 24

redirect the process flow in order to contact a human to create a new workflow to support the
request.

Figure 24: The enhanced DMN table with default output.

Figure 25 presents a BPMN process that is similar to the one depicted in Figure 22, where a user
task has been added. An expert user is contacted in case no adequate workflow is available. The
expert creates then a new MuPIF workflow satisfying the requirements, which can then be used to
run the requested simulation.

Figure 25: Workflow selection

Note that DMN tables can be used to provide a complete automatic answer if all requirements are
fulfilled or used to start a human interaction in order to provide a solution for an unexpected
situation.

Conclusions
In this Deliverable, it has been model selection methodologies and tools are proposed. A tool based
on graph theory is developed. DMN is used for the selection process within the BDSS. DMN plays a
determinant role in analyzing, representing and documenting the decision logic that drives the
model selection aspects of the modeling approach. Moreover, DMN provides an executable
expression language, such that the decision model can be executed, without the need for

D4.2: Model Selection 25

programmers to translate the decision logic to other programming languages. However, the most
important point is that DMN is an industry standard. Decision tables can be created by any tool and
executed by any engine, provided they are compliant with the standard. In this way, the decision
process proposed in COMPOSELECTOR is absolutely general with no proprietary restrictions. The
examples presented in this document describes initial examples of successful representation of
model section using DMN. These examples include automatic decision-making activities and also
decisions where humans (Translator) are involved, proving support for the envisioned selection
process. External activities like optimization, data analysis and other tasks can be integrated by using
user tasks which involve external human operators, providing inputs to DMN tables supporting
model selection and decision making activities more generally.

D4.2: Model Selection 26