Model Validation

Oleh: Erma Suryani

 Outline
• Validation Definition
• 3 Steps in Determining if a Simulation is an
Accurate Representation
• Logical Sequence of Validation
• Model Verification &Validation
 Capaian Pembelajaran
• Mampu Menjelaskan Definisi Validasi
• Mampu Menjelaskan Tahapan Validasi Model
• Mampu Menjelaskan Logical Sequence of
Validation
• Mampu Menjelaskan Model Verification
&Validation
 Validation Definition
• Establishing confidence in the usefulness of a
model with respect to its purpose (Forrester,
1961 & Senge 1980)
• The process of determining the simulation
model based on an acceptably accurate
representation of reality (Giannanasi et al.
2001)
 3 Steps in Determining if a Simulation is an Accurate
Representation (R.F. Garzia and M.R.Garzia, 1990):
In developing a model, it necessary to
validate the assumption regarding with
the real system under consideration 
Model
Conceptual Conceptual Validation
Validation
Verification
Once simulation is developed, it is
necessary to verify that the model
contains with no bug programs 
Real Verification
Simulation
System
Finally, once the simulation has been
verified, the final validation step
Operational
Validation (Credibility)
consists of comparing the model’s
(simulation) with the real system 
Operational Validation (Credibility)
Logical Sequence of Validation (Barlas, 1996)

Simulation involved

No simulation involved Structure oriented

behavior test
• Compare the • Comparing the
model to the • Applying certain means
available behavior test on • Comparing the
knowledge the model variations
(indirect test)
Direct structure Behavior pattern
test test

Behavior Validity

Structure Validity
 Direct structure test

Structure confirmation test

• Compare the model equations with the relationships that
exist in the real system

Parameter confirmation test

• Evaluating the constant parameters against knowledge of
the real system both conceptually & numerically

Conceptually : able to identify elements in the real system that

correspond to the parameter of the model

Numerically : estimating the numerical value of the parameter with

enough accuracy
 Structure oriented behavior test

Extreme-condition test
• Assigning extreme values to selected parameters and
comparing the model to the real system

Behavior sensitivity test

• Determining those parameters to which the model is highly
sensitive, and asking the real system would exhibit the similar
high sensitivity to the corresponding parameters.
 Validasi Model - Behavior Pattern Test
a. Perbadingan Rata-Rata (Mean Comparison)

E1 
S  A
A

S  nilai _ rata  rata _ hasil _ simulasi

A  nilai _ rata  rata _ data

Model dianggap valid bila E1  5%

Validasi Model - Behavior Pattern Test (Cont’d)
b. Perbandingan Variasi Amplitudo (% Error
Variance
Ss  Sa
E2 
Sa
Dimana:
Ss = standard deviasi model
Sa = standard deviasi data
Model dianggap valid bila E2  30%
Model Verification &Validation
 Model Verification
Technique that is deployed to ensure that

Its conceptual
model design
• The • Have been
requirement • And transformed
s for

The simulation computer

model model

with sufficient accuracy

 Model Validation
Technique that is deployed to reach

Its intended
• An acceptable use • applicable to
level of • are sufficiently
confidence accurate for
that
The simulation The system
model ‘s results being modeled
 Verification &Validation
Verification is done to ensure that:
The model is programmed correctly
The algorithms have been implemented
properly
The model does not contain errors, or
bugs
The specification is complete & mistakes
have not been made in implementing the
model
Validation is done to ensure that:
The model meets its intended
requirements
The model useful or
addresses the right problem
Provides accurate information about the
system being modeled
Makes the model actually used
 Three main dimensions of validity
• Substantive: perceived representativeness of models
• Constructive: process effectiveness or learning by
participants
• Instrumental: outcome or impact of the project
 Substantive Dimension
Test:
• Structure
• Behavior

Must include essential tests : dimensional consistency

 Constructive Dimension
• Understanding of causal mechanisms
• Insights gained that have general applicability
 Instrumental Dimension
• Model facilitates development of later versions in
modeling process
• Is implementation likely?
• Acceptable cost
 Untuk Emi
• Meningkatkan efektivitas dan efisiensi rantai
pasok komoditas jagung.
• Mengembangkan sistem pertanian ramah
lingkungan melalui model dan skenario
simulasi.
• Meningkatkan efisiensi energi dalam
mengelola usahatani.
• Membuat model adaptasi untuk mengatasi
dan mencegah bencana.