Composite, Real-time Validation for Business Process
Modelling
Martin Smuts Clayton Burger
Department of Computing Sciences Department of Computing Sciences Department of Computing Sciences
Nelson Mandela Metropolitan Nelson Mandela Metropolitan Nelson Mandela Metropolitan
University University
Port Elizabeth, South Africa Port Elizabeth, South Africa Port Elizabeth, South Africa
Martin.Smuts@nmmu.ac.za Clayton.Burger2@nmmu.ac.za Brenda.Scholtz@nmmu.ac.za
ABSTRACT
Graduates with Business Process Management (BPM)
competencies are highly sought after and Business Process
(BP) modelling skills form an integral part of the BPM skillset.
One challenge facing modellers is that only a few existing BP
modelling languages have intuitive validation techniques
which check for syntactic issues or rule violations. Not having
an intuitive validation technique to support the effective and
accurate creation of BP models can affect the success of BP
modelling projects and have a negative impact on the
operations of the business. In this paper a composite, real-time
validation technique is proposed which provides both graphical
and textual feedback. The Event-Driven Process Chain (EPC)
modelling notation is used to demonstrate the proposed
validation technique. Investigating the effectiveness and
usability of the approach when implemented in a Microsoft
Visio environment reveals that a composite, real-time
validation technique reduces the volume of issues that typically
are made during the modelling activity and enables modellers
to distinguish between the severities of the different types of
issues. The validation technique can also assist modellers in
learning the rules and syntax of a modelling language.
Categories and Subject Descriptors
K.3.2 [Computers and Education]: Computers and Information
Science Education
General Terms
Business Process Modelling, Business Process Modelling
Languages, Validation.
Keywords
Event Driven Process Chain (EPC), Business Process
Modelling, validation, Business Process Management,
experiential learning and education.
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. Copyrights
for components of this work owned by others than ACM must be honored.
Abstracting with credit is permitted. To copy otherwise, or republish, to
post on servers or to redistribute to lists, requires prior specific
permission and/or a fee. Request permissions from
Permissions@acm.org.
SAICSIT2014, September 29 - October 01 2014, Centurion, South Africa
Copyright 2014 ACM 978-1-4503-3246-0/14/09…$15.00
http://dx.doi.org/10.1145/2664591.2664603
Brenda Scholtz
University
1. INTRODUCTION
Forecasts for the Business Process Management (BPM) market
predict a growth from $2.8 billion in 2013 to $8.3 billion in
2019 [1]. This growth has led to a need for Business Process
(BP) modelling skills which have become essential in various
fields, ranging from Information Systems (IS) development,
Software Engineering and BPM practices [2, 3]. BP modelling
is used in organisations before they implement new systems or
make crucial decisions, and can assist with understanding
workflows and identifying manual, repetitive and system
automated tasks within BPs [4, 5]. However, the objective of
BP modelling is not simply to configure manual or repetitive
tasks, but to create a common understanding of the interactions
and dependencies among: people, organisational operations,
the systems they use, and the information required to do their
tasks effectively [5]. Furthermore, BP modelling is used to
identify process improvements and to demonstrate compliance
with external regulations [6]. A BP modeller typically creates a
BP model, which consists of a network of BP diagrams that
graphically depict descriptions and sequence of at least the
activities, events/states and control flows that constitute the BP
process [7]. BP modelling is an activity that is complex and
time-consuming as the core activities need to be arranged in
their execution order and exceptions dealt with if and when
they occur [8]. Although various languages for BP modelling
exist, each language has its own unique notation and syntax
rules which vary in complexity and provide challenges during
training and education, which can often be attributed to the
semantic and syntactical complexity [7, 9]. Notation typically
refers to a set of symbols or elements that are used to
graphically depict a process model as a visual model, whereas,
syntax provides a set of rules or constructs [7, 10].
Seethamraju [11] argues that there is a global need for BP
education and that graduates have inadequate BPM
capabilities. As a result several Higher Educational Institutions
(HEIs) have introduced BPM courses in their IS programs [3,
11, 12]. The heavy demand for skilled BP modellers poses
several challenges for HEIs offering BPM courses [2]. These
HEIs often struggle to provide sufficient software and training
resources to support graduates during the course of their study
[12]. The complexity and learning curve of a particular
notation and modelling tool requires a significant amount of
time and money for teaching modellers to perfect their BP
models [2, 12]. Students of BP modelling courses often
struggle with the complexity of modelling languages and the
associated rules, which highlights the need to investigate
effective methods for teaching these. Furthermore, there exists
little insight into the way modellers interact with a BP
modelling tool, the pages they are using, what areas they get
stuck on for longer than anticipated, or alternative ways that
they find for addressing problems that repeatedly arise [5]. Not
93
addressing these issues will inevitably affect the performance
and quality of the BPs.
Individual stakeholders are not necessarily familiar with BP
modelling [13]. Moreover, BPM tools heavily rely on experts
that model idealised “to-be” processes and do not help the
stakeholders to understand the “as-is” BP [9]. Therefore, it is
difficult for non-expert individuals to derive at a BP model that
expresses their specific systems requirements. Barjis [14]
reveals that an alarming amount of software projects fail due to
poor BP modelling. The cost of fixing errors in the
specification of a BP model is high and can have a significant
impact on an organisation’s operations [15]. A reasonable
explanation for this phenomenon is twofold. Firstly, a lack of
BP modelling skill is applied by the modeller. Secondly,
current BP modelling tools lack direct model checking features
or validation techniques [14]. Allowing novice modellers, such
as student modellers with virtually no practical modelling
experience, to create BP models without a sufficient validation
technique increases the probability of making errors in their
designs [16]. These issues are typically less evident in smaller
models, but as the models expand and increase in complexity
modellers often lose track of the modelling notation’s syntax
rules and interrelations due to their limited cognitive ability
[17]. The result is that modellers are often unaware of errors
that exist in the created BP models as traditional modelling
tools do not provide optimal support on issue feedback [10]. A
real-time validation technique allows the modeller to detect
and fix issues during a modelling activity [18]. A validation
technique should also provide feedback to the modeller by
reporting the reason for the rule violation or issue [19].
The main aim of this paper is to propose and evaluate a
composite, real-time validation technique for BP modelling.
The evaluation will determine if the technique can assist
novice BP modellers with their knowledge of notational syntax
rules, thereby improving the accuracy and quality of the
produced BP models. This study therefore addresses gaps in
research related to BP validation techniques and tools. The
need for a validation technique has been motivated by various
authors [7], [10, 20]. In an attempt to satisfy this need,
literature provides evidence of a variety of validation
techniques and tools being contributed. However, these tools
lack intuitiveness and are often applied after the BP model is
created, which has a great impact in terms of the BPM
lifecycle (Section 2). The real-time validation technique is
applied to a popular modelling language, the Event-Driven
Process Chain (EPC) language (Section 3). The research
methodology followed for this study is a field study approach
whereby the proposed composite validation technique was
developed and evaluated at a HEI in South Africa (Section 4).
The evaluation results suggest that the application of the
composite validation technique was successful and that it was
well received by students (Section 5). Several conclusions and
recommendations from this study can be made (Section 6).
2. BUSINESS PROCESS MANAGEMENT
BPM is a wide management approach that evolved from
multiple disciplines and consists of a lifecycle that involves
four generic phases (Section 2.1). The EPC modelling
language consists of several elements and syntax rules (Section
2.2). Several rules for the validation for BP models can be
identified (Section 2.3). The BP modelling tool, Microsoft
Visio, is the platform used to implement the composite
validation technique (Section 2.4).
2.1 Business Process Management Lifecycle
A BP consists of a set of tasks and activities within an
organisational structure that are linked with a logical flow
using task dependencies and events. Each BP serves to produce
a desired result which is valuable to both the organisation and
its stakeholders [21]. BPM is a multi-disciplinary practice and
process-centric approach for improving organisational
performance using a combination of methods, techniques and
software to assist organisations during the BPM lifecycle
activities [5]. Moreover, Harmon [6] defines BPM as a
management approach that typically focusses on process
optimisation, process analysis and operations management
using technology-enabled software that drives organisational
agility and innovation. BPM software tools are often bundled
together as a complete software package known as a Business
Process Management Suite (BPMS) that enables the BPM
lifecycle activities for design, execution, monitoring and
controlling of BPs [22]. Within the scope of BPM, processes
are modelled to bridge the gap between business audiences,
technical audiences and technologies, thereby aligning the
organisation’s resources according to its requirements and
business rules [23]. BP modelling tools typically support
languages, such as, Business Process Modelling and Notation
(BPMN), Unified Modelling Language (UML), Petri-net and
traditional flowcharts. Although BPMS do exist that provide
functionality for each activity in the BPM lifecycle, smaller BP
modelling tools are typically used for the creation of BP
models, which can be exported to external operational systems
and execution engines. Popular BP modelling software tools
are IDS Scheer ARIS Business Architect and Express,
ADONIS, IBM WebSphere Business Modeller and Microsoft
Visio [22, 24].
Recker [15] describes BP modelling as an essential cornerstone
of the initial capturing and documentation of existing BPs, as
well as the specification of the redesigned, automated or
amended BPs. The use of BP modelling is critical throughout
the BPM lifecycle, as it forms the foundation of the
understanding and restructuring of process activities,
information flows and roles of participating individuals and
applications for process optimisation [4, 7]. A lack of BP
modelling training has become a key issue in BP modelling
[12]. Recker and Rosemann [3] suggest that these challenges
can be addressed by improving students’ technical knowledge
of BP modelling as well as the students’ actual modelling
skills. Students, therefore, require knowledge of the most
popular modelling methods and practice with market leading
BP modelling tools.
However, BP modelling languages often lack sufficient
understandability due to the complexity of BP modelling
methods, notation and syntax rules [25]. A study by Mendling,
Reijers and Cardoso [26] confirms that students tend to
improve their understanding of process models when more
time is spent on training with BP modelling tools.
Additionally, Mendling, Reijers and van der Aalst [27] explain
that BP modelling tools that support the modelling process are
favoured by modellers; for example, tools which provide
warnings when syntax rules are violated or when models
become too large.
2.2 The Event-Driven Process Chain (EPC)
Modelling Language
The Event-Driven Process Chain (EPC) modelling language is
developed within the Architecture of Integrated Information
Systems (ARIS) framework by IDS Scheer [28]. The
94
framework provides a central method to conceptually integrate
various dependencies that exist between data, activities,
functions, organisational units and its outputs in terms of
designing for information systems [29]. The advantage of
using the EPC modelling language lies in its high user
acceptance and extensive tool support. EPC models are easily
understood and to express BPs in an intuitive way [7, 30]. EPC
models are most often used in practice with ERP systems such
as SAP R/3 [31] and for teaching introductory BP modelling
[32].
The importance of rule-adhering EPC models are emphasised
in situations when EPC models need to be interchanged with
respective software execution engines such as SAP R/3 ERP
systems and other BPMS products. EPC models that embed
issues hinder the smooth interchange and execution of BPs
across the organisation. For this reason, EPC models often
require intensive revision during the design activity before
being applied to software execution engines in the enactment
activity [31, 33]. EPC models are used to logically construct
the business activities at a low, detailed, technical level to
enhance understanding (Section 2.2.1) and must adhere to the
set of EPC rules (Section 2.2.2).
2.2.1 Elements of the Event-Driven Process Chain
Notation
EPC models are constructed as a set of nodes and arcs as a
directed graph. Nodes are either: Functions, Events or Logical
Connectors (Figure 1). A Function represents a task or an
activity that needs to be executed within a process, which uses
time and resources. An Event describes a condition or a status
that exists at a particular time within a process. Events trigger
Functions which are in turn triggered by Events. Three types of
Logical Connectors (AND, OR and XOR) exist, which each
can serve as a connector split or a connector join.
Figure 1. Elements of the EPC notation.
Each of the Logical Connector types differs in their semantic
meaning. Arcs between these elements depict the sequence and
direction of control flows (inflows and outflows) in order to
represent the interdependencies that exist between them. The
AND-split logical connector enables a control flow to be split
into two or more parallel outgoing branches. The AND-split
activates all the outgoing branches to execute concurrently.
The AND-join is required to wait for all active branches to
complete in order to synchronise the branches into a single
control flow. The OR-split enables one incoming control flow
to be split into two or more outgoing branches. The OR-split
logical connector either activates a single branch or multiple
outgoing branches. In doing so, a specific condition is satisfied
and the OR-split will activate at least one of the alternative
branches and disable the rest. The OR-join is required to wait
for all the activated branches to complete in order to
synchronise the branches into a single control flow. The XOR-
split logical connector enables one incoming control flow to be
split into two or more outgoing branches. The XOR-split
creates a situation where a decision needs to be made. Only
one branch amongst the multiple outgoing branches is allowed
to be activated. The XOR-join is required to wait for the
activated branch to complete in order to synchronise the
branches into a single control flow.
2.2.2 Event-Driven Process Chain Rules
According to various authors [21], 31] a strict set of syntax
rules and recommendations apply to the EPC modelling
language. Examples of the main EPC rules are:
 Events must be used at either the start or end of a process
and when important conditions exist during the process.
 Events are passive elements and cannot make decisions.
Events can, therefore, only be followed by an ‘AND’
Logical Connector to trigger Functions in parallel and
not an ‘OR’ or ‘XOR’ Logical Connector. Figure 2
depicts the allowable logical connections between Events
and Functions.
 An Event can only be followed by a Function, unless
followed by a Logical Connector.
 Events and Functions should always alternate. It is still
important to alternate between the Event-and-Function
patterns when intercepted by a Logical Connector.
 Functions are allowed to be followed by all the Logical
Connector types, as Functions are allowed to make
decisions and trigger activities in parallel.
 Every element used within the EPC model must be
connected using a control flow and provided with an
appropriate text label. Elements that are free-standing or
have no text label add no value to the created EPC
model.
Single Event Single Function Multiple Event Multiple Function
Trigger Trigger Trigger Trigger
A
N V
V
XOR
V
V
V V
V V
O V V
R Not Allowed
VXOR
V
X V
XOR
V
XOR
O Not Allowed
VXOR XOR
R
Figure 2. Adapted Logical Connections between Events and
Functions [34].
2.3 Validation of Business Process Models
An EPC model, whether it is a high-level or low-level
representation of a BP, should always adhere to the set of EPC
rules. For this reason, the importance of a validation technique
for EPC modelling is emphasised. Having the correct
validation technique that can assist in the identification of
issues is critical to the quality of the overall EPC model.
Mendling [7] substantiates this need by stating that novice
modellers often create BP models of a low quality without
truly understanding the impact that the models have on
technologies that are intended to execute the modelled
processes. It is therefore important to identify issues at the
earliest possible stage during the design of a BP as the cost to
repair a BP at a later stage is extremely high [18].
Validating a BP model for compliance with respect to some
syntax or business rule is critical and should be supported by
95
appropriate tools and concepts [35]. Furthermore, Kühne et al.
[10] explain that a validation technique should follow a
framework pertaining to three quality goals namely: syntactic
quality, semantic quality and pragmatic quality. Syntactic
quality refers to the extent to which the process models adhere
to the syntax rules for a given BP modelling language.
Semantic quality is concerned with the completeness and
validity of the process model, implying that a process model is
only valid when all statements made in the model are correct
and that the model has no missing elements that give meaning
to the process as a whole. Pragmatic quality refers to the
assistance that is provided to the modeller by recommending a
better alternative in the given situation. Pragmatic quality is
concerned with the selection of the most practical alternative to
derive at a construct that depicts the same meaning.
In order to provide automated support for compliance
checking, compliance rules have to be expressed in a formal
language [35, 36]. Declarative validation languages typically
constrain the patterns in the model by using Boolean queries
(If-clause) to suit the compliance rule, such as “A True value is
returned If Element A is connected to Element Z” to indicate
that a fragment adheres to the rule [35]. Declarative validation
languages enable queries to be executed rapidly as small
fragments are targeted in the created BP model. The
advantages of using declarative validation languages are
twofold. Firstly, it supports the creation of complex syntax
rules that are applied to parts of the BP model. Secondly, the
majority of these languages have the ability to provide real-
time textual and visual feedback based on the implemented
rules [10, 20, 36, 37].
Transgressions of each EPC rule category can be classified into
five logical categories of similar rules (Table 1). These logical
groupings can be further decomposed into two high-level
groups: major and minor rule violations. Major rule violations
are syntactic violations, while minor rule violations assist in
the readability of the constructed model. Although minor rule
violations do not directly violate syntactical correctness of the
EPC model, they can cause deadlocks when interpreted by
execution engines if not addressed.
Table 1. Rule Categories (Based on Monk and Wagner
[34])
Rule Description
Category
Alternation Includes rules that govern the Event-
to-Function alternations throughout the
created EPC model. This category also
includes rules that govern the
alternation of Events and Functions
around Logical Connectors.
Major
Logical Includes rules that govern the
Connectors allowable Logical Connections (Figure
2). This category prevents Event
elements being used for decision
making.
Control Includes rules that prohibit elements
Flow from exceeding the allowable amount
of inflows and outflows.
Text Label Ensures that all elements are labeled.
Minor
Connectivity Includes rules ensuring no elements
are free standing and have at least a
single outflow or inflow.
2.4 Microsoft Visio
Microsoft Visio is an example of a simple modelling tool
which is used both in industry for professional modelling and
in educational environments for modelling and diagraming
instruction [24, 25]. Users tend to be familiar with Visio’s
simple drag-and-drop approach and the tool provides a pre-
defined set of templates for the most common modelling
languages. Visio has been reported to be easy to use and can
easily be customised. The modeller manipulates the model by
selecting elements, also known as shapes, from a stencil and
structures those elements on a canvas. Microsoft Visio
Professional 2013 offers a standard validation technique for
BPMN, Cross-Functional flowchart, Basic Flowchart, Six
Sigma and Microsoft SharePoint 2013 Workflow templates
[38]. This validation technique is executed through a validation
engine, which is programmed in a declarative validation
language. The validation technique provides the modeller with
textual feedback on any issues that are embedded in the created
BP model. Visio provides a Validation Application
Programming Interface (API), which enables programmers to
develop their own rules for any Visio template [38]. The
default feedback provided for any models that fail validation is
provided as “issues” and provided as textual feedback only.
Thus, graphical feedback or visual alerts are not natively
supported in the Visio environment and validation is performed
manually by the user through the use of a “check diagram”
button. Furthermore, the standard EPC template supported in
Visio does not offer a set of EPC rules to validate the created
EPC model against.
3. A COMPOSITE VALIDATION
TECHNIQUE
A validation technique should not only detect existing issues in
a model, but it should communicate additional information to
the modeller about the particular cause and solution to the
issue, as well as, any warnings to prevent issues [18]. Wynn et
al. [39] and Kühne et al. [10] suggest that a validation
technique should give instant and continuous feedback about
any issues that exist whilst modelling. This is supported by
Speck et al. and Awad et al. [18, 35] who propose that a
validation technique should be interactive and real-time,
allowing the modeller to correct an issue as it occurs.
Furthermore, a validation technique should provide assistance
and recommendations based on the solution to an issue [40].
Four main features of BPM validation techniques are therefore
identified in this study and together comprise a composite
validation technique which serves as the key contribution of
this paper (Figure 3). The features were all incorporated into
the Visio platform as a proof of concept (Section 3.1) and are:
 Rule set categorisation (Section 2.3);
 Textual feedback (Section 3.2);
 Graphical feedback (Section 3.2); and
 Real-Time validation (Section 3.3).
96

Figure 3. Composite Validation Technique.
3.1 Application of the Composite Validation
Technique
This paper proposes the composite validation technique and
applies it to the Visio platform as proof of concept. The Visio
platform was therefore modified by changing the default
canvas behaviour. Firstly, the set of EPC rules were
programmed for the EPC template in Visio. Programming the
EPC rules enabled the native Visio validation engine to
compare the rule-logic with the created model onscreen. In
addition to the rule-logic, a description of each rule and the
rule-category was programmed along with a possible solution
to provide the modeller with textual feedback when a rule is
transgressed. Visual alerts were incorporated to indicate the
exact element(s) onscreen that transgresses a particular
validation rule, as well as, whether the rule transgression is of a
minor or major nature. Lastly, the validation process in Visio
was automated by incorporating continuous sectional checking
of subsets of the created diagram. Thus, the modeller receives
instant, real-time graphical and textual feedback as the
elements are manipulated onscreen. Providing multiple types
of feedback allows the modellers to decide which means of
feedback is most desirable for them at any given stage.
3.2 Textual and Graphical Feedback
When a model or part thereof fails validation due to a rule
violation, such as incorrectly linked EPC elements, the
modeller should be informed with the cause together with a
suggestion to solve the issue [10]. The textual feedback lists
descriptions for each incurred issue in a separate feedback sub-
window (Figure 4, note d). The Visio environment provides
native support for the display of textual feedback to the
modeller when a declarative rule indicates that a validation
issue has been detected. The modeller will typically click on an
issue’s text and expect the offending element to be selected
(Figure 4, note b), which is natively supported by Visio.
Graphical feedback is a feature that applies graphical effects to
the elements that are involved with issues in a BP model.
Awad, Weidlich and Weske [35] apply the graphical
highlighting of process paths in a BP modelling language
similar to EPC. Graphical feedback is also enabled using a
combination of pattern recognition and declarative validation
languages to highlight elements according to a colouring
algorithm [41]. Kühne et al. [10] illustrate a different method
for graphical feedback by annotating the elements involved in
issues with markers. Graphical feedback features are
advantageous to modellers as it provides feedback in an
intuitive manner which is easy to understand to enhance direct
understanding [20]. In addition to the textual feedback,
graphical feedback is provided using two different colours to
differentiate between minor and major issues (Figure 4, notes a
and c).
a) Red graphical b) Visio c) Yellow
effect for major supported graphical effect
issues selection for minor issues
d) Textual descriptions
and categories
Figure 4. Colour coding graphical (a, c) and textual
feedback (d).
If a serious modelling rule violation is detected, the element, or
subset of elements, is given a red glowing border for
immediate notice. Less serious issues are highlighted in a
similar fashion but with a yellow colour which indicates that
they are not as urgent as red issues. Three alternatives to
glowing effects were considered (Figure 5), namely: adding a
glow effect around the border of offending elements, changing
the line weight of borders of offending elements and changing
the actual colour of the elements. By consulting BPM students
during a focus group discussion, it was determined that the
glow effect would be most effective graphical effect and was
revealed to be the most visually striking and was thus selected.
Figure 5. Candidate graphical compliance effects for rule
violations.
3.3 Real-Time Validation
To apply validation to a model, the Visio environment only
supports user-requested validation through the use of a “check
diagram” button [38]. Continuous, or real-time, validation is
considered to be a favourable alternative to user-requested
validation [10]. The composite validation technique uses real-
time validation to allow novice modellers to detect issues as
they are modelling and to thus be aware of any modelling
flaws before the model is finalised.
By ensuring that novice modellers receive feedback in real-
time, the Visio environment was adapted through macros
which trigger whenever an element is placed or interacted
with. This allows the modeller to ignore an issue if they desire,
while the feedback about the issue will remain persistent. By
applying persistent feedback, the risk of the modeller
temporarily ignoring an issue and subsequently forgetting to
rectify it is minimised [20, 35].
4. RESEARCH METHODOLOGY
The goal of this paper is to propose and evaluate a real-time
composite validation technique for novice BP modellers.
97
Therefore the main research question this paper addresses is
“Can the proposed real-time, composite validation technique
successfully assist novice modellers with their knowledge of the
rules of a BP modelling language?” The proposed, real-time
composite validation technique was implemented in the Visio
environment and the success of the technique was evaluated by
means of a field study involving an informal focus group and a
usability evaluation (Section 4.1). A number of research
instruments (Section 4.2) were used to measure the success of
the technique. Students of a BPM course were used as
participants in the study to evaluate the modified Visio
software (Section 4.3).
4.1 Research Design
In order to determine the preferred features for a BP validation
technique a focus group approach was used with BP modellers.
A cognitive walkthrough was used in a focus group to identify
any usability problems or cognitive issues, by analysing the
mental processes, assumptions and goals required by users of
the validation technique [42]. Determining the tasks that
modellers perform during a modelling activity is particularly
useful for selecting the most appropriate validation features to
assist in those tasks. A single cognitive walk-through was
performed using six participants, to determine broad issues and
ideas. The participants were selected through purposive
sampling, ensuring that the participants had some experience
with BP modelling. These participants were all HEI students
that have either completed or were still attending a one-
semester BP modelling course. The cognitive walkthrough was
initiated by presenting participants with a specific set of tasks
that are typically performed during a modelling activity. Each
of the proposed validation features was individually introduced
and discussed by demonstrating the functionality of each
feature using a combination of paper-based prototypes and
Microsoft Visio software. This allowed participants to evaluate
the proposed validation features that appeared to be preferred
in terms of accomplishing modelling tasks. Any suggestions
and concerns surrounding the usefulness of the proposed
design were raised and discussed. The different validation
feedback methods were evaluated and the most preferred
methods were incorporated into the design of the modified
Visio software (the prototype) which implemented the real-
time, composite validation technique.
Once development of the prototype was complete, a pilot test
was conducted with two modellers who had some BP
modelling experience, both external to the sampled users for
the cognitive walkthrough and usability studies. Two minor
problems were identified and the changes were made to the
prototype. The final prototype was then evaluated according to
its usability, since usability is increasingly recognised as an
important quality factor for interactive software systems.
Usability focusses on measures for effective, efficient and
satisfactory task execution and aims to support the ordinary
and uninterrupted interaction between the user and the system
[43]. Evaluating the usability of a system is particularly useful
when aiming to improve the user interface or to establish the
quality in use within a given context.
4.2 Research Instruments
A post-test questionnaire was administered to participants
which was based on the System Usability Scale (SUS). The
first section of the questionnaire addresses the participants’
previous experience since this can have an effect on the results
of the usability evaluation [44]. The SUS is a popular standard,
ten-statement, usability measurement tool that measures users’
perceived impression levels and system preferences, and
covers a variety of characteristics of the system under
evaluation, such as usefulness, ease of use and complexity
[45]. Each statement in the SUS is answered on a five-point
Likert scale on which participants rate their level of agreement
[46]. Half of the statements were positively worded and half
were negatively worded. The five-point Likert scale ratings
ranged from strongly disagree (a rating of one) to strongly
agree (rating of five). According to Tullis and Albert [46], it is
not recommended to report on the ten statements individually,
but should rather be reported on as a combined rating for
overall usability. Furthermore, it is convenient to think of the
SUS scores as a percentage, since they are on a scale of 0-100,
with 100 representing a perfect score. Each statement’s
contribution towards the combined SUS score differed and was
calculated according the method suggested by Brooke [45].
The third section of the questionnaire consisted of statements
regarding the validation technique’s perceived usefulness for
preventing successive issues from occurring which participants
were required to rate on a five-point Likert scale. All
participants were required to complete the same task-list,
which consisted of three major tasks with predefined subtasks.
The subtasks instructed the participants to specifically structure
the EPC model in violation of the set of EPC rules. It was
necessary to include subtasks which triggered the validation
aspects of the prototype. The subtasks also assisted participants
with rectifying the issues. Task 1 was simple with the goal of
getting participants acquainted with the EPC validation
technique in Visio. Tasks 2 and 3 evaluated the validation
technique’s ability to assist modellers in preventing and
rectifying rule violations throughout the modelling activity.
Tasks 2 and 3 included a single “problem-solving” subtask
which required participants to use their own initiative to rectify
existing issues in the created EPC model. The problem-solving
subtasks required participants to rectify the issues from the
validation feedback provided on the screen. Participants were
also allowed to use their own knowledge to solve problem-
solving tasks. The major tasks included rule violations relating
to all categories. It would have been impractical to test every
rule that was implemented in the prototype. A decision was
made to include at least one rule from each category. Eye-
tracking has become significantly more common over the past
few years as it shows researchers exactly where participants
are looking at a specific point in time [46]. In order to analyse
the performance of participants’ task list actions, the onscreen
video Pupil Centre Corneal Reflection (PCCR) technique was
used to analyse eye-movement, including gaze plots and
fixation patterns, as well as mouse-movements involving
mouse-clicks and mouse roll-overs [47].
4.3 Participant Profile
Information Systems (IS) students from the Nelson Mandela
Metropolitan University (NMMU) in South Africa were
selected as a purposive sample for the focus group (n=6) and
usability evaluation (n=14) with no overlap between samples.
The usability evaluation was conducted with participants who
are representative of students that typically enroll in the BP
modelling course every year. A total of 14 participants were
selected for the usability evaluations with the majority (64%)
being male (n=9) and the remainder (36%) were female (n=5).
The majority of the students (93%) were between the ages of
20 and 22 (n=13) and all of the students had prior experience
(1-6 months) with BP modelling and Microsoft Visio. The
application of the composite validation technique is
particularly suitable within this BPM course as the students are
98
expected to draw BP models with Visio. Since the validation
technique was tested with the EPC modelling language,
participants were required to have some knowledge of, and
practical experience with, the EPC modelling language.
5. RESULTS OF USABILITY
EVALUATION
5.1.1 Effectiveness and Efficiency
The effectiveness metric was assigned two sub-metrics: a)
task-completion and b) issue correction rate. Task-completion
indicated whether the validation technique assisted modellers
to successfully complete the modelling activity. Issue
correction rate indicate whether the modellers were able to
correct issues in order to derive at a successfully completed
and rule-adhering EPC model. The majority of the participants
(n=12) were able to complete all the tasks successfully to
derive at the final EPC model. Task 3 was deemed
unsuccessfully completed by only two (n=2) participants, be
because they failed to place a matching “OR” Logical
Connector to merge the process in the created EPC model.
They placed a Logical Connector in the correct position, but
placed an “XOR” and an “AND” Logical Connector
respectively. A rule governing the matching Logical
Connectors are typically related to the EPC modelling
language’s semantics. The proposed validation technique does
not attempt to address the semantics of the EPC modelling
language. For this reason, the validation technique relies on
modellers to use their own initiative to correct any issues
surrounding the matching Logical Connectors. Although the
scope of this study excluded the evaluation of matching
Logical Connectors, this issue provides evidence that such a
validation rule is a requirement. Since Task 3 was the final task
and two participants failed to successfully complete the final
problem-solving task, two of the final EPC models were
declared to be unsuccessfully completed.
A total of nine rule violations were made by participants
during Task 3 of which only seven were successfully
corrected. The remaining two issues were made by same
participants that caused two of the final EPC models to be
declared unsuccessful. These seven corrected issues related to
spelling, reading, partial disobedience and connecting multiple
inflowing or outflowing Dynamic Connectors to an Event or
Function element. A total of 20 issues were made during the
entire usability evaluation. The validation technique enabled
participants to correct 90% of the total issues that were made,
which supports the validation technique’s ability to effectively
enable participants to correct issues. The validation technique
is efficient since the mean time overall for participants to
complete the entire task-list was 16 minutes and 18 seconds,
which is less than the target time of 20 minutes.
5.1.2 Issue Awareness
Issue awareness was used as a metric for measuring the support
for learning of BP rules provided by the software. Issue
awareness was further categorised into a) awareness of minor
and b) major issues, as well as, c) awareness of textual
feedback. The minor issues were represented by a medium-
sized, yellow glow effect. Minor issues were typically noticed
for elements at the beginning of each task, such as, when a
newly added element required a text label or had no incoming
or outgoing Dynamic Connector. A common trend was
observed amongst participants (n=14) for the minor issues. The
awareness of minor issues gradually diminished as participants
progressed through the task-list. This result is supported by the
aggregated heat map of participants (n=14) for task 1 (Figure
6). The heat map indicates a variety of spots consisting of
green, yellow or red colours. Green spots represent little focus
or a typical glance on the affected area onscreen. Yellow spots
represent a moderate amount of focus was applied on the
affected area onscreen, whereas, red spots represent extensive
focus on the affected screen areas.
It can be deduced that from the diminishing awareness of
minor issues that the participants perceived the minor issues as
a low priority. Participants eventually started to temporarily
ignore the minor issues and devoted less effort to read the
associated textual descriptions to solve the issue. The
decreased awareness in minor issues did not, however,
decrease the effectiveness of the validation technique as
participants in due course fixed the minor issues successfully.
The fact that participants started to temporarily ignore the
minor issues to address the major issues supports the result that
learning did occur during the course of the evaluation. This is
evident since participants could distinguish that the minor
issues were of a preventative nature. Participants learned that
although minor issues were temporarily ignored, they would
not affect the core structure of the EPC model and could rather
focus on major issues that would have a significant effect on
the model if not fixed immediately.
c) Extensive
focus on major
issue
a) Less focus
on minor
issues
b) Less focus
d) Extensive on category
focus on column
first part of
the textual
descriptions
Figure 6. Visual heat map aggregated from all participants
for task 1.
The major issues were represented by a large-sized, red glow
effect. The awareness of major issues remained constant
throughout the evaluation. Elements that had undergone a red
glow effect caused participants to generate extensive focus on
the elements involved in the issue (Figure 6, note b). An
observation was made as participants were timeously inclined
to execute a task onscreen and instantly read the list of task
instructions once more. For this reason, some elements
involved in major issues were temporarily unnoticed. These
elements were, however, instantly noticed when participants
refocused on the screen.
The participants were inclined to perceive the major issues as
high priority. Another observation was made as the major
issues encouraged participants to read the textual descriptions
more thoroughly compared to minor issues. Participants were
also inclined to devote more effort to solve the major issues
instantly. This was done by observing the surrounding EPC
structure and related textual descriptions of the issues in the
EPC model. An observation was made from the eye-tracking
video recordings that participants were more likely to identify
issues from the graphical feedback than the textual feedback.
The eye-tracking indicated that focus was typically generated
99
on the glow effect before the participant focused on the textual
descriptions. The heat maps and eye-tracking videos depicted
that less focus was generated to read the entire textual
description of reoccurring issues (Figure 6, noted). This can be
viewed from the heat map as extensive focus (large red spots)
appear on the first parts of the descriptions, which fade to a
lighter green spot towards the end of the textual description.
Occasionally, participants found it sufficient to read the initial
part of the textual description to gain an understanding. This
was particularly identified for participants that devoted less
effort to read the textual descriptions associated with minor
issues. Participants were also more likely to notice an update of
textual descriptions when the Issues Window was less
crowded. The heat map depicts that the participants did not
regularly use the Category column to solve issues (Figure 6,
note c). Therefore, the textual descriptions of categories cannot
be proven to be as effective as the full textual descriptions for
each issue.
Participants also displayed trust in the validation technique, as
minor issues could be temporarily ignored and textual
descriptions could be read less thoroughly, as they progressed
through the task list, to gain an understanding of the issue. For
this reason, participants became more confident with the
validation technique during the evaluation. A deduction can be
drawn from these results to support the fact that learning of the
validation technique did occur.
5.1.3 Usefulness
The perceived usefulness of the validation technique was
analysed according to six metrics: a) to d) the perceived
usefulness ratings of the features: textual feedback, graphical
feedback, real-time validation and rule set categorisation, and
e) the perceived usefulness rating for increasing a participant’s
knowledge of EPC rules and f) the perceived usefulness for
preventing successive issues from occurring. Figure 7 indicates
the mean score ratings for each perspective and includes error
bars indicating the standard deviation (σ) from the mean. Three
frequency ranges of the Likert scale item responses were
statistically identified and calculated in order to classify mean
ratings as negative (1.0 ≥ µ < 2.6); neutral (2.6 ≥ µ ≤ 3.4) or
positive (3.4 > µ ≤ 5.0).
Figure 7. Usefulness of composite, real-time validation
features.
All six metrics had overall mean ratings in the positive range.
The graphical validation feature had the highest mean rating (µ
= 4.93). The second highest rated feature was “Enhanced
knowledge of EPC rules” (µ = 4.71). This supports the studies
[20, 35] showing that graphical feedback can enhance the
understanding of BP rules. Three features “Textual feedback”,
“Real-time validation” and “Prevents successive issues” all
received the same rating (µ = 4.57). The lowest rated metric
was for the rule set categorisation feature (µ = 4.36); however
this is still a positive rating. Since all of the overall metrics
were in the positive range, it can be deduced that the
implementation of the real-time, composite validation
technique in the EPC language in Visio was perceived as
useful by the participants.
5.1.4 System Usability Scale (SUS)
A mean overall SUS score below 60% is considered relatively
poor, while one over 80% is considered good [46]. The mean
overall SUS rating for the prototype (which implemented the
proposed validation technique) was 87%. It can be deduced
therefore that the participants were satisfied with the usability
of the validation technique (Figure 8). For demonstration
purposes, the ratings of negative worded statements are
depicted in a lighter colour. All of the positively worded
statements in the SUS questionnaire had overall mean ratings
in the positive range. The statement “I would use the
validation technique frequently” was the highest rated (µ =
4.64) SUS statement. The second highest rated (µ = 4.57) SUS
statement was, “The system was easy to use”.
Figure 8. Mean score per SUS statement.
The statement “I felt confident using the validation technique”
was also rated positively and had the third highest mean rating
(µ = 4.50). Four of the five (80%) of the negatively worded
statements had overall mean ratings in the positive range. The
statement “The system had a low complexity” was the highest
rated negatively worded statement (µ = 3.64). The second
highest rated negatively worded SUS statement was
“Inconsistency in the system is low” (µ = 3.64). From this it
can be deduced that the participants felt that the system was
not complex and was consistent.
The open-ended questions were analysed and grouped
according to themes. Comments from participants revealed that
textual feedback was thought to be easy-to-understand and
informative. The textual feedback could be easily read,
informing participants about the cause of the issue and possible
solutions. The graphical feedback was thought to be precise,
distinguishable, encouraging and visually appealing. The
precision of the glow effect around elements enabled
participants to focus on the exact element involved in the issue.
It was identified that the precision of the graphical effect
served as an encouragement to correct issues instantly.
Participants stated that the yellow and red glow effects were
visually appealing and assisted with distinguishing between
major and minor issues. The real-time validation feature
100
enabled the validation technique to be perceived as being
reliable, preventative, responsive, and natural. The real-time
validation enabled the textual and graphical validation features
to serve as reliable and natural guide to modellers that
informed them of any issues that existed within the created BP
model instantly. The qualitative data analysis revealed that
some participants perceived the re-occurring textual
descriptions as cumbersome to read and required a sufficient
description for the exact matching Logical Connectors.
Another limitation identified was that the participants
perceived the Issues Window to be overcrowded with textual
descriptions at times. In terms of graphical feedback, some
participants stated that the meaning of the yellow glow effect
was initially confusing, but could assume that it represented
minor issues.
6. CONCLUSIONS AND
RECOMMENDATIONS
In this paper a real-time, composite validation technique is
proposed which detects and to a certain extent prevents
modellers from incurring issues within the created BP model.
The validation technique provides textual descriptions of the
issues identified and possible solutions to correct the issue.
Furthermore, issues are detected in real-time and a distinction
is presented to the modeller between minor and major issues
using graphical methods. The validation technique was
demonstrated with the EPC modelling language and evaluated
by a group of BP modelling students.
The results of the field study evaluations were positive, since
the validation technique allowed modellers to successfully
detect and correct issues. Furthermore, modellers appeared to
quickly understand the modelling language’s feedback method.
The participants in the field study rated the graphical feedback
to be the most useful feature of the proposed validation
technique for detecting issues. The real-time feedback was also
rated positively and enabled participants to detect problems
before the modelling activity was completed. The textual
feedback was also rated positively and textual descriptions
were used to assist participants in understanding the underlying
causes and solutions of the issues. From this it can be deduced
that the proposed real-time, composite validation technique
significantly assisted modellers in terms of understanding the
rules of a BP modelling language. One limitation of the study
was that the prototype does not fully offer support to deactivate
all validation functions. In addition, the validation technique
was only evaluated with high-level BPs on a single modelling
canvas with novice modellers.
The results of this study reveal that HEIs can adopt a real-time,
composite validation technique to assist novice modellers in
their BP modelling courses. The proposed rule set
categorisation incorporated in the technique provides a
valuable contribution to the research field of real-time
validation techniques for BP modelling. Whilst the field study
was limited to a small group of BP modelling students at one
institution, this is a sufficient number of participants for
usability studies. The results of the field study provided a more
in-depth understanding of the challenges encountered by
modellers in drawing BP models and trying to understand the
rules of the modelling language. The benefit of implementing
the proposed validation technique is that it can assist modellers
with improving their knowledge of a particular BP modelling
language’s syntax rules, thereby improving the overall quality
of the created BP model. Thus, the real-time, composite
validation technique could contribute towards the attainment of
BP modelling competencies of IS graduates, and in this way
can reduce the gap that exists between industry needs and the
competencies of IS graduates. The validation technique can
also be used by modellers working in industry to improve BP
model accuracy and BP modelling project success. Future
research could be undertaken to investigate the effectiveness of
the proposed validation technique in other environments and
on other platforms such as mobile devices.
7. REFERENCES
[1] Curtiss, E. T. 2013. Business Process Management (BPM),
Cloud, Mobile, and Patterns: Market Shares, Strategy, and
Forecasts, Worldwide, 2013 to 2019.
[2] Recker, J. and Dreiling, A. 2007. Does It Matter Which
Process Modelling Language We Teach or Use ? An
Experimental Study on Understanding Process Modelling
Languages without Formal Education. In Proceedings of the
Australasian Conference of Information Systems
(Toowoomba, Australia, December5-7, 2007). 356–366.
[3] Recker, J. and Rosemann, M. 2009. Teaching Business
Process Modelling: Experiences and Recommendations.
Communications of the Association for Information Systems.
25, 32 (September 2009), 379–394.
[4] Ko, R. K. L. 2009. A Computer Scientist’s Introductory
Guide to Business Process Management (BPM). Crossroads.
15, 4 (June 2009), 4.
[5] van Greunen, D., van der Merwe, A. and Kotze, P. 2010.
Factors Influencing BPM Tools: The Influence on User
Experience and User Interfaces. International Journal of
Computing and ICT Research. 4, 1 (October 2010), 47–57.
[6] Harmon, P. 2009. Business Process Change: A Guide for
business Managers and BPM and Six Sigma Professionals,
Second Edition. Morgan Kaufmann Publishers, Burminham.
[7] Mendling, J. 2007. Detection and Prediction of Errors in
EPC Business Process Models. Doctoral Thesis. Vienna
University of Economics and Business Administration.
[8] Li, Y., Cao, B., Xu, L., Member, S., Yin, J, Deng, S. and
Yin, Y. 2014. An Efficient Recommendation Method for
Improving Business Process Modeling. IEEE Transactions
on Industrial Informatics. 10, 1 (February 2014), 502–513.
[9] van der Aalst, W. M. P. 2013. Business Process
Management : A Comprehensive Survey. ISRN Software
Engineering, 1,1 (September 2012), 1–37.
[10] Kühne, S., Kern, H., Gruhn, V. and Laue, R. 2010. Business
Process Modeling with Continuous Validation. Journal of
Software Maintenance and Evolution: Research and
Practice. 22, 7 (October 2010), 547–566.
[11] Seethamraju, R. 2012. Business Process Management: A
Missing Link in Business Education. Business Process
Management Journal. 18, 3 (2012), 532–547.
[12] Bandara, W., Chand, D. R., Chircu, A. M., Hintringer, S.,
Karagiannis, D., Recker, J., van Rensburg, A., Usoff, C. and
Welke, R. J. 2010. Business Process Management Education
in Academia: Status, Challenges, and Recommendations.
101
 Communications of the Association for Information Systems.
27, 1 (November 2010), 743–776.
[13] Gröner, G., Bošković, M., Silva Parreiras, F. and Gašević, D.
2013. Modeling and validation of business process families.
Information Systems. 38, 5 (July 2013), 709–726.
[14] Barjis, J. 2008. The Importance of Business Process
Modeling in Software Systems Design. Science of Computer
Programming. 71, 1 (March 2008), 73–78.
[15] Recker, J. 2010. Opportunities and Constraints : The Current
Struggle with BPMN. Business Process Management
Journal. 16, 1 (2010), 181–201.
[16] Reijers, H. and Mendling, J. 2011. A Study Into the Factors
That Influence the Understandability of Business Process
Models. IEEE Transactions on Systems, Man, and
Cybernetics - Part A: Systems and Humans. 41, 3 (May
2011), 449–462.
[17] Simon, H. 1996. The Sciences of the Artificial, Third
Edition. The MIT Press, Cambridge.
[18] Speck, A., Witt, S., Feja, S., Lotyzc, A. and Pulvermüller, E.
2011. Framework for Business Process Verification. In
Proceedings of the 14th International Conference on
Business Information Systems (Pozna´n, Poland, June 15-17
2011), Springer LNBIP, 50–61.
[19] Ly, L. T., Rinderle-ma, S. and Dadam, P. 2010. Design and
Verification of Instantiable Compliance Rule Graphs in
Process-aware Information Systems. In Advanced
Information Systems Engineering. Springer, Berlin-
Heidelberg, 9–23.
[20] Laue, R. and Awad, A. 2011. Visual Suggestions for
Improvements in Business Process Diagrams. Journal of
Visual Languages & Computing. 22, 5 (October 2011), 385–
399.
[21] Becker, J., Krugeler, M. and Rosemann, M. Process
Management: A Guide for the Design of Business Processes,
Second Edition. Springer, Heidelberg.
[22] Hallerbach, A., Bauer, T. and Reichert, M. 2010. Capturing
Variability in Business Process Models: The Provop
Approach. Journal of Software Maintenance and Evolution:
Research and Practice. 22, 7 (October 2010), 519–546.
[23] Rozman, T., Vajde-Horvat, R. and Draghici, A. 2011.
Business Process Management Certifications Overview and
Transfer of Innovation. In Proceedings of the Management,
Knowledge and Learning International Conference
(Romania, December, 2010).
[24] Recker, J. 2012. Modeling With Tools is Easier, Believe Me
- The Effects of Tool Functionality on Modeling Grammar
Usage Beliefs. Information Systems. 37, 3 (May 2012), 213–
226.
[25] Rosemann, M. 2006. Potential pitfalls of process modeling:
Part A. Business Process Management Journal, 12, 2 (2006)
, 249–254.
[26] Mendling, J., Reijers, H. A. and Cardoso, J. 2007. What
Makes Process Models Understandable?. In Business
Process Management. Springer, Berlin-Heidelberg, 48–63.
[27] Mendling, J., Reijers, H. and van der Aalst, W. M. P. 2010.
Seven Process Modeling Guidelines (7PMG). Information
and Software Technology. 52, 2 (January 2008), 127–136.
[28] Opocenská, K. and M. Kopecký. 2008. Incox–a language for
XML integrity constraints description. In Proceedings of the
Dateso 2008 Workshop (Cerna Ricka, Czech Republic, April
16-17, 2008), 1–12.
[29] Mendling, J. and van der Aalst, W. M. P. 2007.
Formalization and Verification of EPCs with OR-joins Based
on State and Context. In Proceedings of the 19th International
Conference, CAiSE 2007 (Trondheim, Norway, June 11-15,
2007), 439–453.
[30] Bögl, A., Schrefl, M., Pomberger, G. and Weber, N. 2009.
Semantic Annotation of EPC Models in Engineering
Domains to Facilitate an Automated Identification of
Common Modelling Practices. In Enterprise Information
Systems. Springer, Berlin-Heidelberg, 155–171.
[31] van Dongen, B. F. and Jansen-Vullers, M. H. 2005. EPC
Verification in the ARIS for MySAP Reference Model
Database. In Proceedings of the Business Process
Management with Event-Driven Process Chains Conference
(Hamburg, Germany, December 8-9, 2005), 24–40.
[32] Laue, R. and Mendling, J. 2010. Structuredness and its
Significance for Correctness of Process Models. Information
Systems and E-Business Management. 8, 3 (June 2010), 287–
307.
[33] van der Aalst, W. M. P. 2008. Challenges in Business
Process Analysis. In Enterprise Information Systems.
Springer, Berlin-Heidelberg, 27–42.
[34] Monk, E. and Wagner, B. 2009. Concepts in Enterprise
Resource Planning: International Edition, Third Edition.
Course Technology, Boston.
[35] Awad, A., Weidlich, M. and Weske, M. 2011. Visually
Specifying Compliance Rules and Explaining Their
Violations for Business Processes. Journal of Visual
Languages & Computing. 22, 1 (February 2011), 30–55.
[36] Knuplesch, D., Reichert, M., Ly, L. T., Kumar, A. and
Rinderle-Ma, S. 2013. Visual Modeling of Business Process
Compliance Rules with the Support of Multiple Perspectives.
In Conceptual Modeling. Springer, Berlin-Heidelberg, 106–
120.
[37] Semmelrodt, F., Knuplesch, D. and Reichert, M. 2014.
Modeling the Resource Perspective of Business Process
Compliance Rules with the Extended Compliance Rule
Graph. In Enterprise, Business-Process and Information
Systems Modeling. Springer, Berlin-Heidelberg, 48–63.
[38] Hopkins, C. R. 2013. Creating Custom Validation Rules in
Visio 2013. [Online]. Available:
http://blogs.msdn.com/b/chhopkin/archive/2013/01/03/creati
ng-custom-validation-rules-in-visio-2013.aspx. [Accessed:
06-Aug-2013].
[39] Wynn, M.T., Verbeek, H. M. W., van der Aalst, W. M. P.,
ter Hofstede, A. H. M. and Edmond, D. 2007. Business
Process Verification - Finally a Reality!. Business Process
Management Journal. 15, 1 (August 2007), 74–92.
102
[40] Gruhn, V. and Laue, R. 2007. Checking Properties of
Business Process Models with Logic Programming. In
Proceedings of the 5th International Workshop on
Modelling, Simulation, Verification and Validation of
Enterprise Information Systems, MSVVEIS-2007, In
conjunction with ICEIS 2007 (Funchal, Madeira, Portugal,
June, 2007), 84–93.
[41] van Dongen, B.F., Mendling, J. and van der Aalst, W. M. P.
2006. Structural Patterns for Soundness of Business Process
Models. In Proceedings of the Enterprise Distributed Object
Computing Conference, EDOC’06. 10th IEEE International
(Hong Kong, China, October 16-20, 2006), 116–128.
[42] Lewis, C. and Wharton, C. 1997. Cognitive Walkthroughs. In
Handbook of Human-Computer Interaction. Elsevier,
Amsterdam, 717–732.
[43] Tsakonas, G. and Papatheodorou, C. 2008. Exploring
Usefulness and Usability in the Evaluation of Open Access
Digital Libraries. Information Processing & Management.
44, 3 (May 2008), 1234–1250.
[44] Effinger, P., Seiz, S., Jogsch, N. and Eberhard, K. 2011.
Evaluating Single Features in Usability Tests for Business
Process Modeling Tools. In Proceedings of the Informatics
2011 conference (Berlin, Germany, October 4-7, 2011).
[45] Brooke, J. 1996. A Quick and Dirty Usability Scale. In
Usability Evaluation in Industry. Taylor & Francis Ltd,
London, 189–191.
[46] Tullis, T. and Albert, B. 2013. Measuring the User
Experiecne, Second edition. Morgan Kaufmann, Burlington,
138–139.
[47] Coltekin, A., Heil, B., Garlandini, S. and Fabrikant, S.I.
2009. Evaluating the Effectiveness of Interactive Map
Interface Designs: A Case Study Integrating Usability
Metrics with Eye-movement Analysis. Cartography and
Geographic Information Science. 36, 1 (August 2009), 5–17.
103